Accuracy for Mortality Prediction With Additive Biomarkers Including Interleukin-6 in Critically Ill Patients: A Multicenter Prospective Observational Study

Supplemental Digital Content is available in the text.

Abstract

Objectives:

Several inflammation markers have been reported to be associated with unfavorable clinical outcomes in critically ill patients. We aimed to elucidate whether serum interleukin-6 concentration considered with Sequential Organ Failure Assessment score can better predict mortality in critically ill patients.

Design:

A prospective observational study.

Setting:

Five university hospitals in 2016–2018.

Patients:

Critically ill adult patients who met greater than or equal to two systemic inflammatory response syndrome criteria at admission were included, and those who died or were discharged within 48 hours were excluded.

Interventions:

Inflammatory biomarkers including interleukin (interleukin)–6, -8, and -10; tumor necrosis factor–α; C-reactive protein; and procalcitonin were blindly measured daily for 3 days. Area under the receiver operating characteristic curve for Sequential Organ Failure Assessment score at day 2 according to 28-day mortality was calculated as baseline. Combination models of Sequential Organ Failure Assessment score and additional biomarkers were developed using logistic regression, and area under the receiver operating characteristic curve calculated in each model was compared with the baseline.

Measurements and Main Results:

Among 161 patients included in the study, 18 (11.2%) did not survive at day 28. Univariate analysis for each biomarker identified that the interleukin-6 (days 1–3), interleukin-8 (days 0–3), and interleukin-10 (days 1–3) were higher in nonsurvivors than in survivors. Analyses of 28-day mortality prediction by a single biomarker showed interleukin-6, -8, and -10 at days 1–3 had a significant discrimination power, and the interleukin-6 at day 3 had the highest area under the receiver operating characteristic curve (0.766 [0.656–0.876]). The baseline area under the receiver operating characteristic curve for Sequential Organ Failure Assessment score predicting 28-day mortality was 0.776 (0.672–0.880). The combination model using additional interleukin-6 at day 3 had higher area under the receiver operating characteristic curve than baseline (area under the receiver operating characteristic curve = 0.844, area under the receiver operating characteristic curve improvement = 0.068 [0.002–0.133]), whereas other biomarkers did not improve accuracy in predicting 28-day mortality.

Conclusions:

Accuracy for 28-day mortality prediction was improved by adding serum interleukin-6 concentration to Sequential Organ Failure Assessment score.

Most critically ill patients experience significant morbidity or mortality despite receiving intensive care by a multidisciplinary medial team (1, 2). The ability to accurately predict mortality for critically ill patients can help healthcare providers optimize care and provide valuable information for patients and caregivers (3, 4). Various prognostic indices have been proposed, and several scoring systems to calculate the severity of organ dysfunction have been externally validated and are globally used (4–7).

The Sequential Organ Failure Assessment (SOFA) score is one of the well-accepted scales to quantify organ function and predict in-hospital mortality (4). The use of the SOFA score to assess patient status changes over time in the ICU has been validated to represent better mortality prediction (8). Both the European Society of Intensive Care Medicine and Society of Intensive Care Medicine have proposed that acute changes in the SOFA score may be used to define organ dysfunction among patients with infection and to diagnose sepsis (9, 10). However, the SOFA score, as well as other prognostic scales such as Acute Physiology and Chronic Health Evaluation (APACHE) II and Simplified Acute Physiology Score, does not include biological markers correlated with systemic inflammation as a variable for score calculation (5, 6, 8). Notably, such inflammation markers, including cytokines/chemokines and acute phase proteins, are associated with unfavorable clinical outcomes (11–13).

Interleukin (IL)–6 is a cytokine released by immune cells and plays a role in systemic inflammatory changes caused by infection or tissue injury (14). Several studies have reported that serum IL-6 concentration is associated with disease severity, adverse events, and overall mortality among patients with sepsis, burn and trauma injury, cardiovascular diseases, and hemodialysis (15–19). However, the diagnostic accuracy of IL-6 for sepsis has been inconclusive, despite extensive analysis (13), and the clinical feasibility of IL-6 for mortality prediction in critically ill patients remains unclear (11, 20, 21). This study sought to elucidate whether serum IL-6 concentration can be a valid component of the SOFA scoring system to better predict mortality in critically ill patients. The hypothesis is that the addition of serum IL-6 concentration to SOFA score provides better mortality prediction in this population.

MATERIALS AND METHODS

Study Design and Setting

This prospective observational study used emergency department (ED) and ICU data from five university hospitals in Japan. All hospitals received individual local institutional review board approval for conducting research on human subjects. The Ethics Committee at the Keio University School of Medicine approved this study (approval number 16-03-007). Informed consent was obtained from all patients for participation included in the study.

Study Population

The study enrolled critically ill patients admitted to the participating centers between September 2016 and September 2018. The inclusion criteria were as follows: age greater than or equal to 20 years, greater than or equal to two systemic inflammatory response syndrome (SIRS) criteria of the American College of Chest Physicians/Society of Critical Care Medicine on ED/ICU admission (22), and expected ICU stay greater than or equal to 48 hours. In addition to meeting these criteria, burn patients with a burn index (full-thickness burn area + 1/2 partial-thickness burn area) of greater than or equal to 15 and trauma patients with injuries in greater than or equal to two body regions on the abbreviated injury scale coding system and with an Injury Severity Score of greater than or equal to 10 were included. The exclusion criteria were as follows: current medications that affect serum IL-6 concentration (e.g., corticosteroids, immunosuppressants) within 1 week before study inclusion; discharge or death within 48 hours after admission; deviation from study protocol for biomarker tests and SOFA score calculation (e.g., blood sample was not obtained per protocol, SOFA score was not calculated); HIV infection; pregnancy; and any other condition precluding suitability for enrollment in the investigators’ opinion.

Data Collection and Definitions

The following patient information were included: demographic characteristics, admission source, comorbidities, medications administered within 1 week before study inclusion, etiology at admission, episode of cardiac arrest before study inclusion, presence of hemodynamic instability defined as vasopressor requirement or persistent hypotension despite fluid resuscitation, and any ED/ICU treatments.

Blood samples were obtained within 6 hours after ED/ICU admission (day 0) and the next morning (day 1). Blood tests were then performed daily from days 2–3 and as needed until 7 days after admission. These inflammatory biological markers were as follows: C-reactive protein (CRP), IL-6, IL-8, IL-10, tumor necrosis factor (TNF)–α, and procalcitonin. Serum CRP concentration was measured immediately with commercially available assays at each hospital; ILs, TNF-α, and procalcitonin were measured blindly to treating physicians at an outside facility after serum samples were frozen and stored at –20°C (IL-6, procalcitonin; Roche Diagnostics, Mannheim, Germany and IL-8, IL-10; BioSource Europe, Nivelles, Belgium and TNF-α; R&D Systems, Minneapolis, MN).

Arterial blood gas analysis and other blood tests for calculating each SOFA score component were performed at each hospital simultaneously with blood sampling for biological markers. The SOFA score was recorded daily until day 3 and as needed until day 7; APACHE II score was also calculated on ED/ICU admission.

Outcome Measures

The primary outcome was 28-day all-cause mortality. The secondary outcome was ICU-free days, defined as the number of days alive and out of the ICU between admission and day 28.

Statistical Analysis

To assess improved accuracy for mortality prediction by adding a biomarker test to SOFA score, a baseline model was developed using logistic regression analysis to predict 28-day mortality, in which the SOFA score at day 2 was chosen as a sole explanatory variable. The day 2 score was chosen because the original SOFA score validation study showed that the SOFA score at 48 hours post admission had a higher discrimination power for predicting ICU mortality than the score at admission or difference in scores between admission and 48 hours later (8). To identify the best time for each biomarker to predict mortality, receiver operating characteristic (ROC) curves were drawn for serum concentration of each biomarker at days 0–3 based on 28-day mortality, and area under the ROC curve (AUROC) was calculated. The day with the highest AUROC was considered as the best time point for each biomarker.

Logistic regression analyses to predict 28-day mortality were performed again to derive the linear combination of the baseline model (day 2 SOFA score) and an additional biomarker as measured based on the best time point just described. As biomarkers were expected to be skewedly distributed, they were log transformed before analyses (11, 15, 16). Some biomarkers, including ILs, were analyzed with sex as suggested in other studies (23, 24). The ROC curves were drawn, and AUROC was compared between the baseline model and combination model developed with the additional biomarker. Improvement of AUROC from baseline was shown with 95% CI. The sensitivity and specificity of each model were also obtained at a best cut off point defined as the Youden index (25). To assess optimism of the combination model using the additional biomarker, a corrected AUROC was calculated with bootstrap analysis (resampling the model 1,000 times) (26). The combination models with additional biomarkers were also examined in linear regression analyses to predict ICU-free days. The clinical applicability of biomarker was then assessed by calculating observed 28-day mortalities in subgroups classified as SOFA score and the biomarker dichotomized at the median value.

Descriptive statistics are presented as the mean (sd), median (interquartile range), or number (percentage). No imputation was used to estimate missing data. The improvement of predictive ability for mortality by adding biomarkers was unclear before the study, and sample size estimation was not performed for the main analysis. Sample size estimation for ROC analysis in which an AUROC of 0.7 was expected for an event with an incident rate of 15% indicated that 150 cases were needed with a power of 80% and an α error of 0.05 (27). Results were compared using Mann-Whitney U tests, chi-square tests, or Fisher exact tests, as appropriate. For testing all hypotheses, a two-sided α threshold of 0.05 was considered statistically significant. All statistical analyses were conducted using SPSS, Version 26.0 (IBM, Armonk, NY), and R Version 4.0.2 (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Among 199 patients who met all inclusion criteria, the following patients were excluded: four who took corticosteroids before inclusion, seven who died within 48 hours post admission, and 21 who deviated from the protocol for biomarker tests and SOFA score calculation (Fig. 1). Among the 161 eligible patients, 18 (11.2%) did not survive at day 28. Table 1 shows the characteristics of the participant population. The most common etiology at admission was infectious disease (105 [65.2%]); greater than 50% required mechanical ventilation (86 [53.4%]), and about 33% underwent renal replacement therapy (51 [31.7%]).

Figure 1.

Study flow diagram. Among 199 patients who met the inclusion criteria, the following patients were excluded: four who took corticosteroids before inclusion, seven who died within 48 hr post admission, and 21 who deviated from the study protocol for biomarker tests and Sequential Organ Failure Assessment score calculation. Among 161 patients eligible for the analysis, 18 (11.2%) did not survive at day 28.

TABLE 1.

Characteristics of Study Population

	Study Population
Case, n	161
Age, years, mean (sd)	69 (15)
Sex, male, n (%)	101 (62.7)
Type of admission, n (%)
Medical, infectious disease	105 (65.2)
Medical, noninfectious disease	21 (13.0)
Surgical, trauma/burn	23 (14.3)
Surgical, nontrauma/burn	12 (7.5)
Comorbidity, n (%)
Cerebrovascular disease	23 (14.3)
Diabetes	23 (14.3)
Cardiovascular disease	7 (4.3)
Chronic lung disease	9 (5.6)
Chronic kidney disease	15 (9.3)
Liver disease	10 (6.2)
Acute Physiology and Chronic Health Evaluation II score, median (IQR)	25 (19–34)
Sequential Organ Failure Assessment score, median (IQR)
Day 0	7 (3–11)
Day 1	8 (4–12)
Day 2	7 (3–11)
Cardiac arrest prior to admission, n (%)	11 (6.8)
Hemodynamic instabilitya, n (%)	66 (41.0)
Mechanical ventilation, n (%)	86 (53.4)
Renal replacement therapy, n (%)	51 (31.7)
Length of ICU stay, d, median (IQR)	9 (5–17)
Mortality, n (%)
7 d mortality	5 (3.1)
28 d mortality	18 (11.2)

IQR = interquartile range.

^aHemodynamic instability was defined as vasopressor requirement or persistent hypotension despite fluid resuscitation.

Univariate analyses for each biomarker identified that the median IL-6 concentration at days 1–3 was higher among nonsurvivors than among survivors at day 28 (Fig. 2) (Table S1, http://links.lww.com/CCX/A575). Similarly, the median IL-8 serum concentration at days 0–3 and the median IL-10 concentration at days 1–3 were higher among nonsurvivors than among survivors at day 28. Conversely, serum CRP, procalcitonin, and TNF-α concentrations were comparable between nonsurvivors and survivors until day 3. Mortality prediction by a single biomarker on ROC analyses showed that the serum IL-6, IL-8, and IL-10 concentrations at days 1–3 had a significant discrimination power to predict 28-day mortality. The best time point for mortality prediction was day 3 for IL-6, day 1 for IL-8, and day 2 for IL-10. The IL-6 concentration at day 3 had the highest discrimination power (AUROC = 0.766; 95% CI = 0.656–0.876) (Table S1, http://links.lww.com/CCX/A575).

Figure 2.

Biomarkers in survivors and nonsurvivors. Univariate analyses for each biomarker identified that the median interleukin (IL)–6 concentration at days 1–3, IL-8 at days 0–3, and IL-10 at days 1–3 were higher in patients who did not survive at day 28 after admission compared with survivors. Conversely, serum C-reactive protein (CRP), procalcitonin (PCT), and tumor necrosis factor (TNF)–α concentrations were comparable between survivors and nonsurvivors until day 3. The IL-6 concentration at day 3 had the highest discrimination power.

Accuracy in predicting 28-day mortality with SOFA score at day 2 was assessed as the baseline model using a logistic regression analysis (AUROC = 0.776; 95% CI = 0.672–0.880). Multivariate logistic regression analyses were performed to derive combination models with SOFA score and additional biomarkers (IL-6, CRP, procalcitonin, TNF-α at day 3; IL-8 at day 1; IL-10 at day 2). On AUROC comparison, the combination model with additional serum IL-6 concentration at day 3 was significantly higher than the baseline model using only the SOFA score (AUROC = 0.844; AUROC improvement = 0.068; 95% CI = 0.002–0.133) (Table 2) (Fig. S1, http://links.lww.com/CCX/A574). Conversely, other combination models using CRP, procalcitonin, IL-8, IL-10, and TNF-α had comparable discrimination power with the baseline model. Furthermore, a combination model using all of IL-6, IL-8, and IL-10 showed similar comparable discrimination power to the baseline model. Improvement of accuracy for mortality prediction by adding serum IL-6 concentration to the SOFA score was maintained in bootstrap analysis that estimated optimism of the combination model (corrected AUROC = 0.815). The serum IL-6 concentrations were also associated with decreased ICU-free days in the combination model (coefficient = 2.4 d decrease; 95% CI = 0.1–4.7 d decrease; p = 0.042) (Table S2, http://links.lww.com/CCX/A576), whereas models with other biomarkers were not.

TABLE 2.

Accuracy for Mortality Prediction With Additive Biomarkers

	AUROC	Improvement of AUROC (95% CI)	Optimism	Corrected AUROC	Sensitivity, %	Specificity, %
SOFA score at day 2	0.776				82.4	70.1
SOFA with IL-6	0.844	0.068 (0.002 to 0.133)	0.029	0.815	94.1	64.2
SOFA with C-reactive protein	0.783	0.008 (–0.038 to 0.055)	0.020	0.763	64.7	86.4
SOFA with procalcitonin	0.776	0.000 (–0.012 to 0.012)	0.008	0.769	82.4	71.6
SOFA with IL-8	0.799	0.031 (–0.028 to 0.089)	0.034	0.765	82.4	74.4
SOFA with IL-10	0.828	0.061 (–0.057 to 0.179)	0.040	0.788	81.8	73.5
SOFA with tumor necrosis factor–α	0.773	–0.002 (–0.010 to 0.007)	0.015	0.758	82.4	72.2

AUROC = area under the receiver operating characteristic curve, IL = interleukin, PCT = procalcitonin, SOFA = Sequential Organ Failure Assessment.

Logit-transformed predictive mortality rate was calculated in each model as follows (Log-transformed values of biomarker levels were entered in the calculation): SOFA score at day 2 (baseline model): SOFA score at day 2 × 0.190–3.871; SOFA with IL-6: SOFA score at day 2 × 0.102 + IL-6 at day 3 × 1.226 + male × 1.011–6.381; SOFA with C-reactive protein: SOFA score at day 2 × 0.211–CRP at day 3 × 0.867–3.167; SOFA with PCT: SOFA score at day 2 × 0.181 + PCT at day 3 × 0.134–0.3862; SOFA with IL-8: SOFA score at day 2 × 0.146 + IL-8 at day 1 × 0.471 + male × 0.834–5.000; SOFA with IL-10: SOFA score at day 2 × 0.155 + IL-10 at day 2 × 2.180 + male × 0.372–6.096; SOFA with tumor necrosis factor (TNF)–α: SOFA score at day 2 × 0.189–TNF-α at day 3 × 0.023–3.856.

On subgroups analysis of observed 28-day mortalities classified with SOFA score at day 2 and the median serum IL-6 concentration at day 3, low IL-6 concentration (≤ 74 pg/dL) was associated with mortality less than or equal to 10% among patients with SOFA scores less than or equal to 11 (Fig. 3). For high IL-6 concentrations (> 74 pg/dL), the mortality rate averaged greater than 20% in patients with SOFA scores of 8–11 and 25.8% with SOFA scores of greater than 11.

Figure 3.

Mortalities in subgroups classified by with Sequential Organ Failure Assessment (SOFA) score and interleukin-6 concentration. Observed 28 d mortalities were calculated in the subgroups that were divided with SOFA score at day 2 and serum interleukin-6 concentration at day 3. Low interleukin-6 concentrations (≤ 74 pg/dL) were associated with mortality rate of less than or equal to 10% among patients with SOFA score less than or equal to 11. Conversely, when interleukin-6 concentrations were high (> 74 pg/dL), the mortality rate averaged greater than 20% in patients with a SOFA of 8–11% and 25.8% in patients with a sequential organ failure assessment score of greater than 11.

DISCUSSION

This multicenter observational study examined the accuracy of mortality prediction with additive biomarkers and found that serum IL-6 concentration had the highest discrimination power to predict 28-day mortality in critically ill patients. The improvement of accuracy for mortality prediction by adding serum IL-6 concentration at day 3 to SOFA score was identified as increased AUROC from baseline that used only SOFA score. Higher serum IL-6 concentration was also associated with longer ICU stay when used as an additional inflammation biomarker with SOFA score.

Several pathophysiologic mechanisms for the high prediction ability of IL-6 may be considered based on previous studies (14, 28, 29). IL-6 activates target genes involved in host defense mechanisms and is a major player in pro- and anti-inflammatory responses to infection and injury (28). Because IL-6 is synthesized in a local lesion in the initial stage of inflammation or infection and then moves to the liver where acute phase proteins such as CRP are rapidly induced, elevation of serum IL-6 concentration usually precedes elevation of other inflammatory biomarkers and also clinical signs such as fever (14, 29). In addition, removal of the inflammation source is quickly followed by cessation of IL-6–mediated cascade and degradation of IL-6 messenger RNA (29). Therefore, alteration of serum IL-6 concentration closely reflects the degree or severity of systemic inflammation. Furthermore, persistent IL-6 production with high serum concentration has been identified in patients with severe SIRS who experience cytokine storm (30), suggesting that dysregulated IL-6 abnormally accelerates inflammatory pathways and organ insult (14, 30). Considering that serum IL-6 concentrations at day 3 versus days 0–2 had the highest discrimination in predicting mortality, persistent systemic inflammation would be detected by high IL-6 concentration at day 3 among critically ill patients.

Although several mortality prediction models have been developed, most of them used clinical and physiologic variables at admission or within the first 24 hours in the ICU (5–7). Although useful to predict early consequences such as ICU adverse events, ignoring deteriorations and improvements of patient status as a result of initial responses to treatment limit projecting later clinical outcomes, such as 28-day mortality. Furthermore, some prediction models involving IL-6 concentrations did not include clinical variables (11, 12, 19). Given that the use of both clinical and biological variables would better capture patient status, the SOFA score at day 2 and the serum IL-6 concentration at day 3, representing the patient condition altered by early treatment, would be feasible to predict 28-day mortality. A very high sensitivity (94.1%) for mortality was detected with the combination model using SOFA score and serum IL-6 concentration.

The clinical applicability of serum IL-6 concentration was assessed, and survival at 28 day will likely be predicted in patients with a SOFA score of less than 7 at day 2 and IL-6 concentration of less than or equal to 74 pg/dL at day 3. This point is significant because a retrospective study reported patients with initial or highest SOFA score of 6–7 had an ICU mortality of about 20% (8). Among patients with SOFA scores of 8–11, 28-day mortality almost doubled when the IL-6 concentration was greater than 74 pg/dL at day 3, suggesting such persistent elevation of serum IL-6 concentration would warn of unfavorable clinical consequences.

The results of this study must be interpreted in the context of the design. First, neither did it develop a new scoring scale using serum IL-6 concentration nor did it elucidate a cut off value of IL-6 concentration to predict 28-day mortality. Although results suggested that adding IL-6 concentration to SOFA score would be valuable to develop a better prediction system and that patients with an IL-6 concentration of less than or equal to 74 pg/dL at day 3 would be expected to survive even with a SOFA score of less than or equal to 7, more cases are required to derive and validate a new scale using IL-6.

Another limitation was that the study population included patients with various diseases. Given that another biomarker, such as procalcitonin, was extensively examined among patients with bacterial infection (31), the mortality of such population would be better predicted by procalcitonin rather than by IL-6 concentration. Although the small sample size precluded subgroup analyses, a disease-specific prediction model should be further examined.

Finally, the study did not collect data regarding long-term mortality or functional outcomes, including physical impairment and cognitive function, which may be more important than 28-day mortality among critically ill patients. Although the serum IL-6 concentration at day 3 was associated with length of ICU stay, further study on long-term and/or functional outcomes should be performed.

CONCLUSIONS

In this multicenter observational study, accuracy for 28-day mortality prediction was improved by adding serum IL-6 concentration to the SOFA score. Persistent high IL-6 concentration until 3 days after admission would predict longer ICU stay and higher probability of mortality at day 28. Further study is needed to develop a new scoring scale using both SOFA score and serum IL-6 concentrations.