Sequential Organ Failure Assessment (SOFA) Score for predicting mortality in patients with sepsis in Vietnamese intensive care units: a multicentre, cross-sectional study

Abstract

Objectives

To compare the accuracy of the Sequential Organ Failure Assessment (SOFA) and Acute Physiology and Chronic Health Evaluation II (APACHE II) Scores in predicting mortality among intensive care unit (ICU) patients with sepsis in a low-income and middle-income country.

Design

A multicentre, cross-sectional study.

Setting

A total of 15 adult ICUs throughout Vietnam.

Participants

We included all patients aged ≥18 years who were admitted to ICUs for sepsis and who were still in ICUs from 00:00 to 23:59 of the specified study days (ie, 9 January, 3 April, 3 July and 9 October of the year 2019).

Primary and secondary outcome measures

The primary outcome was hospital all-cause mortality (hospital mortality). We also defined the secondary outcome as all-cause deaths in the ICU (ICU mortality).

Results

Of 252 patients, 40.1% died in hospitals, and 33.3% died in ICUs. SOFA Score (areas under the receiver operating characteristic curve (AUROC): 0.688 (95% CI 0.618 to 0.758); cut-off value≥7.5; P_AUROC<0.001) and APACHE II Score (AUROC: 0.689 (95% CI 0.622 to 0.756); cut-off value ≥20.5; P_AUROC<0.001) both had a poor discriminatory ability for predicting hospital mortality. However, the discriminatory ability for predicting ICU mortality of SOFA (AUROC: 0.713 (95% CI 0.643 to 0.783); cut-off value≥9.5; P_AUROC<0.001) was fair and was better than that of APACHE II Score (AUROC: 0.672 (95% CI 0.603 to 0.742); cut-off value≥18.5; P_AUROC<0.001). A SOFA Score≥8 (adjusted OR (AOR): 2.717; 95% CI 1.371 to 5.382) and an APACHE II Score≥21 (AOR: 2.668; 95% CI 1.338 to 5.321) were independently associated with an increased risk of hospital mortality. Additionally, a SOFA Score≥10 (AOR: 2.194; 95% CI 1.017 to 4.735) was an independent predictor of ICU mortality, in contrast to an APACHE II Score≥19, for which this role did not.

Conclusions

In this study, SOFA and APACHE II Scores were worthwhile in predicting mortality among ICU patients with sepsis. However, due to better discrimination for predicting ICU mortality, the SOFA Score was preferable to the APACHE II Score in predicting mortality.

Clinical trials registry – India: CTRI/2019/01/016898.

Strengths and limitations of this study.

• An advantage of the present study was data from multi centres, which had little missing data.

• Due to the absence of a national registry of intensive care units (ICUs) to allow systematic recruitment of units, we used a snowball method to identify suitable units, which might have led to the selection of centres with a greater interest in sepsis management.

• Due to the study’s real-world nature, we did not make a protocol for microbiological investigations. Moreover, we mainly evaluated resources used in ICUs; therefore, the data detailing the point-of-care testing and life-sustaining treatments were not available. Additionally, to improve the feasibility of conducting the study in busy ICUs, we opted not to collect data on antibiotic resistance and appropriateness.

• Due to our independent variables (eg, Sequential Organ Failure Assessment Score that was greater than or equal to the cut-off value) that might be associated with primary outcome only measured on ICU admission, the mixed-effects logistic regression model could not be used to predict discrete outcome variables measured at two different times, that is, inside and outside the ICU settings.

• Although the sample size was large enough, the CI was slightly wide (±6.03%), which might influence the normal distribution of the sample.

Introduction

Sepsis is a clinical syndrome which has physiological, biological and biochemical abnormalities caused by a dysregulated host response to infection and is a critical global health problem.^{1 2} Sepsis is the most common cause of in-hospital deaths, with most of the burden in low-income and middle-income countries (LMICs), and extracts a high economic and social cost;^3–5 mortality rates remain high at 30%–45% and contribute to as much as 20% of all deaths worldwide.2 4 6 7 There is no reference standard that allows easy, accurate diagnosis and prognosis of sepsis.^{1 8} Although the 1991 International Consensus Definition Task Force proposed the systemic inflammatory response syndrome criteria to identify patients with a septic host response,⁹ these criteria do not measure whether the response is injurious, and their utility is limited.^{1 8}

The Acute Physiology and Chronic Health Evaluation II (APACHE II) Score was originally developed for critically ill patients in intensive care units (ICUs).¹⁰ It has 12 physiological measures and extra points based on age and the presence of chronic disease.¹⁰ The APACHE II Score was shown to have good prognostic value in acutely ill or surgical patients.^{10 11} However, some limitations of the APACHE II Score are that (1) It is complex and cumbersome to use, (2) It does not differentiate between the sterile and infected necrosis, and finally, (3) It has a poor predictive value at 24 hours.¹²

In 2016, the Sepsis-3 Task Force proposed that for patients with suspected infection, an increase of 2 points or more in the Sequential Organ Failure Assessment (SOFA) Score could serve as clinical criteria for sepsis,¹ and the consensus has not changed since then.¹³ This approach was justified based on content validity (SOFA reflects the facets of organ dysfunction) and predictive validity (the proposed criteria predict downstream events associated with the condition of interest).^14–17 However, the validity of this score was mainly derived from critically ill patients with suspected sepsis by interrogating over a million ICU electronic health record encounters from ICUs in high-income countries (HICs).^{1 17 18} Moreover, the patients, pathogens and clinical capacity to manage sepsis differ considerably between HIC and LMIC settings.⁷ Therefore, it’s still unclear whether this score could be applied to different types of infections, locations within the hospital and countries.

Vietnam is an LMIC, ranked fifteenth in the world and third in South-East Asia by population with 96.462 million people.¹⁹ Vietnam is also a hot spot for emerging infectious diseases in South-East Asia, including SARS-CoV-1,²⁰ avian influenza A(H5N1)^{21 22} and the ongoing global COVID-19 outbreaks.^{23 24} Additionally, severe dengue,²⁵ Streptococcus suis infection,²⁶ malaria²⁷ and increased antibiotic resistance are other major causes of sepsis in ICUs across Vietnam.^{28 29} Despite its recent economic growth spurt,³⁰ Vietnam is still struggling to provide either enough resources or adequate diagnostic, prognostic and treatment strategies for patients with sepsis in both local and central settings.^{31 32} In addition, within the healthcare system in Vietnam, central hospitals are responsible for receiving patients who have difficulties being treated in local hospital settings.³³ Therefore, the diagnosis, prognosis and initiation of treatment for patients with sepsis are often delayed.

In resource-limited settings, the early identification of infected patients who may go on to develop sepsis or who may be at risk of death from sepsis using accurate scoring systems is a way to decrease sepsis-associated mortality. Therefore, this study aimed to investigate the mortality rate and compare the accuracy of the SOFA Score and the APACHE II Score in predicting mortality in ICU patients with sepsis in Vietnam.

Methods

Source of data

This multicentre observational, cross-sectional, point prevalence study is part of the Management of Severe sepsis in Asia’s Intensive Care unitS (MOSAICS) II Study,^34–37 which enrolled patients on 9 January (Winter), 3 April (Spring), 3 July (Summer) and 9 October (Autumn) of the year 2019. All patients received a follow-up till hospital discharge, death in the ICU/hospital or up to 90 days postenrolment, whichever was earliest. In this study, we only used data from Vietnam. A total of 15 adult ICUs (excluding predominantly neurosurgical, coronary, and cardiothoracic ICUs) participated in the MOSAICS II study from 14 hospitals, of which 5 are central, and 9 are provincial, district, or private hospitals throughout Vietnam. Each ICU had one or two representatives who were part of the local study team and the MOSAICS II Study group, as shown in eAppendix 2 of a previously published paper.³⁶ Participation was voluntary and unfunded.

Participants

All patients admitted to participating ICUs on 1 of the 4 days (ie, 9 January, 3 April, 3 July and 9 October, 2019) which represented the different seasons of the year 2019 were screened for eligibility. We included all patients, aged ≥18 years, who were admitted to the ICUs for sepsis, and who were still in the ICUs from 00:00 to 23:59 of the study days. We defined sepsis as infection with a SOFA Score of 2 points or more from baseline (assumed to be 0 for patients without prior organ dysfunction).¹

Data collection

We used a standardised classification and case record form (CRF) to collect data on common variables as shown in online supplemental file 1. The data dictionary of the MOSAICS II Study is available as an online supplement of previously published papers.^{35 36} Data were entered by the representatives of the participating hospitals into the database of the MOSAICS II Study via the password-protected online CRFs. We checked the data for implausible outliers and missing fields and contacted ICU representatives for clarification. We then merged the data sets for the 14 hospitals.

Outcome measures

The primary outcome was hospital all-cause mortality (hospital mortality). We also defined the secondary outcomes as all-cause deaths in the ICU (ICU mortality) and the ICU and hospital lengths of stay (LOS).

Predictor measures

We defined exposure variables as the SOFA and the APACHE II Scores.^{10 14} All data elements required for calculating the SOFA Score at the time of ICU admission and the APACHE II Score over the first 24 hours of ICU admission were prospectively collected on a CRF and entered into a database via the online CRF for later analysis.

We determined confounding factors as the variables of hospital and ICU characteristics collected on a questionnaire by representatives before patient enrolment, as shown in online supplemental file 2. We also determined confounding factors as variables collected on a CRF by investigators. The CRF contained four sections which is available in online supplemental file 1. The first section focused on baseline characteristics (demographics, documented comorbidities and details of admission). The second section comprised of vital signs on ICU admission, laboratory parameters, site of infection and microbiology. Only microorganisms detected via all cultures, serology, molecular and histological investigations, and deemed to be true pathogens rather than commensals or contaminants were recorded. The third section captured the timing of sepsis bundle elements referencing time zero, determined as follows: (A) Time of triage in the emergency department (ED) for those presenting with sepsis to the ED; (B) Time of clinical documentation of deterioration in the general wards or other non-ED areas for those who developed sepsis after hospital admission; (C) Time of ICU admission for those in which (A) or (B) could not be determined from the clinical documentation. The bundle elements were based on the Surviving Sepsis Campaign’s 2018 update: antibiotics administration, blood cultures, lactate measurement, fluid administration (amount of fluids administered in the first and third hours from time zero) and vasopressor initiation.³⁸ The fourth section concerned life-sustaining treatments provided during the ICU stay.

Sample size

In the present study, hospital mortality served as the primary outcome. We, therefore, used the formula to determine the minimal sample size for estimating a population proportion with a confidence level of 95%, a CI (margin of error) of 6.03% and an assumed population proportion of 61.0%, based on the hospital mortality rate (61.0%) of our cohort reported in a previously published study.³⁹ Therefore, we should have at least 252 patients in our sample. Because of this, our sample size was sufficient and reflected a normal distribution.

${\displaystyle n=\frac{z^{2}x\hat{p}(1-\hat{p})}{{\epsilon }^2}}$

where:

z is the z score (z score for a 95% confidence level is 1.96)

ε is the margin of error (ε for a CI of ±6.03% is 0.0603)

${\displaystyle \hat{p}}$ is the population proportion ( ${\displaystyle \hat{p}}$ for a population proportion of 61.0% is 0.61)

n is the sample size

Statistical analyses

We used IBM SPSS Statistics V.22.0 (IBM Corp, Armonk, New York, USA) for data analysis. We report data as numbers (no.) and percentages (%) for categorical variables and medians and IQRs or means and SDs for continuous variables. Comparisons were made between survival and death in the hospital and ICU for each variable, using the χ² test or Fisher’s exact test for categorical variables and the Mann-Whitney U test, Kruskal-Wallis test, one-way analysis of variance for continuous variables.

Receiver operating characteristic (ROC) curves were plotted and the areas under the ROC curve (AUROC) were calculated to determine the discriminatory ability of the SOFA and APACHE II Scores for deaths in the hospital and ICU. The cut-off value of the SOFA and the APACHE II Scores was determined by the ROC curve analysis and defined as the cut-off point with the maximum value of Youden’s Index (ie, sensitivity+specificity – 1). Based on the cut-off value of the scores, we assigned the patients to two groups: either a score that was less than the cut-off value or a score that was greater than or equal to the cut-off value.

We assessed factors associated with death in the hospital using logistic regression analysis. To reduce the number of predictors and the multicollinearity issue and resolve the overfitting, we used different ways to select variables as follows: (A) We put all variables (including exposure and confounding factors) of hospital and ICU characteristics, baseline characteristics, clinical and laboratory characteristics, and treatments into the univariable logistic regression model; (B) We selected variables if the value of p was <0.05 in the univariable logistic regression analysis between survival and death in the hospital, as well as those that are clinically crucial to put in the multivariable logistic regression model. These variables included university affiliation, training programme in ICU, documented comorbidities (ie, cardiovascular disease, chronic neurological disease), the severity of illness (ie, SOFA and APACHE II Scores that were greater than or equal to the cut-off value), sites of infection (ie, urinary tract, abdominal, skin or cutaneous sites), pathogens detection (ie, no pathogens detected, Gram-negative bacteria), completion of the 1-hour or 3-hour sepsis bundle of care, completion of the initial administration of antibiotics within 1 hour or 3 hours, respiratory support (ie, mechanical ventilation (MV), high-flow nasal oxygen), and additional ICU support (ie, vasopressors/inotropes, renal replacement therapy (RRT), red blood cell transfusion, platelet transfusion, fresh frozen plasma transfusion, surgical source control, and non-surgical source control). Using a stepwise backward elimination method, we started with the full multivariable logistic regression model that included the selected variables. This method then deleted the variables stepwise from the full model until all remaining variables were independently associated with the risk of death in the hospital in the final model. Similarly, we used these methods of variable selection and analysis for assessing factors associated with death in the ICU. We presented the ORs and 95% CIs in the univariable logistic regression model and the adjusted ORs (AORs) and 95% CIs in the multivariable logistic regression model.

For all analyses, significance levels were two-tailed, and we considered p<0.05 as statistically significant.

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Results

Data on 252 patients with sepsis were submitted to the database of the MOSAICS II Study (figure 1), in which there were little missing data.

Figure 1.

Flow chart of the study design, patient enrolment and follow-up. ICU, intensive care unit; discharged to die, defined as the patients who were in grave condition or dying and were classified with deaths in the ICU at the time of discharge.

Clinical characteristics and outcomes

In our study cohort, 64.3% (162/252) were men and the median age was 65 years (IQR: 52–76.75) (table 1). Among the total patients, the median SOFA Score was 7 (IQR: 4.75–10) at the time of ICU admission, the median APACHE II Score was 18 (IQR: 13–24) over the first 24 hours of ICU admission, and 29.4% (74/252) of patients had septic shock (table 1). Table 1 also shows that the most common documented comorbidities included cardiovascular disease (31.0%; 78/252), diabetes mellitus (26.6%; 67/252) and chronic neurological disease (14.3%; 36/252), the most common sites of infection included respiratory (56.7%; 143/252), abdominal cavity (24.2%; 61/252), urinary tract (14.7%; 37/252), and skin or cutaneous sites (7.5%; 19/252) and Gram-negative bacteria were isolated in 61.9% (156/252) of patients. Table 2 shows that MV was provided for 68.9% (173/251) of patients and RRT for 40.2% (101/251). Overall, 40.1% (101/252) of patients with sepsis died in the hospital, 33.3% (84/252) of whom died in the ICU (figure 1 and table 2). The median hospital and ICU LOS were 16 (IQR: 10–25) and 10 (IQR: 6–18) days, respectively (table 2). The clinical characteristics, severity of illness, sites of infection and microbiology, compliance with sepsis bundle elements, and life-sustaining treatments during ICU stay were compared between patients who survived and patients who died in the hospital and ICU, as shown in tables 1 and 2, and tables S1–S14 (online supplemental file 3).

Table 1.

Baseline characteristics according to hospital survivability of patients with sepsis

Variables	All cases	Survived	Died	P value*
Hospital and ICU characteristics	n=252	n=151	n=101
University affiliation, no. (%)	99 (39.3)	46 (30.5)	53 (52.5)	<0.001
Training programme in ICU, no. (%)	202 (80.2)	129 (85.4)	73 (72.3)	0.010
Demographics	n=252	n=151	n=101
Age (years), median (IQR)	65 (52–76.75)	65 (53–76)	65 (52–78)	0.810‡
Sex (male), no. (%)	162 (64.3)	93 (61.6)	69 (68.3)	0.275
Documented comorbidities	n=252	n=151	n=101
Cardiovascular disease, no. (%)	78 (31.0)	41 (27.2)	37 (36.6)	0.111
Chronic lung disease, no. (%)	30 (11.9)	18 (11.9)	12 (1.9)	0.992
Chronic neurological disease, no. (%)	36 (14.3)	28 (18.5)	8 (7.9)	0.018
Chronic kidney disease, no. (%)	23 (9.1)	14 (9.3)	9 (8.9)	0.922
Peptic ulcer disease, no. (%)	9 (3.6)	5 (3.3)	4 (4.0)	>0.999†
Chronic liver disease, no. (%)	27 (10.7)	14 (9.3)	13 (12.9)	0.365
Diabetes mellitus, no. (%)	67 (26.6)	40 (26.5)	27 (26.7)	0.966
Connective tissue disease, no. (%)	3 (1.2)	2 (1.3)	1 (1.0)	>0.999†
Immunosuppression, no. (%)	10 (4.0)	7 (4.6)	3 (3.0)	0.744
Haematological malignancies, no. (%)	5 (2.0)	3 (2.0)	2 (2.0)	>0.999†
Solid malignant tumours, no. (%)	12 (4.8)	6 (4.0)	6 (5.9)	0.551†
Vital signs (on admission into ICU)	n=252	n=151	n=101
GCS, median (IQR)	13 (9–15)	14 (10–15)	10 (8–14)	<0.001‡
HR (beats per min), median (IQR)	110 (95.25–125.75)	110 (92–125)	110 (100–129.5)	0.083‡
Temperature (oC), mean (SD)	37.79 (1.01)	37.80 (1.08)	37.77 (0.91)	0.871‡
MBP (mmHg), mean(SD)	75.82 (22.08)	79.75 (22.88)	69.93 (19.51)	0.002‡
RR (breaths per min), median (IQR)	25 (22–30)	25 (22–30)	25 (20–30)	0.693‡
Blood investigations	n=252	n=151	n=101
Total WBC (x109/L), mean (SD)	15.73 (9.20)	15.63 (8.67)	15.88 (9.98)	0.914‡
PLT (x109/L), mean (SD)	185.98 (137.85)	200.71 (129.67)	163.95 (147.15)	0.002‡
Hb (g/dL), mean (SD)	11.14 (2.59)	11.36 (2.68)	10.82 (2.44)	0.088‡
K+ (mmol/L), mean (SD)	3.89 (0.79)	3.90 (0.80)	3.87 (0.77)	0.865‡
Na+ (mmol/L), mean (SD)	136.05 (8.24)	135.62 (8.81)	136.69 (7.80)	0.068‡
Creatinine (µmol/L), mean (SD)	187.85 (151.92)	186.15 (171.60)	190.38 (117.27)	0.030‡
Bilirubin (µmol/l), mean (SD)	32.80 (61.49)	31.74 (72.67)	34.35 (40.09)	0.007‡
pH, mean (SD)	7.37 (0.50)	7.41 (0.64)	7.32 (0.14)	0.004‡
PaO2 (mmHg), mean (SD)	116.17 (74.28)	110.23 (56.25)	124.73 (94.07)	0.665‡
PaO2/FiO2 ratio, mean (SD)	262.48 (149.58)	281.52 (149.39)	235.26 (146.32)	0.003‡
Severity of illness scores	n=252	n=151	n=101
SOFA, median (IQR), n=250	7 (4.75–10)	6 (4-9)	9 (6-12)	<0.001‡
APACHE II, median (IQR)	18 (13–24)	15 (12–21)	22 (16–27)	<0.001‡
Septic shock	74 (29.4)	35 (23.2)	39 (38.6)	0.008
Site of Infection	n=252	n=151	n=101
Respiratory, no. (%)	143 (56.7)	82 (54.3)	61 (60.4)	0.339
Urinary tract, no. (%)	37 (14.7)	30 (19.9)	7 (6.9)	0.004
Abdominal, no. (%)	61 (24.2)	34 (22.5)	27 (26.7)	0.444
Neurological, no. (%)	12 (4.8)	8 (5.3)	4 (4.0)	0.767†
Bones or joints, no. (%)	2 (0.8)	2 (1.3)	0 (0.0)	0.518†
Skin or cutaneous sites, no. (%)	19 (7.5)	7 (4.6)	12 (11.9)	0.033
Intravascular catheter, no. (%)	1 (0.4)	1 (0.7)	0 (0.0)	>0.999†
Infective endocarditis, no. (%)	1 (0.4)	0 (0.0)	1 (1.0)	0.401†
Primary bacteraemia, no. (%)	7 (2.8)	5 (3.3)	2 (2.0)	0.705†
Systemic, no. (%)	6 (2.4)	4 (2.6)	2 (2.0)	>0.999†
Microbiology	n=252	n=151	n=101
No pathogens detected, no. (%)	67 (26.6)	47 (31.1)	20 (19.8)	0.046
Gram-negative bacteria, no. (%)	156 (61.9)	88 (58.3)	68 (67.3)	0.147
Gram-positive bacteria, no. (%)	34 (13.5)	22 (14.6)	12 (11.9)	0.540
Fungi, no. (%)	7 (2.8)	4 (2.6)	3 (3.0)	>0.999†
Viruses, no. (%)	2 (0.8)	0 (0.0)	2 (2.0)	0.160†
Other pathogens, no. (%)	4 (1.6)	3 (2.0)	1 (1.0)	0.651†

See tables S1–S4 (online supplemental file 3) for additional information.

*Comparison between the patients who survived and died using χ² test.

†Fisher’s exact test.

‡Mann–Whitney U test.

APACHE II, Acute Physiology and Chronic Health Evaluation II; FiO₂, fraction of inspired oxygen; GCS, Glasgow Coma Scale; Hb, haemoglobin; HR, heart rate; ICU, intensive care unit; MBP, mean blood pressure; no, number; PaO₂, partial pressure of oxygen in the arterial blood; PLT, platelet count; RR, respiratory rate; SOFA, Sequential Organ Failure Assessment; WBC, white blood cell.

Table 2.

Treatments and outcomes according to hospital survivability of patients with sepsis

Variables	All cases	Survived	Died	P value*
Completion of the sepsis bundle of care	n=241	n=146	n=95
Completion of the sepsis bundle within 1 hour, no. (%)	87 (36.1)	53 (36.3)	34 (35.8)	0.936
Completion of the initial administration of antibiotics within 1 hour, no. (%)	173 (71.8)	109 (74.7)	64 (63.4)	0.219
Completion of the sepsis bundle within 3 hours, no. (%)	108 (44.8)	66 (45.2)	42 (44.2)	0.879
Completion of the initial administration of antibiotics within 3 hours, no. (%)	205 (85.1)	131 (89.7)	74 (77.9)	0.012
Life-sustaining treatments	n=251	n=150	n=101
Respiratory support, no. (%)
Mechanical ventilation	173 (68.9)	82 (54.7)	91 (90.1)	<0.001
Non-invasive ventilation	20 (8.0)	13 (8.7)	7 (6.9)	0.618
High-flow nasal oxygen	38 (15.1)	29 (19.3)	9 (8.9)	0.024
Additional ICU support, no. (%)
Vasopressors/inotropes	163 (64.7)	82 (54.3)	81 (80.2)	<0.001
Renal replacement therapy	101 (40.2)	43 (28.7)	58 (57.4)	<0.001
Red blood cell transfusion	93 (37.1)	48 (32.0)	45 (44.6)	0.043
Platelet transfusion	50 (19.9)	20 (13.3)	30 (29.7)	0.001
Fresh frozen plasma transfusion	58 (23.1)	28 (18.7)	30 (29.7)	0.042
Surgical source control	25 (10.0)	19 (12.7)	6 (5.9)	0.081
Non-surgical source control	78 (31.1)	54 (36.0)	24 (23.8)	0.040
Outcomes	n=252	n=151	n=101
Patient status, no. (%)				<0.001†
Alive on current hospital discharge	150 (59.5)	150 (99.3)	0 (0.0)
Alive on discharge from current ICU stay, but died in current hospital stay	17 (6.7)	0 (0.0)	17 (16.8)
Alive on discharge from current ICU stay, but still in current hospital stay after 90 days	1 (0.4)	1 (0.7)	0 (0.0)
Still in current ICU stay after 90 days	0 (0.0)	0 (0.0)	0 (0.0)
Died in current ICU stay	84 (33.3)	0 (0.0)	84 (83.2)
Length of stay, median days (IQR)
Hospital	16 (10–25)	17 (11–24.25)	13 (7–26)	0.027‡
ICU	10 (6–18)	6–1710.5 (6-17)	10 (5–21)	0.740‡

See tables S5–S7 (online supplemental file 3) for additional information.

*Comparison between the patients who survived and died using χ² test.

†Fisher’s exact test.

‡Mann–Whitney U test.

ICU, intensive care unit; no, number.

Overall prognostic performance of the severity scoring systems

The SOFA Score (AUROC: 0.688 (95% CI 0.618 to 0.758); cut-off value≥7.5; sensitivity: 64.4%; specificity: 69.8%; P_AUROC<0.001) and APACHE II Score (AUROC: 0.689 (95% CI 0.622 to 0.756); cut-off value≥20.5; sensitivity: 61.4%; specificity: 71.8%; P_AUROC<0.001) both had a poor discriminatory ability for the hospital mortality (figure 2). The discriminatory ability for the ICU mortality of SOFA Score (AUROC: 0.713 (95% CI 0.643 to 0.783); cut-off value≥9.5; sensitivity: 53.6%; specificity: 80.1%; P_AUROC<0.001), however, was fair and was better than that of the APACHE II Score (AUROC: 0.672 (95% CI 0.603 to 0.742); cut-off value≥18.5; sensitivity: 69.0%; specificity: 60.8%; P_AUROC<0.001) (figure 3).

Figure 2.

Comparisons of the AUROCs: Comparing the overall diagnostic performance of the SOFA Score (AUROC: 0.688 (95% CI 0.618 to 0.758); cut-off value≥7.5; sensitivity: 64.4%; specificity: 69.8%; P_AUROC<0.001) and the APACHE II Score (AUROC: 0.689 (95% CI 0.622 to 0.756); cut-off value≥20.5; sensitivity: 61.4%; specificity: 71.8%; P_AUROC<0.001) for predicting hospital mortality in ICU patients with sepsis. APACHE II, Acute Physiology and Chronic Health Evaluation II Score; AUROC, areas under the ROC curve; ICU, intensive care unit; ROC, receiver operating characteristics; SOFA, Sequential Organ Failure Assessment.

Figure 3.

Comparisons of the AUROCs: Comparing the overall diagnostic performance of the SOFA Score (AUROC: 0.713 (95% CI 0.643 to 0.783); cut-off value≥9.5; sensitivity: 53.6%; specificity: 80.1%; P_AUROC<0.001) and the APACHE II Score (AUROC: 0.672 (95% CI 0.603 to 0.742); cut-off value≥18.5; sensitivity: 69.0%; specificity: 60.8%; P_AUROC<0.001) for predicting ICU mortality in ICU patients with sepsis. APACHE II, Acute Physiology and Chronic Health Evaluation II; AUROC, areas under the ROC curve; ICU, intensive care unit; ROC, receiver operating characteristics; SOFA, Sequential Organ Failure Assessment.

Risk factors for mortality

In the multivariable analysis, a SOFA Score of 8 and above (AOR: 2.717; 95% CI 1.371 to 5.382) and an APACHE II Score of 21 and above (AOR: 2.668; 95% CI 1.338 to 5.321) were independently associated with an increased risk of hospital mortality (table 3). Additionally, a SOFA Score of 10 and above (AOR: 2.801; 95% CI 1.332 to 5.891) was independently associated with an increased risk of ICU mortality, in contrast to an APACHE II Score of 19 and above, for which this independent association was not observed (table 4). Other factors were significantly or independently associated with the risk of hospital and ICU mortalities, as shown in tables 3 and 4, and tables S15–S18) (online supplemental file 3).

Table 3.

Factors relating to hospital mortality in patients with sepsis

Factors	Univariable logistic regression analyses*				Multivariable logistic regression analyses†
	OR	95% CI for OR		P value	AOR	95% CI for AOR		P value
		Lower	Upper			Lower	Upper
Hospital and ICU characteristics
University affiliation	2.520	1.495	4.248	0.001	NA	NA	NA	NA
Training programme in ICU	0.445	0.237	0.833	0.011	0.392	0.162	0.949	0.038
Documented comorbidities
Cardiovascular disease	1.551	0.903	2.664	0.112	2.181	1.019	4.664	0.044
Chronic neurological disease	0.378	0.165	0.867	0.022	0.179	0.058	0.546	0.003
Severity of illness scores
SOFA Score≥8	4.173	2.440	7.137	<0.001	2.717	1.371	5.382	0.004
APACHE II Score≥21	4.126	2.414	7.051	<0.001	2.668	1.338	5.321	0.005
Site of infection
Urinary tract	0.300	0.126	0.714	0.006	0.312	0.105	0.932	0.037
Abdominal	1.256	0.701	2.249	0.444	NA	NA	NA	NA
Skin or cutaneous sites	2.774	1.053	7.309	0.039	NA	NA	NA	NA
Microbiology
No pathogens detected	0.546	0.300	0.994	0.048	NA	NA	NA	NA
Gram-negative bacteria	1.475	0.871	2.498	0.148	NA	NA	NA	NA
Completion of sepsis bundle elements
Completion of the sepsis bundle within 1 hour	0.978	0.571	1.675	0.936	NA	NA	NA	NA
Completion of the administration of antibiotics within 1 hour	0.701	0.397	1.237	0.220	NA	NA	NA	NA
Completion of the sepsis bundle within 3 hours	0.961	0.571	1.615	0.879	NA	NA	NA	NA
Completion of the administration of antibiotics within 3 hours	0.403	0.196	0.830	0.014	0.381	0.151	0.965	0.042
Life-sustaining treatments during ICU stay
Respiratory support
Mechanical ventilation	7.546	3.645	15.625	<0.001	4.391	1.912	10.085	<0.001
High-flow nasal oxygen	0.408	0.184	0.904	0.027	NA	NA	NA	NA
Additional ICU support
Vasopressors/inotropes	3.408	1.899	6.116	<0.001	NA	NA	NA	NA
Renal replacement therapy	3.356	1.976	5.702	<0.001	NA	NA	NA	NA
Red blood cell transfusion	1.708	1.014	2.876	0.044	NA	NA	NA	NA
Platelet transfusion	2.746	1.455	5.185	0.002	NA	NA	NA	NA
Fresh frozen plasma transfusion	1.841	1.018	3.329	0.043	NA	NA	NA	NA
Surgical source control	0.435	0.168	1.132	0.088	NA	NA	NA	NA
Non-surgical source control	0.554	0.314	0.977	0.041	NA	NA	NA	NA
Constant					0.230			0.007

See tables S15 and S16 (online supplemental file 3) for additional information.

*Each variable of hospital and ICU characteristics, baseline characteristics, clinical and laboratory characteristics, and treatments was analysed in the univariable logistic regression model and was considered in the multivariable logistic regression model if the value of p was<0.05 in univariable logistic regression analysis between survival and death in the hospital, as well as clinically crucial factors.

†All selected variables were included in the multivariable logistic regression model with the stepwise backward elimination method. Variables, then, were deleted stepwise from the full model until all remaining variables were independently associated with death in the hospital.

AOR, adjusted OR; APACHE II, Acute Physiology and Chronic Health Evaluation II; ICU, intensive care unit; NA, not available; SOFA, Sequential Organ Failure Assessment.

Table 4.

Factors relating to intensive care unit mortality in patients with sepsis

Factors	Univariable logistic regression analyses*				Multivariable logistic regression analyses†
	OR	95% CI for OR		P value	AOR	95% CI for AOR		P value
		Lower	Upper			Lower	Upper
Hospital and ICU characteristics
University affiliation	2.260	1.322	3.862	0.003	2.562	1.164	5.639	0.019
Intensivist to patient ratio
1 intensivist : 5 or fewer patients	Reference			0.082	NA			NA
1 intensivist : 6 to 8 patients	0.553	0.298	1.025	0.060	NA	NA	NA	NA
1 intensivist : 12 or more patients	1.750	0.540	5.668	0.351	NA	NA	NA	NA
Training programme in ICU	0.458	0.243	0.861	0.015	0.267	0.100	0.713	0.008
Documented comorbidities
Cardiovascular disease	1.506	0.863	2.627	0.150	2.047	0.954	4.391	0.066
Chronic neurological disease	0.526	0.229	1.212	0.131	4.630	1.130	18.970	0.033
Solid malignant tumours	2.077	0.649	6.648	0.218	NA	NA	NA	NA
Severity of illness scores
SOFA Score≥10	4.650	2.620	8.254	<0.001	2.801	1.332	5.891	0.007
APACHE II Score≥19	3.535	1.025	6.171	<0.001	NA	NA	NA	NA
Site of infection
Urinary tract	0.340	0.136	0.851	0.021	0.276	0.087	0.878	0.029
Abdominal	1.416	0.779	2.575	0.254	NA	NA	NA	NA
Skin or cutaneous sites	2.387	0.931	6.123	0.070	3.074	0.982	9.629	0.054
Microbiology
No pathogens detected	0.599	0.320	1.121	0.109	NA	NA	NA	NA
Gram-negative bacteria	1.258	0.729	2.171	0.409	NA	NA	NA	NA
Completion of sepsis bundle elements
Completion of the sepsis bundle within 1 hour	0.931	0.532	1.630	0.802	NA	NA	NA	NA
Completion of the administration of antibiotics within 1 hour	0.671	0.374	1.202	0.180	NA	NA	NA	NA
Completion of the sepsis bundle within 3 hours	0.938	0.546	1.609	0.815	NA	NA	NA	NA
Completion of the administration of antibiotics within 3 hours	0.434	0.211	0.889	0.023	0.344	0.122	0.970	0.044
Life-sustaining treatments during ICU stay
Respiratory support
Mechanical ventilation	6.856	3.109	15.116	<0.001	3.086	1.180	8.072	0.022
High-flow nasal oxygen	0.257	0.096	0.685	0.007	NA	NA	NA	NA
Additional ICU support
Vasopressors/inotropes	2.956	1.600	5.460	0.001	NA	NA	NA	NA
Renal replacement therapy	4.239	2.432	7.388	<0.001	3.433	1.669	7.058	0.001
Red blood cell transfusion	1.682	0.983	2.879	0.058	NA	NA	NA	NA
Platelet transfusion	2.966	1.571	5.597	0.001	NA	NA	NA	NA
Fresh frozen plasma transfusion	1.891	1.036	3.453	0.038	NA	NA	NA	NA
Surgical source control	0.599	0.230	1.562	0.295	NA	NA	NA	NA
Non-surgical source control	0.535	0.293	0.977	0.042	0.385	0.175	0.842	0.017
Constant					0.182			0.004

See tables S17 and S18 (online supplemental file 3) for additional information.

*Each variable of hospital and ICU characteristics, baseline characteristics, clinical and laboratory characteristics, and treatments was analysed in the univariable logistic regression model and was considered in the multivariable logistic regression model if the value of p was <0.05 in univariable logistic regression analysis between survival and death in the ICU, as well as clinically crucial factors.

†All selected variables were included in the multivariable logistic regression model with the stepwise backward elimination method. Variables, then, were deleted stepwise from the full model until all remaining variables were independently associated with death in the ICU.

AOR, adjusted OR; APACHE II, Acute Physiology and Chronic Health Evaluation II; ICU, intensive care unit; NA, not available; SOFA, Sequential Organ Failure Assessment.

Discussion

Of 252 patients with sepsis included in our analysis, two-fifths (40.1%) died in the hospital, and about a third (33.3%) died during the ICU stay (figure 1 and table 2). The SOFA and APACHE II Scores had a poor discriminatory ability for predicting hospital mortality (figure 2). However, the overall performance of the SOFA Score for predicting ICU mortality was fair and was better than that of the APACHE II Score (figure 3). A SOFA Score of 8 and above and an APACHE II Score of 21 and above were independently associated with an increased risk of hospital mortality (table 3). Additionally, a SOFA score of 10 and above was an independent predictor of ICU mortality, in contrast to an APACHE II score of 19 and above, for which this role did not appear (table 4).

In our study, the hospital mortality rate was lower than that of the MOSAICS I Study (44.5%; 572/1285),⁴⁰ as well as the rates previously reported from LMICs in South-East Asia, including Indonesia (68.3%; 41/60),⁴¹ Thailand (42%; 263/627)⁴² and Vietnam (61.0%; 75/123).³⁹ These findings may be because the diagnosis and treatment of sepsis have significantly changed over the previous 10 years to increase patient survival in sepsis and septic shock.1 8 13 36 38 43 44 However, our study showed rates for ICU and hospital mortality that were higher than rates reported in the international Extended Study on Prevalence of Infection in Intensive Care III (EPIC III) (28% (99/352) and 31.1% (110/352) in LMICs, 26.4% (821/3114) and 32.7% (1019/3114) in upper-middle-income countries (UMICs), and 21.3% (950/4470) and 28.5% (1275/4470) in HICs).⁴⁵ These variations might be because EPIC III included ICU-acquired infection rather than only sepsis.⁴⁵ Despite the distinct inclusion criteria, our median SOFA Score at the time of ICU admission was comparable to that of EPIC III (7 points (IQR: 4–11) in LMICs/UMICs/HICs).⁴⁵ However, patients in our study received invasive organ support treatments (ie, MV and RRT) during ICU stays more frequently than those in EPIC III (54.4% (4377/8045) and 15.7% (1253/8045)).⁴⁵ Previous studies showed that MV was a crucial predictor of mortality at any point throughout the ICU stay.^{4 35} Additionally, the utilisation of RRT at any time during the ICU stay was also associated with a higher fatality rate.4 35 46–48 Furthermore, Acinetobacter baumannii (17.9%, 45/252; table S4, online supplemental file 3), one of the most harmful pathogens, was more frequently isolated from patients in the present study than in those from the HIC cohort (4.4%; 137/3113) of the EPIC III Study.⁴⁵ The previous studies showed that A. baumannii infection was often due to a lack of strict infection control bundles⁴⁹ and associated with an increased risk of death.^{50 51} The fact that our proportions for ICU and hospital mortality were higher than those reported in EPIC III suggested that patients, pathogens and clinical capacity to manage sepsis vary significantly between regions, particularly between HIC and LMIC settings.

In this study, we found a poor ability of both SOFA and APACHE II Scores to predict hospital mortality (figure 2). However, with the SOFA Score, the discrimination for predicting ICU mortality was fair, and it was better than those of the APACHE II Score (figure 3). The APACHE scoring system is among the most widely used, of which there are four versions (APACHE I through IV Scores). Although APACHE IV Score is the most up-to-date version, some centres still use older versions including APACHE II Score. In the present study, despite having a poor discriminatory ability for predicting hospital and ICU mortalities, an APACHE II Score of 21 and above was independently associated with an increased risk of deaths in hospitals (table 3). However, in contrast to a SOFA Score of 10 and above, an APACHE II Score of 19 and above was not an independent predictor of ICU mortality (table 4). Previous studies revealed that the APACHE II Score had a good prognostic value in acutely ill or surgical patients^{10 11} but did not differentiate between sterile and infected necrotising pancreatitis and had a poor predictive value for the severity of acute pancreatitis at 24 hours.¹²

In contrast, the SOFA Score was proposed for patients with a suspected infection that an increase of 2 points or more could serve as clinical criteria for sepsis.¹ In ICU patients with suspected infection, discrimination of the SOFA Score was fair for predicting hospital mortality, with an AUROC value of 0.74 (95% CI, 0.73 to 0.76; P_AUROC<0.001), reported in the previously published studies.^{1 17} However, our study showed that the discriminatory ability of the SOFA Score was poor for predicting hospital mortality (figure 2). This difference might be due to our SOFA Score only calculated on ICU admission, in contrast to the SOFA Score in the previously published study that was calculated for the time window from 48 hours before to 24 hours after the onset of an infection, as well as on each calendar day.¹⁷ This difference also might be because the burden and causes of sepsis and its management differ considerably between HIC and LMIC settings,^{7 35 37} which might make the accuracy of critical illness severity scoring systems vary widely in the different countries, particularly between HICs and LMICs. However, our study revealed that the SOFA Score had a fair discriminatory ability for predicting ICU mortality (figure 3). Moreover, a SOFA Score of 8 and above and a score of 10 and above were independently associated with an increased risk of deaths in hospitals and ICUs, respectively (tables 3 and 4). Overall, this study shows that both SOFA and APACHE II Scores were worthwhile in predicting hospital and ICU mortalities in ICU patients with sepsis. However, because of having better discrimination for predicting ICU mortality, the SOFA Score was preferable to the APACHE II Score in predicting mortality.

The present study’s data from many centres, which contained few missing data points, was a benefit (tables S19, online supplemental file 3). The following are some drawbacks of the current study, though: first, since there isn't a national registry of ICUs to enable systematic recruitment of units, we used the snowball method to find suitable units, which may have resulted in the selection of centres with a greater interest in managing sepsis; as a result, our data are subject to selection bias and might not accurately reflect intensive care in all of Vietnam; second, we did not create a protocol for microbiological investigations due to the study’s real-world aspect. The data on point-of-care tests (such as lactate clearance) and life-sustaining therapies (such as fluid balance, steroid administration, and modalities of RRT and MV) were also missing since we primarily evaluated resources used in ICUs. Additionally, we decided not to gather information on antibiotic resistance and appropriateness to increase the practicality of performing the study in busy ICUs; third, the mixed-effects logistic regression model could not be used to predict the discrete outcome variables measured at two different times, that is, inside and outside the ICU settings, due to our independent variables (eg, SOFA Score that was greater than or equal to the cut-off value), which might be associated with the primary outcome only measured on ICU admission; finally, even though the sample size was sufficient, the CI was a little bit broad (6.03%), which may have an impact on the sample’s normal distribution. Therefore, more studies with bigger sample sizes may be required to strengthen the findings.

Conclusions

Our cohort was a selected population of patients with sepsis admitted to the ICUs in Vietnam with a high mortality rate. The SOFA and APACHE II Scores were worthwhile in predicting mortality among ICU patients with sepsis. However, due to better discrimination for predicting ICU mortality, the SOFA Score was preferable to the APACHE II Score in predicting mortality.

Data availability statement

Data are available upon reasonable request. All data relevant to the study are included in the article or uploaded as supplementary information.

Ethics statements

Patient consent for publication

Not applicable.

Ethics approval

This study involves human participants. The Scientific and Ethics Committees of Bach Mai Hospital approved this study (approval number: 2919/QĐ–BM; project code: BM-2017-883-89). The authors also obtained permission from the heads of institutions and departments of all participating hospitals and their respective institutional review boards wherever available. The study was conducted according to the principles of the Declaration of Helsinki. In this non-intervention study, all collected information has received verbal informed consent from patients or, when unavailable, from family members at the ICUs, and witnessed by the on-duty medical staff. Written informed consent, however, was waived by the Bach Mai Hospital Scientific and Ethics Committees since it was not feasible to undergo such a methodical process of collection when the subject was comprised of an urgent situation in which a patient or a family member’s condition was severe or life-threatening. Public notification of the study was made by public posting. All data analyses were based upon data sets kept in password-protected systems, and all final presented data have been made anonymous. Participants gave informed consent to participate in the study before taking part.