qSOFA as a predictor of ICU outcomes in a resource-limited setting in KwaZulu-Natal Province, South Africa

Abstract

Background

Sepsis is a major cause of morbidity and mortality, especially in critical care patients. Developing tools to identify patients who are at risk of poor outcomes and prolonged length of stay in intensive care units (ICUs) is critical, particularly in resource-limited settings.

Objectives

To determine whether the quick sequential organ failure assessment (qSOFA) score based on bedside assessment alone was a promising tool for risk prediction in low-resource settings.

Methods

A retrospective cohort of adult patients admitted to the intensive care unit (ICU) at Edendale Hospital in Pietermaritzburg, South Africa (SA), was recruited into the study between 2014 and 2018. The association of qSOFA with in-ICU mortality was measured using multivariable logistic regression. Discrimination was assessed using the area under the receiver operating characteristic curve and the additive contribution to a baseline model using likelihood ratio testing.

Results

The qSOFA scores of 0, 1 and 2 were not associated with increased odds of in-ICU mortality (adjusted odds ratio (aOR) 1.24, 95% confidence interval (CI) 0.86 - 1.79; p=0.26) in patients with infection, while the qSOFA of 3 was associated with in-ICU mortality in infected patients (aOR 2.82; 95% CI 1.91 - 4.16; p<0.001). On the other hand, the qSOFA scores of 2 (aOR 3.25; 95% CI 1.91 - 5.53; p<0.001) and 3 (aOR 6.26, 95% CI 0.38 - 11.62, p<0.001) were associated with increased odds of in-ICU mortality in patients without infection. Discrimination for mortality was fair to poor and adding qSOFA to a baseline model yielded a statistical improvement in both cases (p<0.001).

Conclusion

qSOFA was associated with, but weakly discriminant, for in-ICU mortality for patients with and without infection in a resource-limited, public hospital in SA. These findings add to the growing body of evidence that support the use of qSOFA to deliver low-cost, high-value critical care in resource-limited settings

Contributions of the study

This study expanded the data supporting the use of qSOFA in resource-limited settings beyond the emergency department or ward to include patients admitted to the ICU. Additionally, this study demonstrated stronger predictive abilities in a population of patients admitted with trauma without suspected or confirmed infection, thus providing an additional use of qSOFA as a risk-prediction tool for a broader population.

Background

The global burden of sepsis poses significant human and economic costs. Recent studies reported 31.5 million cases of sepsis, leading to 5.3 million sepsis-related deaths annually worldwide.^[1–4] This burden is disproportionately felt in developing countries, which are often limited in access to both diagnostics and therapeutics.^[5]

The third international consensus definitions for sepsis and septic shock (Sepsis-3) proposed the sequential (sepsis-related) organ failure assessment (SOFA) as an updated definition of sepsis to replace the systemic inflammatory response syndrome (SIRS; Sepsis-2) criteria. ^[6] Additionally, they introduced the quick sequential (sepsis-related) organ failure assessment score (qSOFA) as a novel risk score to identify patients with infection who are at risk of poor outcomes including in-hospital mortality and long intensive care unit (ICU) length of stay.^[2–4] In the Sepsis-3 validation study for predicting in-hospital mortality, qSOFA was equivalent to SOFA and superior to SIRS among patients outside the ICU, and inferior to SOFA but equivalent to SIRS among patients in the ICU^[4]. As with many risk predictors, qSOFA was initially derived and validated only in well-resourced settings.^[3,4] In response, a large retrospective secondary analysis evaluated qSOFA in 10 developing countries across sub-Saharan Africa, Asia and the Americas, and reported that qSOFA was associated with hospital mortality but with variable predictive validity. ^[7] Other smaller studies have reported similar findings.^[8,9]

qSOFA is of particular immediate interest for under-resourced settings because it relies on bedside physical exam findings alone, without requiring laboratory studies such as those needed for SOFA and SIRS calculations, among other risk scores which may be unavailable or prohibitively expensive. An inexpensive and accessible tool to guide risk stratification could prove useful in optimising allocation of scarce acute and critical care resources such as ICU beds, and in identifying patients at high risk for poor outcomes early in their treatment course.^[10] The present study sought to further elucidate the use of qSOFA in resource-limited settings by examining its association with and discrimination for mortality in: (i) the South African public health system, a previously unstudied resource-limited setting; (ii) an ICU population; and (iii) in both infection and non-infection populations.

Methods

The integrated critical care electronic database is the first published multicentre database of its kind in South Africa (SA)^[11,12]] and includes all referrals and admissions for ICU care at two public hospitals within the KwaZulu-Natal Department of Health. This study evaluated ICU admissions at Edendale Hospital, a regional hospital with 900 in-patient beds and one mixed medical-surgical ICU that admits adult and paediatric patients with any aetiology of critical illness and has a closed, high-intensity staffing model.

The integrated critical care electronic database and the study protocol were approved by the Biomedical Research Ethics Administration of the University of KwaZulu-Natal (ref. no. BCA211/14, BE156/19). The study protocol was approved by the Institutional Review Board of the University of Pennsylvania (ref. no. 824688).

All adult patients (≥18 years) admitted to the ICU at Edendale Hospital from September 2014 to August 2018 were included in this study.

A retrospective cohort study of the association and discrimination of qSOFA for in-ICU mortality was performed. For comparison, parallel analyses using the SIRS criteria were also performed.^[13] Comparison with the full SOFA score was not possible owing to data availability. Both inferential and predictive analytical approaches were used to align this study with prior recent work evaluating qSOFA in resource-limited settings.^[7]

The study population was stratified based on the presence or absence of confirmed or suspected infection. Confirmed or suspected infection was defined as a primary ICU admission diagnosis of infection, as recorded prospectively by the clinical team or an antibiotic order at the time of ICU admission. Antibiotic orders have been used in prior studies as a component of confirmed or suspected infection,^[14,15] but culture orders, used to remove antibiotics other than for acute infection, were not currently available in the database. This potential source of bias is addressed in sensitivity analyses.

The primary exposure variable was qSOFA at the time of the ICU admission. qSOFA includes one point each for altered mental status (Glasgow Coma Score <15), systolic blood pressure ≤100 mmHg, and respiratory rate ≥22 breaths/minute. Data were collected at the time of ICU referral (e.g. from the emergency department (ED), wards or another hospital) and updated at the time of ICU admission. SIRS criteria included exposure in a secondary analysis. qSOFA and SIRS calculations used the worst values for each component collected during this referral to admission window. Owing to the low frequency of qSOFA scores of 0 (4.8%) and SIRS criteria of 0 (0.3%) in this ICU population, qSOFA scores and SIRS criteria of 0 and 1 were pooled as the reference levels.

The primary outcome was in-ICU mortality, defined as death in the ICU or a palliative discharge from the ICU. The database does not follow patients longitudinally after ICU discharge and, as a result, hospital and other longer-term outcomes were not available.^[15]

To align with prior validation studies of qSOFA in developing countries, a priori covariates in baseline risk models included age, sex and HIV status.^[7]

Descriptive statistical analyses were performed on the total cohort of patients admitted to the ICU. Multivariable logistic regression was used to evaluate the association with in-ICU mortality first of a baseline risk model including only the a priori covariates age, sex and HIV status, and then of the baseline risk model plus qSOFA score or SIRS criteria. Discrimination of each model was assessed using area under the receiver operating characteristic curve (AUROC). Finally, the additive contribution of qSOFA or SIRS was assessed by performing likelihood ratio (LR) testing between the baseline model and the baseline plus qSOFA or SIRS model. All of the above analyses were stratified by infection status.

The qSOFA exposure variable, in-ICU mortality outcome variable, and all adjustment variables had either complete data or were missing in less than 1.5% of patients, allowing for complete case analyses. SIRS criteria used in the secondary comparative analyses were missing in 14.6% of patients, driven almost entirely by the sole laboratory input of white blood cell count (missing in 14.6% of patients compared with 0.8 - 3.4% missing for the non-white blood cell count SIRS criteria). White blood cell count derangements are likely to correlate with derangements in the other SIRS criteria that were missing at low levels; thus, a complete case approach was performed for these secondary analyses.

To reduce the risk of misclassifying patients into the suspected or confirmed infection groups who were receiving antibiotics for peri-procedural prophylaxis but who did not have a true active or suspected infection, qSOFA analyses were repeated, further restricting the suspected or confirmed infection group to patients referred to the ICU by a medical rather than surgical service, and to patients with a primary ICU admission diagnosis of infection as recorded prospectively by the ICU team. All analyses were done using Stata v14.1 (StataCorp LP, USA).

Results

The study population characteristics and infection status are illustrated in Table 1.

Table 1. Study population characteristics by infection status (N=2 119).

Characteristics	Confirmed or suspected infection (n=1 190), n(%)*	No confirmed or suspected infection (n=929), n(%)*
Age (years), mean (SD)	40.7 (15.9)	37.7 (15.6)
Female gender	570 (47.9)	378 (40.7)
Black race	1 142 (96.0)	869 (93.5)
Referring specialty
Surgical	875 (73.5)	684 (73.6)
Medical	287 (24.1)	217 (23.4)
Unknown	28 (2.4)	28 (3.0)
Pre-ICU hospital LOS, median days (IQR)	1 (0 - 3)	0 (0 - 1)
Invasive mechanical ventilation at time of ICU admission	710 (59.7)	518 (55.8)
HIV-positive	321 (27.0)	175 (18.8)
On HAART	245 (76.3)	143 (81.7)
qSOFA score at ICU admission
0 - 1	412 (34.6)	406 (43.7)
2	509 (42.8)	384 (41.3)
3	257 (21.6)	120 (12.9)
Missing	12 (1.0)	19 (2.1)
SIRS criteria at ICU admission
0 - 1 criteria	27 (2.3)	35 (3.8)
2 criteria	184 (15.5)	165 (17.8)
3 criteria	439 (36.9)	345 (17.1)
4 criteria	382 (32.1)	232 (25.0)
Missing	158 (13.3)	152 (16.4)
Sepsis
qSOFA score definition	766 (64.4)	n/a
SIRS criteria definition	1 005 (84.5)	n/a
ICU LOS, median days (IQR)	2.6 (1.2 - 5.2)	2.0 (1.0 - 4.3)
In-ICU mortality	232 (19.5)	110 (11.8)

SD = standard deviation

ICU = intensive care unit

LOS = length of stay

IQR = interquartile range

qSOFA = quick sequential organ failure assessment

SIRS = systemic inflammatory response syndrome

* Unless otherwise specified

Association of qSOFA and in-ICU mortality among patients with confirmed or suspected infection

Unadjusted predicted in-ICU mortality based on qSOFA is shown in Fig. 1. qSOFA scores of 0, 1 and 2 were not associated with in-ICU mortality (adjusted odds ratio (aOR) 1.24; 95% confidence interval (CI) 0.86 - 1.79; p=0.26) in patients with confirmed or suspected infection, but a qSOFA score of 3 was associated with increased odds of in-ICU mortality in these patients (aOR 2.82; 95% CI 1.91 - 4.16; p<0.001) (Table 2).

Fig. 1.

qSOFA score and ICU mortality by infection status

ICU = intensive care unit

qSOFA = quick sequential organ failure assessment

Table 2. Association of qSOFA with in-ICU mortality.

Characteristics	Confirmed or suspected infection (n=1 175); aOR (95% CI), p-value	No confirmed or suspected infection (n=906); aOR (95% CI), p-value
qSOFA (Ref: 0 - 1 points)
2 points	1.24 (0.86 - 1.79), 0.26	3.25 (1.91 - 5.53), <0.001*
3 points	2.82 (1.91 - 4.16), <0.001*	6.26 (3.38 - 11.62), <0.001*
Age	1.02 (1.01 - 1.0), 0.001*	1.03 (1.01 - 1.04), <0.001*
Male sex	0.66 (0.49 - 0.90), 0.008*	1.12 (0.72 - 1.74), 0.61
HIV positive	1.33 (0.95 - 1.86), 0.09	0.61 (0.33 - 1.15), 0.13

qSOFA = quick sequential organ failure assessment

aOR = adjusted odds ratio

CI = confidence interval

Discrimination for mortality was poor for both the baseline model (AUROC 0.61; 95% CI 0.58 - 0.65) and the baseline plus qSOFA model (AUROC 0.66; 95% CI 0.62 - 0.70), but the addition of qSOFA to the baseline model made a statistical improvement (LR test χ² 30.8; p<0.001) (Table 3, http://sajcc.org.za/public/sup/433.zip).

Table 3. qSOFA as a predictor of in-ICU mortality.

	AUROC (95% CI)	LR χ² (p-value)
Confirmed or suspected infection (n=1 175)
Baseline factors	0.61 (0.58 - 0.65)	30.8 (<0.0001)
qSOFA + baseline factors	0.66 (0.62 - 0.70)
No confirmed or suspected infection(n=906)
Baseline factors	0.61 (0.55 - 0.67)	39.6 (<0.0001)
qSOFA + baseline factors	0.71 (0.66 - 0.77)

qSOFA = quick sequential organ failure assessment

AUROC = area under the receiver operating characteristic curve

CI = confidence interval

LR = likelihood ratio test

Association of qSOFA and in-ICU mortality among patients without confirmed or suspected infection

The adjusted for baseline risk factors qSOFA scores of 2 (aOR 3.25; 95% CI 1.91 - 5.53; p<0.001) and 3 (aOR 6.26; 95% CI 3.38 - 11.62; p<0.001) were associated with increased odds of in-ICU mortality in patients without confirmed or suspected infection (Table 2). Discrimination for mortality was poor for the baseline model (AUROC 0.61; 95% CI 0.55 - 0.67), fair for the baseline plus qSOFA model (AUROC 0.71; 95% CI 0.66 - 0.77) and the addition of qSOFA to the baseline model made a statistical improvement (LR test χ² 39.6; p<0.001) (Table 3, http://sajcc.org.za/public/sup/433.zip).

Association of SIRS and in-ICU mortality

In both the infection and non-infection subgroups, SIRS criteria were not statistically associated with in-ICU mortality in multivariable logistics regression adjusted for baseline risk factors. Discrimination for mortality was poor for both the baseline and the baseline plus SIRS models, and the addition of SIRS to the baseline model did not yield a statistical improvement (Tables 1 and 2)).

Sensitivity analyses

The qSOFA score was unable to discriminate mortality among patients with confirmed or suspected infection who were referred by a medical service (AUROC 0.63; 95% CI 0.56 - 0.71) and those who had infection as the primary ICU admission diagnosis (AUROC 0.66; 95% CI 0.60 - 0.71). However, the addition of qSOFA to the baseline model resulted in a statistically significant difference between both cases (Table 3).

Discussion

In this retrospective cohort study in a resource-limited public hospital in SA, qSOFA score was highly associated with, but weakly discriminant for in-ICU mortality, improved upon a baseline risk model, and was superior to SIRS criteria. These findings are consistent with prior multicentre retrospective studies in other sub-Saharan African countries.^[7–9]

The present study expanded the data supporting the use of qSOFA in resource-limited settings in two notable ways: firstly, this study evaluated the performance of qSOFA for ICU patients at the time of ICU admission, rather than in patients in the ED or on the ward. The superior performance of qSOFA compared to SIRS in our setting differed from the Sepsis-3 validation studies which showed that qSOFA was superior to SIRS for ward patients but comparable with SIRS for ICU patients.^[3,4] While ICUs in resource-limited settings may by definition have greater resources than nearby wards, they may still benefit from access to a well-performing risk predictor that does not require laboratory testing for use in guiding management decisions and allocating scarce resources, including ICU beds. Secondly, this study evaluated the performance of qSOFA for patients both with and without suspected or confirmed infection and showed even stronger predictive abilities in the non-infection population. The potential addition of qSOFA to the risk-prediction toolkit for all patients admitted to the ICU, including those admitted for trauma, who make up large proportions of admissions to typically mixed medical-surgical ICUs in resource-limited settings,^[12] would be further beneficial.

The results of this study should be interpreted in the context of several important limitations. Firstly, this study was performed at a single hospital and the results should be interpreted within the greater body of emerging literature evaluating qSOFA at multiple facilities across diverse countries. Furthermore, South Africa is a developing country^[16] that has immense income and wealth inequality and an under-resourced public healthcare system that serves a very low-income population,^[17] so the results may not apply to extremely low-resourced settings where systems for critical care delivery are largely absent.

Secondly, because microbial culture orders are not available in the electronic database,^[15] the primary definition of suspected or confirmed infection in this study, which includes patients with an antibiotic order alone at the time of ICU admission, certainly misclassifies some patients into the suspected or confirmed infection group who were receiving antibiotics for peri-procedural prophylaxis but who did not have a true active or suspected infection. This issue is addressed with sensitivity analyses using more specific definitions for infection that show a consistent but attenuated result, consistent with a stronger qSOFA-mortality relationship observed in non-infected patients.

Thirdly, while both inferential (association) and predictive analytic (discrimination) methods were used in an effort to align this study with recent studies evaluating qSOFA in resource-limited settings,^[7] this approach could have produced over-fit models and provided overly optimistic performance assessments from a predictive standpoint. Results should be interpreted in this context and future studies may wish to adhere to a strictly predictive statistical framework.

Finally, this study could not make a useful comparison between qSOFA and the full SOFA score owing to unavailability or lack of laboratory data. Future studies should further compare qSOFA with other risk predictors, including those developed and validated in resource-limited settings such as the Rwanda mortality probability model (R-MPM).^[10] Additionally, because this database does not follow patients longitudinally after ICU discharge, hospital and other longer-term outcomes were not explored.

Conclusion

The qSOFA was associated with, but weakly discriminant for, in-ICU mortality for patients both with and without infection. These findings add to the growing body of evidence that support the use of qSOFA to deliver low-cost, high-value critical care in resource-limited settings. Future studies should explore the validity of qSOFA in comparison with other mortality prediction models in both medical and surgical populations.