A Prospective Comparison of Quick Sequential Organ Failure Assessment, Systemic Inflammatory Response Syndrome Criteria, Universal Vital Assessment, and Modified Early Warning Score to Predict Mortality in Patients with Suspected Infection in Gabon

Abstract.

The quick sequential organ failure assessment (qSOFA) score has been proposed for risk stratification of emergency room patients with suspected infection. Its use of simple bedside observations makes qSOFA an attractive option for resource-limited regions. We prospectively assessed the predictive ability of qSOFA compared with systemic inflammatory response syndrome (SIRS), universal vital assessment (UVA), and modified early warning score (MEWS) in a resource-limited setting in Lambaréné, Gabon. In addition, we evaluated different adaptations of qSOFA and UVA in this cohort and an external validation cohort from Malawi. We included 279 cases, including 183 with an ad hoc (suspected) infectious disease diagnosis. Overall mortality was 5%. In patients with an infection, oxygen saturation, mental status, human immunodeficiency virus (HIV) status, and all four risk stratification score results differed significantly between survivors and non-survivors. The UVA score performed best in predicting mortality in patients with suspected infection, with an area under the receiving operator curve (AUROC) of 0.90 (95% confidence interval [CI]: 0.78–1.0, P < 0.0001), outperforming qSOFA (AUROC 0.77; 95% CI: 0.63–0.91, P = 0.0003), MEWS (AUROC 0.72; 95% CI: 0.58–0.87, P = 0.01), and SIRS (AUROC 0.70; 95% CI: 0.52–0.88, P = 0.03). An amalgamated qSOFA score applying the UVA thresholds for blood pressure and respiratory rate improved predictive ability in Gabon (AUROC 0.82; 95% CI: 0.68–0.96) but performed poorly in a different cohort from Malawi (AUROC 0.58; 95% CI: 0.51–0.64). In conclusion, UVA had the best predictive ability, but multicenter studies are needed to validate the qSOFA and UVA scores in various settings and assess their impact on patient outcome.

INTRODUCTION

In 2016, the Third International Consensus Definitions for Sepsis and Septic Shock proposed the use of a simplified Sequential Organ Failure Assessment (SOFA) score, termed quickSOFA (qSOFA), for suspected infection outside intensive care to rapidly identify high-risk patients.^1,2 The qSOFA score includes respiratory rate, altered mentation, and systolic blood pressure, which are readily available in any setting. As such, the qSOFA score could be of particular relevance in resource-limited regions. Since the original publication in 2016, a number of articles and reviews were published on the ability of the qSOFA to predict mortality.^3–6 However, few studies to date have been performed in low-resource settings.

In a retrospective study in the Albert Schweitzer Hospital, in Gabon, we previously found that a qSOFA score of 2 or greater had a sensitivity of 87% (95% confidence interval [CI]: 60–98%) and specificity of 75% (95% CI: 70–80%) with an area under the receiver operating curve (AUROC) of 0.83 (95% CI: 0.74–0.93) in patients with suspected infection (including bacterial infection and/or malaria).⁷ In addition, in a prospective observational study in patients with a suspected infection admitted to a tertiary hospital in Malawi, we observed a sensitivity of 72% (95% CI: 62–80%), a specificity of 68% (95% CI: 63–73%), and an AUROC of 0.73 (95% CI: 0.68–0.78).⁸ Other groups retrospectively evaluated the qSOFA score in patients with an infection presenting to the emergency department of a tertiary care hospital in Rwanda⁹ and in patients with suspected community-acquired infections presenting to the emergency department of a regional hospital in Tanzania.¹⁰ In Rwanda, qSOFA had a sensitivity of 36% (95% CI: 29–44%) and a specificity of 83% (95% CI: 79–86%) with an AUROC of 0.70 (95% CI: 0.65–0.75) for inhospital mortality.⁹ In Tanzania, the sensitivity was 59% (95% CI: 41–76%) and specificity was 88% (95% CI: 84–91%) with an AUROC of 0.80 (95% CI: 0.73–0.87).¹⁰ Finally, a multicenter retrospective secondary analysis on qSOFA was performed on nine studies in low- and middle-income countries from 2003 to 2017. The authors observed an AUROC of 0.70 (95% CI: 0.68–0.72), but predictive values differed highly between cohorts and settings.¹¹ This study also demonstrates that qSOFA can be used in a wide range of infections, including Lassa fever, malaria, and dengue.¹¹ Combined, the scarce number of studies available in resource-limited regions illustrates potential for the use of qSOFA, but also high variation in performance, especially regarding sensitivity. Moreover, several studies in high-income settings observed limited sensitivity of qSOFA, so its added value compared with commonly used scores for diagnosis and risk stratification of sepsis, such as the systemic inflammatory response syndrome (SIRS) criteria¹² and modified early warning score (MEWS),¹³ has been questioned.^3–6

More recently, a new risk stratification score, the universal vital assessment (UVA) score, based on data from hospital-based cohort studies in sub-Saharan Africa, and thus potentially more suited to the African setting, was developed. The first article, a retrospective literature study, demonstrates a sensitivity and specificity of 71% and 59%, respectively, with an AUROC of 0.77 (95% CI: 0.75–0.79), which outperformed MEWS (AUROC 0.70 [95% CI: 0.67–0.71]) and qSOFA (AUROC 0.69 [95% CI: 0.67–0.72]).¹⁴ However, this score has not been prospectively evaluated.

Local validation of these scores in resource-limited settings is important to select a risk stratification score that is both practical and accurate. Therefore, we prospectively evaluated the ability of the qSOFA score to predict mortality in patients presenting to the emergency department of the Albert Schweitzer Hospital, and compared the performance of qSOFA with the SIRS criteria, MEWS, and UVA scores. Furthermore, we reanalyzed our data by adapting details of the qSOFA and UVA scores to examine the potential for improving their performance characteristics beyond the limits of the existing score definitions.

METHODS

Study design and population.

We performed a single-center prospective observational study at the emergency department of the Albert Schweitzer Hospital in Lambaréné, Gabon. The Albert Schweitzer Hospital is a 150-bed referral hospital serving an estimated population of 75,500 patients throughout central Gabon. The emergency department is equipped to serve eight patients at a time, including a two-bed high-care unit adjacent to the emergency room. Data were collected from November 2017 to May 2018.

We included all consenting adults visiting the emergency department. The qSOFA, SIRS, MEWS, and UVA risk stratification scores were calculated and compared with one another to assess the ability to predict mortality. These scores were calculated using clinical parameters, obtained prospectively on admission to the emergency department (Table 1). Our primary analysis focused on patients with a suspected infection. Suspected infection was defined on presentation when available clinical, radiological, and laboratory findings suggest a most likely infectious cause (including bacterial infection, malaria, tuberculosis, and viral infections).

Table 1.

Summary of information required for each prognostic scoring system discussed and maximum score allocated for each parameter

	Quick Sequential Organ Failure Assessment	Systemic inflammatory response syndrome	Modified early warning score	Universal vital assessment
Respiratory rate, breaths/minute	1: ≥ 22	1: > 20	0: 9–14	1: ≥ 30
			1: 15–20
			2: 21–29 or < 9
			3: ≥ 30
Systolic blood pressure, mm Hg	1: ≤ 100	–	0: 101–199	1: < 90
			1: 81–100
			2: 71–80 or ≥ 200
			3: ≤ 70
GCS or AVPU	1: GCS < 15	–	0: Alert	4: GCS < 15
			1: Reacts to voice
			2: Reacts to pain
			3: Unresponsive
Temperature, °C	–	1: > 38 or < 36	0: 35–38.4	2: < 36
			2: <35 or ≥ 38.5
Heart rate, beats/minute	–	1: > 90	0: 51–100	1: ≥ 120
			1: 101–110 or 41–50
			2: 111–129 or <40
			3: ≥ 130
White blood cells, 109 g/L	–	1: >12 or <4	–	–
Oxygen saturation	–	–	–	2: < 92%
Source of infection	–	Yes/no	–	–
HIV status	–	–	–	2: seropositive
Maximum score	3 (≥ 2: high risk)	4 (≥ 2 and source of infection meets sepsis criteria)	14 (> 4: high risk)	13 (0–1 low risk, 2–4 medium risk, > 4 high risk

GCS = Glasgow Coma Scale; AVPU = alert, voice, pain, unresponsive.

Outcome.

The primary outcome was the ability of qSOFA to predict mortality in patients visiting the emergency department, specifically for patients with a suspected infection. All patients were followed up to determine the following possible outcomes: discharge, death, or loss to follow-up (due to absconded patients or transfer to another hospital with unknown outcome). Final diagnoses, required hospitalization, inhospital mortality, and length of stay in hospital were recorded from patient files. For patients transferred to a different hospital, the researcher made a phone call to retrieve information about the final outcome of the patient.

Data collection.

All clinical parameters were collected within 12 hours after admission to the emergency department. Most parameters were measured as part of routine care and retrieved from patient files. Missing information was collected in the emergency department and on the wards. In patients with a known HIV status, this was recorded from the medical notes or patient interview. Training was given on the importance and correct measurement of the physiological parameters. Patients who absconded or were transferred to a different hospital received a follow-up phone call to determine their outcome. Study data were anonymized, collected, and managed using Research Electronic Data Capture (REDCap) tools hosted at Centre de Recherches Médicales de Lambaréné (CERMEL). REDCap is a secure, web-based application designed to support data capture for research studies. Data were verified by two separate researchers.

Sample size.

Because of limited data on mortality, and sensitivity and specificity of qSOFA in our study population, a formal sample size calculation could not be performed before the start of the study. An interim analysis in March 2017 revealed that a total number of 3,144 and 6,880 patients would be needed to reliably assess specificity and sensitivity, respectively.¹⁵ As this was not feasible in our setting, we analyzed the available data of this pilot study, which we intend to use as template for a prospective multicenter study.

Data analysis.

Baseline characteristics are expressed as medians with interquartile ranges (IQRs). Comparisons were made using a Mann–Whitney U test for continuous variables and a chi-squared test for dichotomous variables. To assess the predictive ability of risk stratification scores, we analyzed AUROC, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), using a threshold of 2 for the qSOFA score; 2 and 5 for the middle and high UVA scores, respectively; 5 for the MEWS; and 2 for SIRS score. Missing or undetermined white blood cell counts were assumed to be normal in calculating SIRS. Missing information concerning HIV serostatus was recorded as unknown and not awarded a point in the UVA score. This is in accordance with the proposed use of the UVA score.¹⁴ Patients who had an unknown serostatus at the time-point of scoring, but were diagnosed with HIV after admission, were not allocated any UVA score points for HIV status. Patients with missing vital signs were excluded from analysis. All statistical analyses were performed using GraphPad Prism 7 for Mac OS X (version 7.0d; GraphPad Software, La Jolla, CA).

Calculation of an amalgamated qSOFA/UVA score.

Retrospectively, data were reanalyzed to assess candidate amalgamated qSOFA/UVA scores possibly superior with regard to the outcome “best prognostic score” to inform possible further studies. The following options were calculated: 1) qSOFA with respiratory rate and systolic blood pressure (BP) thresholds according to UVA criteria, using a cutoff of ≥ 1 to identify high-risk patients; 2a) original qSOFA + oxygen saturation, 2b) adapted (see [1]) qSOFA + oxygen saturation; and 3) original qSOFA + 1 extra point for altered Glasgow Coma Scale (GCS). Next, we did a retrospective external validation of adapted qSOFA scores using data from a previous study in Malawi.⁸

Ethics.

Ethical approval was obtained from the Institutional Review Board of CERMEL (CEI-CERMEL:014/2017). Informed consent was obtained from all patients. If patients were unable to understand the informed consent procedure because of illness, a legal representative was asked for informed consent. All patient data were kept confidential and only accessible to members of the study team.

RESULTS

During the study period, 277 patients were included in the study. Four patients visited the emergency department twice for the same problem for which the vital signs, corresponding scores, and outcomes were reported as separate cases. One patient was excluded because of missing data and one patient was lost to follow-up, leaving 279 cases for analysis, including 187 cases with an infectious diagnosis. The most common infectious diagnosis was malaria (n = 97), followed by gastrointestinal and intra-abdominal infections (n = 20), respiratory tract infections (n = 17), skin and soft tissue infections (n = 15), pulmonary tuberculosis (n = 13), suspected viral infections (n = 12), and urinary tract infections (n = 8). The most common diagnoses in patients without an infection were musculoskeletal complaints (n = 19), abdominal pain (n = 17), hypertension (n = 14), anemia (n = 8), and heart failure (n = 8). Baseline characteristics are presented in Table 2. The median age of patients was 38 years (IQR 28–53) and 46% (n = 128) were male. Fourteen patients were known to be HIV positive (5%), including one who was diagnosed after admission to the hospital. Sixteen patients died (mortality 5.7%), including 11 with a suspected infection.

Table 2.

Baseline characteristics

	All cases				Cases with infection
	Total, n = 279	Survivors, n = 263	Non-survivors, n = 16	P-value	Total, n = 187	Survivors, n = 176	Non-survivors, n = 11	P-value
Demographics
Gender–male, number (%)	128 (45.9)	121 (46.0)	7 (43.8)	1.0	93 (49.7)	88 (50.0)	5 (45.5)	1.0.
Age in years, median (IQR)	38 (28–53)	38 (27–51)	52 (34–61)	0.069	38 (26–50)	37 (25–49)	58 (35–60)	0.01
Admission, number (%)	134 (48)	118 (44.9)	16 (100)	Not applicable	103 (55.1)	92 (52.3)	11 (100)	Not applicable
Length of stay in days, median (IQR)	4 (3–6)	4 (3–6)	5 (2–12)	0.98	2 (1–5)	2 (1–4)	5 (2–12)	0.03
Clinical parameters
Systolic blood pressure, mmHg, median (IQR)	120 (110–140)	120 (110–140)	105 (80–130)	0.019	120 (110–130)	120 (110–130)	110 (80–130)	0.10
Heart rate, beats/minutes, median (IQR)	91 (80–107)	90 (80–106)	112 (91–120)	0.003	96 (84–111)	96 (84–110)	104 (91–120)	0.09
Respiratory rate, breaths/minutes, median (IQR)	22 (18–26)	22 (18–26)	24 (23–34)	0.018	24 (20–28)	24 (20–28)	24 (24–48)	0.052
Temperature, °C, median (IQR)	37.2 (36.5–38.4)	37.2 (36.6–38.4)	36–4 (35.9–37.7)	0.005	37.8 (36.8–38.7)	37.8 (36.9–38.7)	36.5 (36.0–38.6)	0.07
Saturation, %, median (IQR)	97 (95–99)	98 (95–99)	93 (76–99)	0.028	97 (94–98)	97 (95–98)	92 (80–96)	0.01
White blood cells, 109g/L, median (IQR)*	7.2 (5.4–9.9)	7.1 (5.4–9.7)	10.5 (6.3–15.6)	0.028	7.2 (5.6–10.4)	7.2 (5.5–10.1)	12.0 (6.1–14.9)	0.10
Glasgow Coma Scale <15, number (%)	13 (4.7)	6 (2.3)	7 (43.8)	< 0.0001	10 (5.3)	5 (2.8)	5 (45.5)	< 0.0001
HIV infection, number (%)†	14 (5.0)	11 (4.2)	3 (18.8)	0.038	14 (7.5)	11 (6.3)	3 (27.3)	0.04
Risk stratification scores
qSOFA, median (IQR)	1 (0–1)	1 (0–1)	1.5 (1–3)	< 0.0001	1 (0–1)	1 (0–1)	2 (1–3)	0.0009
qSOFA ≥ 2, number (%)	43 (15.4)	35 (13.3)	8 (50.0)	0.0009	37 (19.8)	31 (17.6)	6 (54.5)	0.009
SIRS, median (IQR)	2 (1–2)	2 (1–2)	2.5 (2–4)	0.0009	2 (1–3)	2 (1–3)	3 (2–4)	0.02
SIRS ≥ 2, number (%)	160 (57.3)	146 (55.5)	14 (87.5)	0.017	128 (68.4)	119 (67.6)	9 (81.8)	0.51
MEWS, median (IQR)	2 (2–4)	2 (1–4)	5 (3–9)	< 0.0001	3 (2–5)	3 (2–5)	5 (3–7)	0.01
MEWS ≥ 5, number (%)	62 (22.2)	52 (19.8)	10 (62.5)	0.0004	56 (29.9)	49 (27.8)	7 (63.6)	0.02
UVA, median (IQR)	0 (0–1)	0 (0–1)	4.5 (3–7)	< 0.0001	0 (0–2)	0 (0–1)	5 (3–7)	< 0.0001
UVA ≥ 2, number (%)	61 (21.9)	47 (17.9)	14 (85.7)	< 0.0001	48 (25.7)	38 (21.6)	10 (90.9)	< 0.0001
UVA ≥ 5, number (%)	13 (4.7)	5 (1.9)	8 (50.0)	< 0.0001	11 (5.9)	5 (2.8)	6 (54.5)	< 0.0001

IQR = interquartile range, MEWS = modified early warning score; qSOFA = quick Sequential Organ Failure Assessment; SIRS = systemic inflammatory response syndrome; UVA = universal vital assessment.

^*Missing values for white blood cell count were 22, all in survivors.

^†HIV positives include one patient who was diagnosed during this admission; all others were previously known to be positive. HIV status was unknown in 259 cases. Only six patients were known to be HIV negative.

As qSOFA is designed for risk stratification in sepsis patients, we focused our analyses on the subgroup of patients with suspected infection (n = 187). In this population, the clinical parameters that were significantly different between survivors and non-survivors were oxygen saturation, altered mental status, and HIV infection. Oxygen saturation was a median of 97% (95% CI: 95–98%) in survivors, compared with 92% (95% CI: 80–96%) in non-survivors (P = 0.01). A reduced GCS was observed in five (2.8%) survivors and in five (45.5%) non-survivors (P < 0.0001). HIV infection was observed in 11 (6.3%) survivors and three (27.3%) non-survivors (P = 0.02). All risk stratification scores assessed in this study were significantly different between survivors and non-survivors (Table 2).

Next, we determined the predictive ability of different risk stratification scores in all cases and in our subgroup of cases with an infectious diagnosis (Table 3). Although we focus our discussion here on cases with an infectious diagnosis, similar results were observed in unselected patients presenting to the emergency room. In patients with an infection, a qSOFA ≥ 2 had a sensitivity of 55% (95% CI: 23–83) and specificity of 82% (95% CI: 76–88). Positive predictive value was 16% and NPV was 97%. The AUROC was 0.77 (95% CI: 0.63–0.91). The qSOFA score was outperformed by the UVA score, with a sensitivity of 91% (95% CI: 59–100) and specificity of 78% (95% CI: 72–84%) for a cutoff value ≥ 2, with a PPV and NPV of 21% and 99%, respectively. When the cutoff was increased to a UVA score ≥ 5, sensitivity dropped to 55% (95% CI: 23–83%), similar to qSOFA, whereas specificity increased to 97% (95% CI: 94–99%), with a PPV and NPV of 55% and 97%, respectively. The AUROC for the UVA score was 0.90 (95% CI: 0.78–1.0). The MEWS and SIRS score performed less well with an AUROC of 0.72 (95% CI: 0.58–0.87) and 0.70 (95% CI: 0.52–0.88), respectively (Table 3).

Table 3.

Predictive ability of risk stratification scores

	Sensitivity, % (95% CI)	Specificity, % (95% CI)	Positive predictive value (%)	Negative predictive value (%)	Area under the receiver operating curve (95% CI)
All cases
Existing risk stratification scores
qSOFA ≥ 2	50 (25–75)	87 (82–91)	19	97	0.79 (0.69–0.89
SIRS ≥ 2	88 (62–98)	44 (38–51)	9	98	0.74 (0.62–0.86)
MEWS ≥ 5	63 (35–85)	80 (75–85)	16	97	0.80 (0.71–0.90)
UVA ≥ 2	88 (62–98)	82 (77–87)	23	99	0.89 (0.78–1.0)
UVA high ≥ 5	50 (25–75)	98 (96–99)	62	97
Cases with an infection
Existing risk stratification scores
qSOFA ≥ 2	55 (23–83)	82 (76–88)	16	97	0.77 (0.63–0.91)
SIRS ≥ 2	82 (48–98)	32 (26–40)	7	97	0.70 (0.52–0.88)
MEWS ≥ 5	64 (31–89)	72 (65–79)	13	97	0.72 (0.58–0.87)
UVA ≥ 2	91 (59–100)	78 (72–84)	21	99	0.90 (0.78–1.0)
UVA high ≥ 5	55 (23–83)	97 (94–99)	55	97
Adapted risk stratification scores
qSOFA (1) ≥ 1	82 (48–98)	80 (73–86)	20	99	0.82 (0.68–0.96)
qSOFA (2a) ≥ 2	73 (39–94)	79 (72–85)	18	98	0.81 (0.69–0.94)
qSOFA (2b) ≥ 1	82 (48–98)	73 (66–79)	50	97	0.83 (0.68–0.98)
qSOFA (3) ≥2	55 (24–83)	82 (76–88)	16	97	0.78 (0.64–0.93)
UVA (1) ≥ 2	82 (48–98)	81 (75–87)	21	99	0.86 (0.70–1.0)
UVA (1) ≥ 5	55 (23–83)	99 (96–100)	75	97
External validation of existing and adapted qSOFA in the Malawi cohort*
qSOFA ≥ 2	72 (62–80)	68 (63–73)	40	50	0.73 (0.68–0.78)
qSOFA (1) ≥ 1	58 (48–67)	53 (48–59)	27	81	0.58 (0.51–0.64)
qSOFA (3) ≥ 2	79 (70–87)	63 (58–68)	39	91	0.77 (0.72–0.82)

CI = confidence interval; qSOFA = quick Sequential Organ Failure Assessment; SIRS = systemic inflammatory response syndrome; MEWS = modified early warning score; UVA = universal vital assessment. qSOFA (1): qSOFA with respiratory rate and systolic BP thresholds according to UVA criteria but no other changes, using a cutoff ≥1 to identify high-risk patients. qSOFA (2a): original qSOFA + oxygen saturation. qSOFA (2b): adapted (see qSOFA [1]) qSOFA + oxygen saturation, using a cutoff ≥1 to identify high-risk patients. qSOFA (3): original qSOFA + 1 extra point for altered GCS. UVA (1): UVA score without incorporating HIV status.

^*In our previous study in Malawi, data on temperature and oxygen saturation were not collected, so we limited our external validation to amalgamated qSOFA (1) and qSOFA (3). Part of the data in this table have been previously published.⁸

Following our primary analysis, we evaluated different amalgamated risk stratification scores, combining different aspects of qSOFA and UVA (Table 3). In Gabon, we observed that the predictive ability of qSOFA, and sensitivity in particular, increased when the thresholds for blood pressure and respiratory rate in qSOFA were changed to those of the UVA (summarized as qSOFA [1] in Table 3), using a cutoff for the score of ≥ 1 to identify high-risk patients. Adding oxygen saturation as a parameter also increased the performance of qSOFA as a predictor of mortality. Allocating more points for an altered mental state did not change the performance of qSOFA in this cohort. We also evaluated the UVA score without taking HIV status into account. This had no effect on the predictive ability of the UVA score. However, this may be because of the low number of HIV-positive cases in our cohort.

Next, we performed a retrospective external validation of qSOFA with respiratory rate and systolic BP thresholds according to UVA criteria, in a cohort of patients admitted with suspected infection in Malawi where we previously evaluated qSOFA.⁸ Baseline characteristics of this cohort are presented in Supplemental Table 1 and demonstrate demographic similarity between the two cohorts but a much higher mortality rate of 23%. The qSOFA score performed reasonably in Malawi with an AUROC of 0.73 (95% CI: 0.68–0.78), which could be increased to 0.77 (95% CI: 0.72–0.82) when an extra point was allocated for altered mental status. Strikingly, in this cohort the amalgamated qSOFA score using UVA thresholds for respiratory rate and blood pressure performed poorly, with a sensitivity of 58% (95% CI: 48–67%), a specificity of 53% (95% CI: 48–59%), and an AUROC of 0.58 (95% CI: 0.51–0.64) (Table 3). This illustrates the heterogeneity in performance of risk stratification scores between different cohorts.

DISCUSSION

We compared the ability to predict inhospital mortality of the qSOFA score, UVA score, MEWS, and SIRS criteria in patients visiting the emergency department in a low-resource setting. Risk stratification scores performed similarly in the entire cohort and in the subgroup with suspected infection, suggesting that these scores may be useful in an unselected patient population, with and without infection, presenting to emergency services. As qSOFA is designed for risk stratification in sepsis, the findings summarized in the following paragraphs represent the data for all cases with an infectious diagnosis.

Our main finding is that the UVA score yielded the best predictive ability with an AUROC of 0.90 (95% CI: 0.78–1.0), followed by qSOFA with an AUROC of 0.77 (95% CI: 0.63–0.91), whereas MEWS and SIRS had lower predictive values in our setting. Quick Sequential Organ Failure Assessment has previously been criticized for its low sensitivity, which was confirmed by our study (sensitivity 55% [95% CI: 23–83%]). Arguably, a risk stratification score should have high sensitivity to avoid overlooking critically ill patients. On the other hand, a score should be sufficiently specific to avoid overburdening of limited health services. The UVA score performed better with a sensitivity of 91% (95% CI: 59–100%), when using a cutoff value of ≥ 2, without losing much specificity (specificity 78% [95% CI: 72–84%], compared with 82% [95% CI: 76–88%] for qSOFA). The NPV was high for all risk stratification scores, whereas the PPV was low, likely related to the low mortality rate in our cohort. The UVA score also had the best PPV, with a PPV of 21% for scores ≥ 2 and a PPV of 55% for scores ≥ 5.

In the study presented here, qSOFA performance was in line with previous studies.^7–9,11 Regarding the UVA score, our study is, to the best of our knowledge, the first to perform a prospective validation of this score. In the original publication, Moore and others¹⁴ describe an AUROC of 0.77 (95% CI: 0.75–0.79). Key differences between qSOFA and UVA are a higher cutoff value for respiratory rate (≥ 30 compared with ≥ 22) and a lower cutoff value for systolic blood pressure (< 90 compared with ≤ 100) in UVA as compared with qSOFA, which may increase specificity. Adding HIV status and saturation, and allocating four points for altered mental status may increase the sensitivity of the UVA score. We further evaluated this by using an amalgamated qSOFA score. Using UVA thresholds for blood pressure and respiratory rate increased the predictive ability of qSOFA to an AUROC of 0.82 (0.68–0.96). However, in a different cohort from Malawi where we previously evaluated qSOFA, performance of qSOFA decreased with this adaptation. Hence, the optimal threshold for blood pressure and respiratory rate remains uncertain and may not be universal. Adding oxygen saturation as a variable to qSOFA also increased performance in Gabon, but we were unable to evaluate this in our Malawi cohort. We previously found that adding two points for altered mental status improved sensitivity of qSOFA in Malawi. However, this was not the case in Gabon, possibly because of a lower number of HIV-positive patients and, thus, fewer intracerebral infections in Gabon.

The HIV prevalence of 5% in our cohort is most likely an underestimate, as we previously observed a prevalence of 20% in hospitalized febrile patients in the same hospital.⁷ Nevertheless, a known HIV status correlated with mortality and, thus, seems to be a useful parameter to incorporate in a risk stratification score. Because of the low number of patients with a known HIV status, we were unable to further examine the impact of HIV status on UVA performance.

Major strengths of our study are the prospective design and presence of study physicians in the emergency department. We included all patients presenting to the emergency room and later stratified our results according to the presence of an infection. As the source of infection is often unclear at presentation, especially in resource-limited settings where diagnostic tools may be unavailable, a risk stratification score should identify critically ill patients regardless of the source of infection. Therefore, we grouped patients with an infection, including bacterial infections, malaria, tuberculosis, and viral infections. Our presence in the emergency department facilitated collection of vital parameters as soon as possible after presentation, before medical interventions had been performed. Other studies, including the pivotal publication by Seymour and others,^2,11 allow for 24 hours between presentation or onset of infection, and collection of vital parameters to calculate qSOFA. We chose a smaller time frame of 12 hours after presentation, which was both realistic and feasible, to assess qSOFA as a tool for triage. Being present also allowed us to observe routine collection of clinical parameters. Although most parameters were collected by local staff, the respiratory rate was often missing. After training was given and a stopwatch was donated, the measurement frequency of the respiratory rate increased, but it remained hard to incorporate it into standard practice of local staff. This illustrates the challenges of incorporating a risk stratification score in routine clinical practice. Finally, we were able to use a previous cohort for external validation of an amalgamated qSOFA score. This allowed us to illustrate the differences between settings and highlights the need for multicenter studies.

Our study was limited by the number of patients included. Therefore, we were unable to perform a statistically significant comparison between risk stratification scores, and CIs were wide. Our results, thus, need to be interpreted with caution and we recommend further prospective validation studies of both qSOFA and the UVA scores in low-resource settings to determine which score has the best predictive ability, and good applicability in clinical practice. Importantly, the differences we observed between Gabon and Malawi demonstrate that it is vital to perform multicenter studies on this subject.

CONCLUSION

We report here one of few prospective studies on qSOFA in resource-limited settings and the first prospective evaluation of the UVA score. The UVA score was designed based on studies from sub-Saharan Africa and outperformed qSOFA, SIRS, and MEWS in ability to predict mortality in our cohort. Multicenter studies are needed to validate qSOFA and the UVA score and variations thereof as suggested, in various settings, and assess whether the use of these scores can improve patient outcomes in resource-limited settings by rapid diagnosis and intervention for sepsis.

Note: Supplemental table appears at www.ajtmh.org.