Skip to main content
PMC home page
Acute Medicine & Surgery logoLink to Acute Medicine & Surgery
. 2019 Feb 5;6(2):131–137. doi: 10.1002/ams2.391

Comparative study on quick sequential organ failure assessment, systemic inflammatory response syndrome and the shock index in prehospital emergency patients: single‐site retrospective study

Masahiro Harada 1,2,✉, Takeshi Takahashi 1,2, Yoshio Haga 3, Takeshi Nishikawa 2,4
PMCID: PMC6442700  PMID: 30976438

Abstract

Aim

The quick sequential organ failure assessment (qSOFA) score, shock index (SI), and systemic inflammatory response syndrome (SIRS) criteria are simple indicators for the mortality of patients in the emergency department (ED). These simple indicators using only vital signs might be more useful in prehospital care than in the ED due to their quick calculation. However, these indicators have not been compared in prehospital settings. The aim of the present study is to compare these indicators measured in prehospital care and verify whether the qSOFA score is useful for prehospital triage.

Methods

We undertook a single‐site retrospective study on patients transferred by ambulance to the Kumamoto Medical Center ED (Kumamoto, Japan) between January 2015 and December 2016. We compared areas under the receiver operating characteristic (AUROC) curves of the qSOFA score, SI, and SIRS criteria measured in prehospital care. We also carried out sensitivity and specificity analyses using the Youden index.

Results

A total of 4,827 patients were included in the present study. The AUROC (95% confidence interval) of the qSOFA score for in‐hospital mortality was 0.64 (0.61–0.67), which was significantly higher than those of the SIRS criteria (0.59 [0.56–0.62]) and SI (0.58 [0.54–0.62]). According to the optimal cut‐off values (qSOFA ≥ 2) decided on as the Youden index, the sensitivity of the qSOFA score was 52.3% and its specificity was 69.9%.

Conclusions

The qSOFA score had the highest AUROC among three indicators. However, it might not be practical in actual prehospital triage due to its low sensitivity.

Keywords: Prehospital emergency care, quick sequential organ failure assessment score, shock index, systemic inflammatory response syndrome, triage

Background

The annual report on the status of emergency and rescue in 2016 from the Fire and Disaster Management Agency of Japan1 showed that more than 5 million patients were transferred annually to hospitals by ambulance in Japan, with approximately 9% being severe patients, 41% moderate, and 50% mild. As numerous patients receive triage, initial treatment, or require transfers, a simple scoring tool, such as the quick sequential organ failure assessment (qSOFA) score,2 systemic inflammatory response syndrome (SIRS)3 criteria, or shock index (SI),4 are needed for triage during the limited time in prehospital emergency care. Vital signs were one of the objective indicators available in prehospital triage. The qSOFA score was proposed in the Third International Consensus Definitions for Sepsis and Septic Shock (SEPSIS‐3),2 and the qSOFA score is recommended for use in daily routine sepsis screening.5 This score includes systolic blood pressure ≤100 mmHg, a respiratory rate ≥22 breaths/min, and an altered mental status (one point each; 0–3). The SIRS criteria were proposed in the Definitions for Sepsis and Organ Failure guidelines in 1992.3 This score includes a body temperature <36°C or >38°C, heart rate >90 b.p.m., respiratory rate >20 breaths/min or PaCO2 <32 Torr, and white blood cell count (WBC) <4,000/mm3 or >12,000/mm3 (one point each; 0–4). The SI is commonly used for patients with trauma and hemorrhagic shock and is expressed as heart rate/systolic blood pressure.4

The qSOFA score is superior to the SIRS criteria3 and conventional SOFA score6 for predicting the mortality of patients with suspected infection outside of intensive care unit settings.2 Singer et al.7 suggested that the qSOFA score is associated with mortality in adult patients with and without suspected infection in the emergency department (ED). In recent prehospital studies, a strong relationship was observed between the qSOFA score and mortality in infected patients8; however, its sensitivity for detecting sepsis was low.9 Kitahara et al.10 suggested that the prehospital qSOFA score was more strongly associated with in‐hospital mortality in non‐infected patients than the modified early warning score. However, these simple indicators, such as the qSOFA score, SIRS criteria, and SI, have not been compared for in‐hospital mortality in prehospital patients.

Laboratory tests, such as those for WBC and PaCO2, can be carried out rapidly in the ED. Indicators using only vital signs might be more useful in prehospital care than in the ED. Although the calculation of the SIRS criteria requires WBC, it is an indicator that is more frequently compared to the qSOFA score. The SI is one of the most classical indicators that uses only vital signs. Although the modified early warning score is a relatively new indicator using only vital signs, its calculation is slightly complex in prehospital care. We undertook a single‐site retrospective study, the aim of which was to compare the qSOFA score, SIRS criteria without WBC and PaCO2 (because of the impossibility of a blood examination in prehospital care), and SI for predicting the in‐hospital mortality of patients transferred to the ED by ambulance and also to verify whether the qSOFA score is useful for prehospital triage. The qSOFA score and SIRS criteria have mainly been used for infectious diseases, whereas SI is mainly used for trauma;11 therefore, we undertook a subgroup analysis of infectious diseases and trauma.

Methods

This study was approved by the Institutional Review Board of the National Hospital Organization Kumamoto Medical Center (KMC) (Kumamoto, Japan). A retrospective study was carried out among adult (age ≥ 18 years) patients transferred by ambulance to the KMC ED between January 2015 and December 2016. The KMC is a tertiary emergency hospital in Japan. It has 550 beds, including 6 in the intensive care unit, 4 in the coronary care unit, 44 emergency dedicated beds and 50 in the psychiatric ward. We searched prehospital medical records for vital signs (consciousness, body temperature, systolic blood pressure, heart rate, and respiratory rate) during transfers to the ED. We excluded the following: out‐of‐hospital cardiac arrest patients, patients transferred from other hospitals, patients for whom one or more vital signs were not available, patients not requiring admission, and patients whose outcomes were unknown due to their admission to other hospitals. Data on vital signs were the first vital signs measured on patient contact with emergency medical technicians; data on systolic blood pressure and heart rate were simultaneously measured vital signs. The outcome was in‐hospital mortality. Statistical analyses were undertaken with Stata (version 15.0; Stata Corp., College Station, TX, USA). Univariate analyses of baseline characteristics and their indicators were carried out by the Wilcoxon rank‐sum test or Fisher's exact test. A receiver operating characteristic (ROC) analysis and Delong's method12 were used to compare each indicator. The optimal cut‐off values of each indicator were decided upon as the point of the highest sum of sensitivity and specificity (the Youden index).13

Results

A total of 16,764 patients were transferred to the KMC by ambulance between January 2015 and December 2016 (Fig. 1). Excluded patients included 788 (4.7%) <18 years, 2,467 (14.7%) who were transferred from other hospitals, 372 (2.2%) with cardiac arrest, 6,954 (41.5%) not requiring admission, 182 (1.1%) of unknown outcomes due to their admission to other hospitals, and 1,174 (7.0%) for whom one or more vital signs were unavailable, yielding 4,827 patients (28.8%) who met the inclusion criteria. There were 4,542 survivors (94.1%) and 285 non‐survivors (5.9%). A comparison of non‐survivors and survivors with a univariate analysis revealed that survivors were significantly younger than non‐survivors. Heart rate and respiratory rate were significantly higher in non‐survivors than in survivors, systolic pressure was significantly lower in non‐survivors than in survivors, and the number of patients with an altered mental status was significantly higher in non‐survivors than in survivors (Table 1). The number that conformed to the SIRS criteria and qSOFA score was significantly higher in non‐survivors than in survivors. The SI was also significantly higher in non‐survivors than in survivors. Regarding the area under the ROC curve (AUROC) of each score; the 95% confidence interval was 0.64 (0.61–0.67) for the qSOFA score, 0.59 (0.56–0.62) for the SIRS criteria, and 0.58 (0.54–0.62) for the SI (Fig. 2, Table 2). The AUROC of the qSOFA score was significantly higher than those of the SIRS criteria and SI (Fig. 2, Table 2). In the subgroup analysis of infectious diseases, trauma, and others, the AUROC of the qSOFA score was significantly higher than those of the SIRS criteria in infectious diseases and others (Fig. 2, Table 2). The optimal cut‐off values of each indicator were two points of the qSOFA score, two points of the SIRS criteria, and 0.9 of the SI by the Youden index.13 The sensitivity of the qSOFA score (52.3%) was lower than that of the SIRS criteria (54.7%) and higher than that of the SI (32.3%). The specificity of the qSOFA score (69.9%) was higher than that of the SIRS criteria (59.0%) and lower than that of the SI (82.8%) (Table 3).

Figure 1.

Figure 1

Open in a new tab

Flowchart of the present study including patients transferred by ambulance to the Kumamoto Medical Center ED (Kumamoto, Japan) between January 2015 and December 2016.

Table 1.

Comparison of characteristics, vital signs, systemic inflammatory response syndrome (SIRS) criteria, shock index, and quick sequential organ failure assessment (qSOFA) score between survivors and non‐survivors

Survivors Non‐survivors P‐value
Number of all patients n (%) 4,542 (94.1) 285 (5.9)
Infection n (%) 770 (93.9) 50 (6.1)
Trauma n (%) 620 (97.0) 19 (3.0)
Others n (%) 3,152 (93.6) 216 (6.4)
Age Median (IQR) 76 (60–85) 81 (70–87) <0.001
Sex Male : female 2,272:2,270 150:135 0.427
Body temperature Median (IQR) 36.6 (36.0–37.3) 36.6 (36.0–37.6) 0.761
Systolic blood pressure Median (IQR) 139 (117–161) 130 (106–165) <0.001
Heart rate Median (IQR) 90 (76–105) 94 (80–112) <0.001
Respiratory rate Median (IQR) 20 (20–24) 24 (20–30) <0.001
Altered mental status n (%) 2,244 (49.4) 192 (67.4) <0.001
SIRS criteria w/o WBC Median (IQR) 1 (1–2) 2 (1–2) <0.001
Shock index Median (IQR) 0.64 (0.52–0.82) 0.72 (0.53–1.00) <0.001
qSOFA score Median (IQR) 1 (0–2) 2 (1–2) <0.001
Open in a new tab

IQR, interquartile range; WBC, white blood cell count.

Figure 2.

Figure 2

Open in a new tab

Comparison of receiver operating characteristic curves of the quick sequential organ failure assessment (qSOFA) score, systemic inflammatory response syndrome (SIRS) criteria, and shock index in all patients (A), patients with infectious diseases (B), patients with trauma (C), and patients without infectious diseases or trauma (D).

Table 2.

Comparison of the area under the receiver operating characteristic curve (AUROC) (95% confidence interval [CI]) of the quick sequential organ failure assessment (qSOFA) score, systemic inflammatory response syndrome (SIRS) criteria, and shock index

n AUROC (95% CI) P‐value
(A) qSOFA (B) SIRS (C) Shock index A versus B A versus C B versus C
All patients 4,827 0.64 (0.61–0.67) 0.59 (0.56–0.62) 0.58 (0.54–0.62) <0.001 0.001 0.528
Infection 820 0.66 (0.59–0.73) 0.56 (0.48–0.63) 0.62 (0.53–0.70) 0.008 0.377 0.204
Trauma 639 0.60 (0.48–0.71) 0.51 (0.38–0.63) 0.51 (0.36–0.65) 0.100 0.978 0.295
Others 3,368 0.64 (0.60–0.67) 0.60 (0.56–0.64) 0.57 (0.53–0.61) 0.033 0.004 0.207
Open in a new tab

Table 3.

Comparison of diagnostic accuracy of the quick sequential organ failure assessment (qSOFA) score, systemic inflammatory response syndrome (SIRS) criteria, and shock index (SI) for in‐hospital mortality

Patients n Cut‐off value Sensitivity, % Specificity, %
All 4,827 qSOFA ≥2 52.3 69.9
SIRS (w/o WBC) ≥2 54.7 59.0
SI ≥0.9 32.3 82.8
Infection 820 qSOFA ≥2 66.0 62.7
SIRS (w/o WBC) ≥2 70.0 40.5
SI ≥0.8 58.0 60.5
Trauma 639 qSOFA ≥2 31.6 80.2
SIRS (w/o WBC) ≥2 31.6 72.1
SI ≥0.8 21.1 84.4
Others 3,368 qSOFA ≥2 50.9 69.6
SIRS (w/o WBC) ≥2 53.2 61.0
SI ≥0.9 31.9 83.2
Open in a new tab

Cut‐off values were selected as the point of the highest sum of sensitivity and specificity (the Youden index).

WBC, white blood cell count.

Discussion

We compared three simple and commonly used indicators for in‐hospital mortality. In the present study, qSOFA more strongly predicted in‐hospital mortality as per the AUROC than the SIRS criteria and SI in patients transferred to the ED by ambulance. The ROC curves showed that the qSOFA score had a higher AUROC than the SIRS criteria and SI for infectious diseases, trauma, and others as well as in all patients. Furthermore, when the optimal cut‐off values of each indicator were assessed by the Youden index,13 the sensitivity of the qSOFA score was higher than that of the SI and slightly lower than that of the SIRS criteria; the specificity of the qSOFA score was higher than that of the SIRS and lower than the SI. Although the qSOFA score had the highest ability to predict in‐hospital mortality in ROC analyses, the sensitivity of the qSOFA score for in‐hospital mortality was low.

These three indicators, especially SI and the qSOFA score without the need for laboratory testing, are easy to calculate and convenient to use for triaging patients during the limited time in prehospital care by emergency medical technicians. The SI is a simple score defined as heart rate/systolic blood pressure.4 Although it is primarily used for patients with hemorrhagic shock or trauma,4 it is also used for critically ill patients or those with sepsis.14, 15 Elevations in the SI indicate an increased heart rate or decreased systolic blood pressure. As distributive shock is the most common form of shock among patients, followed by cardiogenic and hypovolemic shock, and obstructive shock is relatively rare,16 we considered the SI to generally be applicable to most patients with shock. In the present study, the SI had the highest specificity and lowest sensitivity for mortality among the three indicators tested. It is useful for circulatory disturbances, including hemorrhagic shock or septic shock, but does not detect any other abnormalities, such as respiratory failure or disturbance of consciousness. Therefore, the SI might be less suitable than the qSOFA score for the triage of prehospital emergency patients. The SIRS criteria (w/o WBC) had the highest sensitivity and lowest specificity for mortality among the three indicators tested. We used the SIRS criteria without WBC because the present study was undertaken in prehospital settings. The ability of the SIRS criteria (w/o WBC) to predict in‐hospital mortality was different from the original SIRS criteria. Although this difference could have influenced the results obtained in the present study, the AUROC was similar for in‐hospital mortality among patients with suspected infections in a previous study.17 Because the SIRS criteria has high sensitivity for admission,18 it might have low specificity for in‐hospital mortality. Therefore, the SIRS criteria could be more useful for predicting admission than in‐hospital mortality. After the publication of SEPSIS‐3,2 some investigators reported that the qSOFA score predicted the mortality of patients in EDs with infections, such as pneumonia,19, 20, 21 or regardless of suspected infections;7, 22 furthermore, the qSOFA score was superior to the SIRS criteria for predicting mortality.23 In this cohort, the AUROC of the qSOFA score to predict in‐hospital mortality was superior to that of the SIRS criteria and SI in prehospital emergency patients and it had similar AUROC for mortality in a previous study.7 A previous meta‐analysis showed that the sensitivity (95%CI) of qSOFA for in‐hospital mortality in hospitalized patients with suspected infection was 0.56 (0.47–0.65) and its specificity (95% CI) was 0.78 (0.71–0.83), indicating that its ability as a predictive marker is poor.24 Sensitivity and specificity for in‐hospital mortality in prehospital patients in the present study were similar to these values. Although the qSOFA score had higher AUROC than the SIRS criteria and SI, its sensitivity for in‐hospital mortality was low. Therefore, the qSOFA score might not contribute to triage for in‐hospital mortality in prehospital emergency patients.

Our study had several limitations. It was a single‐site retrospective study. Furthermore, one or more vital signs were missing in 1,174 patients including 111 (9.5%) non‐survivors. These limitations could have contributed to a selection bias.

Conclusions

The qSOFA score is easy to calculate using only vital signs without the need for laboratory testing and had the highest AUROC among the three simple indicators examined; however, it might be insufficient for triage patients in prehospital settings due to its low sensitivity for in‐hospital mortality.

Disclosure

Approval of the research protocol: This study was approved by the Institutional Review Board of the National Hospital Organization Kumamoto Medical Center in Japan (reference nos. 753 and 768).

Informed consent: N/A.

Registry and the registration no. of the study/trial: UMIN000032365.

Animal studies: N/A.

Conflict of interest: None.

Acknowledgements

We appreciate Professor Jonathan D. Kaunitz (University of California, Los Angeles, CA, USA) for his very helpful advice on our study. Furthermore, we thank the Kumamoto City Fire Bureau, Ariake Fire Department, Yamaga Fire Department, Kikuchi Fire Department and Uki Fire Department for their help with collecting prehospital data.

Funding information

No funding information provided.

References

  • 1. fdma.go.jp [homepage on the internet]. Japan: Fire and Disaster Management Agency; Chapter 1 Kyukyu Hen, Kyukyu Kyujo No Genkyo Heisei 28 Nen Ban [cited Dec 2016]. Available from: http://www.fdma.go.jp/neuter/topics/kyukyukyujo_genkyo/h28/01_kyukyu.pdf.
  • 2. Seymour CW, Liu VX, Iwashyna TJ et al Assessment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis‐3). JAMA 2016; 315: 762–74. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3. Bone RC, Balk RA, Cerra FB et al Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. The ACCP/SCCM Consensus Conference Committee. American College of Chest Physicians/Society of Critical Care Medicine. Chest 1992; 101: 1644–55. [DOI] [PubMed] [Google Scholar]
  • 4. Allgower M, Burri C. “Shock index”. Dtsch. Med. Wochenschr. 1967; 92: 1947–50. [DOI] [PubMed] [Google Scholar]
  • 5. Nishida O, Ogura H, Egi M et al The Japanese clinical practice guidelines for management of sepsis and septic shock 2016 (J‐SSCG 2016). Acute Med. Surg. 2018; 5: 3–89. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6. Vincent JL, Moreno R, Takala J et al The SOFA (Sepsis‐related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis‐Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996; 22: 707–10. [DOI] [PubMed] [Google Scholar]
  • 7. Singer AJ, Ng J, Thode HC Jr, Spiegel R, Weingart S. Quick SOFA scores predict mortality in adult emergency department patients with and without suspected infection. Ann. Emerg. Med. 2017; 69: 475–9. [DOI] [PubMed] [Google Scholar]
  • 8. Vaittinada Ayar P, Delay M, Avondo A et al Prognostic value of prehospital quick sequential organ failure assessment score among patients with suspected infection. Eur. J. Emerg. Med. 2018. 10.1097/MEJ.0000000000000570. [Epub ahead of print] [DOI] [PubMed] [Google Scholar]
  • 9. Tusgul S, Carron PN, Yersin B, Calandra T, Dami F. Low sensitivity of qSOFA, SIRS criteria and sepsis definition to identify infected patients at risk of complication in the prehospital setting and at the emergency department triage. Scand. J. Trauma Resusc. Emerg. Med. 2017; 25: 108. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10. Kitahara O, Nishiyama K, Yamamoto B, Inoue S, Inokuchi S. The prehospital quick SOFA score is associated with in‐hospital mortality in noninfected patients: a retrospective, cross‐sectional study. PLoS ONE 2018; 13: e0202111. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 11. Olaussen A, Blackburn T, Mitra B, Fitzgerald M. Review article: shock index for prediction of critical bleeding post‐trauma: a systematic review. Emerg. Med. Australas. 2014; 26: 223–8. [DOI] [PubMed] [Google Scholar]
  • 12. DeLong ER, DeLong DM, Clarke‐Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics 1988; 44: 837–45. [PubMed] [Google Scholar]
  • 13. Youden WJ. Index for rating diagnostic tests. Cancer 1950; 3: 32–5. [DOI] [PubMed] [Google Scholar]
  • 14. Rady MY. Triage of critically ill patients: an overview of interventions. Emerg. Med. Clin. North Am. 1996; 14: 13–33. [DOI] [PubMed] [Google Scholar]
  • 15. Tseng J, Nugent K. Utility of the shock index in patients with sepsis. Am. J. Med. Sci. 2015; 349: 531–5. [DOI] [PubMed] [Google Scholar]
  • 16. Vincent JL, De Backer D. Circulatory shock. N. Engl. J. Med. 2013; 369: 1726–34. [DOI] [PubMed] [Google Scholar]
  • 17. Raith EP, Udy AA, Bailey M et al Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in‐hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA 2017; 317: 290–300. [DOI] [PubMed] [Google Scholar]
  • 18. Vincent JL, Opal SM, Marshall JC, Tracey KJ. Sepsis definitions: time for change. Lancet 2013; 381: 774–5. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19. Wang JY, Chen YX, Guo SB, Mei X, Yang P. Predictive performance of quick Sepsis‐related Organ Failure Assessment for mortality and ICU admission in patients with infection at the ED. Am. J. Emerg. Med. 2016; 34: 1788–93. [DOI] [PubMed] [Google Scholar]
  • 20. Chen YX, Wang JY, Guo SB. Use of CRB‐65 and quick sepsis‐related organ failure assessment to predict site of care and mortality in pneumonia patients in the emergency department: a retrospective study. Crit. Care 2016; 20: 167. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21. Ranzani OT, Prina E, Menendez R et al New sepsis definition (sepsis‐3) and community‐acquired pneumonia mortality. A validation and clinical decision‐making study. Am. J. Respir. Crit. Care Med. 2017; 196: 1287–97. [DOI] [PubMed] [Google Scholar]
  • 22. Ho KM, Lan NS. Combining quick sequential organ failure assessment with plasma lactate concentration is comparable to standard sequential organ failure assessment score in predicting mortality of patients with and without suspected infection. J. Crit. Care 2017; 38: 1–5. [DOI] [PubMed] [Google Scholar]
  • 23. Park HK, Kim WY, Kim MC, Jung W, Ko BS. Quick sequential organ failure assessment compared to systemic inflammatory response syndrome for predicting sepsis in emergency department. J. Crit. Care 2017; 42: 12–7. [DOI] [PubMed] [Google Scholar]
  • 24. Maitra S, Som A, Bhattacharjee S. Accuracy of quick sequential organ failure assessment (qSOFA) score and systemic inflammatory response syndrome (SIRS) criteria for predicting mortality in hospitalized patients with suspected infection: a meta‐analysis of observational studies. Clin. Microbiol. Infect. 2018; 24: 1123–9. [DOI] [PubMed] [Google Scholar]

Articles from Acute Medicine & Surgery are provided here courtesy of Wiley

RESOURCES