Abstract
Background
In Sweden, the rapid emergency triage and treatment system (RETTS-A) is used in the pre-hospital setting. With RETTS-A, patients triaged to the lowest level could safely be referred to a lower level of care. The national early warning score (NEWS) has also shown promising results internationally. However, a knowledge gap in optimal triage in the pre-hospital setting persists. This study aimed to evaluate RETTS-A performance, compare RETTS-A with NEWS and NEWS 2, and evaluate the emergency medical service (EMS) nurse’s field assessment with the physician’s final hospital diagnosis.
Methods
A prospective, observational study including patients (≥16 years old) transported to hospital by the Gothenburg EMS in 2016. Three comparisons were made: 1) Combined RETTS-A levels orange and red (high acuity) compared to a predefined reference emergency, 2) RETTS-A high acuity compared to NEWS and NEWS 2 score ≥ 5, and 3) Classification of pre-hospital nurse’s field assessment compared to hospital physician’s diagnosis. Outcomes of the time-sensitive conditions, mortality and hospitalisation were examined. The statistical tests included Mann–Whitney U test and Fisher’s exact test, and several binary classification tests were determined.
Results
Overall, 4465 patients were included (median age 69 years; 52% women). High acuity RETTS-A triage showed a sensitivity of 81% in prediction of the reference patient with a specificity of 64%. Sensitivity in detecting a time-sensitive condition was highest with RETTS-A (73%), compared with NEWS (37%) and NEWS 2 (35%), and specificity was highest with NEWS 2 (83%) when compared with RETTS-A (54%). The negative predictive value was higher in RETTS-A (94%) compared to NEWS (91%) and NEWS 2 (92%). Eleven per cent of the final diagnoses were classified as time-sensitive while the nurse’s field assessment was appropriate in 84% of these cases.
Conclusions
In the pre-hospital triage of EMS patients, RETTS-A showed sensitivity that was twice as high as that of both NEWS and NEWS 2 in detecting time-sensitive conditions, at the expense of lower specificity. However, the proportion of correctly classified low risk triaged patients (green/yellow) was higher in RETTS-A. The nurse’s field assessment of time-sensitive conditions was appropriate in the majority of cases.
Keywords: Triage, Emergency medical services, Pre-hospital emergency nurse, Field assessment, Patient safety
Background
Triage is commonly performed at most emergency departments (ED) in order to stratify the patients based on the severity of their conditions, when the demand exceeds the available resources [1]. The triage process, regarded as the first step in medical screening, is aimed at caring first for the patients with the most critical condition [2]. With the introduction of registered nurses (RN) trained in pre-hospital triage, in the emergency medical services (EMS) in Sweden, the EMS nurses are tasked to decide on the level of care or on whether to bypass the ED for specific patient groups with myocardial infarction (MI), stroke or sepsis, for example. The EMS in Gothenburg, Sweden uses the same triage system, (RETTS-A the five-level rapid emergency triage and treatment system for adults, [RETTS-A]) as the ED, in order to start the triage process at an earlier stage and to support the EMS nurse in the decision-making process. The Swedish RETTS-A has been evaluated at the ED, where it was used to discriminate between the severity of patients’ conditions on admission and in-hospital mortality and is regarded as a reliable triage method [3]. However, it revealed lower accuracy for mortality in the elderly than in younger patients [4]. There is a lack of evidence relating to the triage systems used for all patient presentations in the EMS [5], and only a few studies have evaluated more complex triage systems. In a previous Swedish pre-hospital study of RETTS-A with non-urgent conditions triaged to the lowest level, these patients remained at a lower level of care after consultation with a general practitioner; indicating that it is possible to reduce transportation to the ED [6]. A study from Taiwan reported a better performance for predicting hospitalisation and medical resource consumption with a five-level pre-hospital triage, compared with a two-level triage conducted by emergency medical technicians (EMT) [7]. Moreover, there is evidence to support the five-level systems instead of the lower level systems [2]. In an ED in Norway, RETTS-A displayed a superior performance in detecting sepsis than the quick sepsis-related organ failure assessment (qSOFA) score [8]. Another system based on vital signs (VS), that has attracted interest in the EMS, is the national early warning score (NEWS). Both NEWS and its latest version, NEWS 2, have been studied in the pre-hospital settings. High NEWS scores are associated with adverse outcomes and predict the risk of early death [9–13]. NEWS has also performed better in hospital studies comparing several other early warning scores for short-term mortality, cardiac arrest and the need for intensive care [14]. However, the performance of RETTS-A in a pre-hospital setting in an unselected patient population is unknown. This study aimed to 1) evaluate the performance of RETTS-A, 2) compare the performance of RETTS-A with that of NEWS and NEWS 2, and, finally, 3) evaluate the EMS nurse’s field assessment with the physician’s final hospital diagnosis.
Methods
Design
This was a prospective observational study with a retrospective analysis of patients ≥16 years of age who were assessed by the EMS nurse at the scene, triaged according to RETTS-A, and transported to the hospital. Prospectively, we informed and conducted repetitive training with the EMS staff on RETTS-A triage, at all the nine ambulance stations in the Gothenburg EMS before the study started.
Setting
The study was carried out in the Gothenburg EMS area, Sweden. This is an urban setting with short transportation times and three hospitals, including one level-1 trauma unit within the catchment area. The EMS covers an area of approximately 900 km2 and is inhabited by 660,000 persons (study year). Annually, the EMS assignments exceed 80,000 and, of these, 58,575 assignments are defined as primary missions with an initial patient assessment. The EMS study organisation operates with a differentiated fleet consisting of 22 units, including 18 advanced lifesaving (ALS) ambulance types, two single responders, one physician-manned unit and one scene-commanding unit. According to Sweden legislation, at least one RN is required to staff the ambulance. The majority of all RNs in the study organisation have a postgraduate education specialising in pre-hospital emergency care. When a patient dials the Swedish emergency number (112), the operator assesses the patient according to a dispatch medical index (DMI) and dispatches a unit with a priority depending on the assessed level of severity. Priority 1 is considered time-sensitive and a unit responds with lights and sirens, priority 2 is acute but not assessed as a time-sensitive condition while priority 3 comprises non-urgent assignments. Priority 4 means assignments where there is no need for medical treatment or monitoring. ALS ambulances responding to priority 1–3 and priority 4 cases are carried out by transportation vehicles staffed by EMTs.
Study population
Monthly, data on the first thousand patients in a 1-year period (2016) were collected as a consecutive convenient sample. Patients were eligible for inclusion if they dialed the emergency number (112) and a unit was dispatched to the scene. The inclusion criteria were: 1) eligible for adult ED (≥16 years old) and 2) assessed at the scene by the EMS nurse and triaged according to RETTS-A. The exclusion criteria were: 1) non-conveyed patients, 2) inter-hospital transportations, 3) assistance to other ambulances, and 4) assignments with no patient contact. A total of 8019 patients were reviewed manually and, of these, 5340 adult patients were initially assessed as needing to be sent to the hospital. Patients with contact on multiple occasions were randomly excluded, making a total number of 4760 individual patients. When calculating NEWS and NEWS 2 scores, which were calculated retrospectively, another 145 patients with between four to all six VS missing, were excluded. For the remainder, if three or less VS missing (respiratory rate 1,7%; oxygen saturation 0,5%; heart rate 0,2%; systolic blood pressure 1,8%; body temperature 9,6%; level of consciousness 0,3%), they were regarded as being mostly at random (MAR) and multivariate imputation via chained equations (MICE) was performed. Another 150 patients that fulfilled all the inclusion criteria but left the ED before being seen or against medical advice and the outcome for these patients could therefore not be evaluated were excluded. Overall, 4465 unique patients were transported to the hospital with full (imputed) records of VS included (Fig. 1). The retrospective data were retrieved from EMS and hospital medical records. Each record was reviewed manually, and anonymised data such as VS, triage level, EMS nurse’s assessment and diagnosis code were then entered into a registry for calculation and statistical analysis.
Fig. 1.
Flow chart of the studied patients
RETTS-A triage system
RETTS-A is a five-level triage system that was developed in the ED at Sahlgrenska University Hospital, Gothenburg, Sweden, in 2005 and was introduced in the EMS in 2010. RETTS-A is maintained, further developed and licensed by a Swedish company (Predicare AB) and consists of 53 flowcharts of the most common ED presentations with annual updates. Each flowchart comprises several emergency signs and symptoms (ESS), which yield a triage colour based on the severity, in combination with the VS, which are recorded in all presentations. RETTS-A colours represent the following levels of severity: red (life-threatening) and orange (potentially life-threatening), which are both defined as acute processes directly; and yellow and green, which both mean ‘can wait’ although yellow is considered to require greater urgency than green. Patients triaged to the lowest level (blue) can be managed at a lower level of care than at the ED. At the time of the study, only levels red to green were used in the EMS organisation.
Definition of time-sensitive condition, complications and the reference patient
A time-sensitive condition is said to occur when a patient has a final diagnosis of, for example, stroke, MI or septic shock, and these conditions require prompt pre-hospital management and limited waiting time at the hospital (Additional file 1). The occurrence of complications was measured from the time of the EMS nurse’s assessment up to 48 h in the hospital. Complication was defined as any of the following occurrences: death, cardiac arrest, ventricular arrhythmias, status epilepticus, severe heart failure, hypotension, syncope and unconsciousness. It also includes deviating VS such as respiratory rate > 30 or < 8/min; oxygen saturation < 90%; heart rate > 130 rate/min (regular) or > 150 rate/min (irregular); systolic blood pressure of < 90 mmHg; body temperature > 41 °C. The reference patient in this study, indicating an ‘emergent’ patient was defined as a deviating VS in accordance with a NEWS score of ≥5; a NEWS score of 3 in a single VS; or a time-sensitive condition. The RETTS-A orange or red group (acute process directly) was then compared with the reference patient, and either of the triage levels would qualify as a true positive (Fig. 2).
Fig. 2.
Definition of the reference patient
Outcomes compared with RETTS-A, NEWS, and NEWS 2
Several outcome measurements have been used to compare RETTS-A, NEWS, and NEWS 2 including time-sensitive condition, occurrence of complications within 48 h, admission to in-patient care, 48-h mortality and 30-day mortality (Fig. 3). The RETTS-A and NEWS definitions of an acute patient were used to compare systems as follows: RETTS-A, orange/red level; NEWS, a score of 5 and above or a score of 3 in a single VS; NEWS 2, a score of 5 and above. In NEWS 2, only the standard scale for saturation was used, as recommended [15].
Fig. 3.
RETTS-A, NEWS and NEWS 2 categorisation in accordance with sensitivity, specificity and predictive values
EMS nurse’s field assessment compared with the final hospital diagnosis
In the assessment of the patient, in addition to the triage level, the EMS nurse registers the pre-hospital-assessed condition or symptoms. To categorise the EMS nurse’s assessment and relate it to the final hospital diagnosis, a classification instrument was used. The classification instrument comprises five different categories (A–E), which are related to the final diagnosis. Category A, B, C, and D correspond to a disease defined as: a time-sensitive diagnosis (stroke); a non time-sensitive diagnosis (bronchitis); a final diagnosis expressed as a symptom (non-specified chest pain); and a non-specified final diagnosis (asthenia). Only categories A–D were used, as category E only includes non-conveyed patients [16, 17]. When there was difficulty in classifying a case, it was discussed in a group consisting of a physician and senior researchers.
Statistical analysis
The results in the study are presented as numbers, percentages or the median with percentiles (25th, 75th). In Table 1, results of group comparisons are shown by Fisher’s exact test for binary variables and the Mann–Whitney U test for continuous/ordinal variables. In Table 2, the results are presented as absolute risk (AR) and relative risk (RR). The numbers in Tables 2, 3, and 4 are presented with the corresponding confidence intervals (CIs), which has been bootstrapped 10,000 times. In Table 3, age stratification was determined by the median age of all patients in Sweden, in contact with the EMS (national pre-hospital reports, 2019). When calculating sensitivity, specificity, positive predicted value (PPV), negative predictive value (NPV), likelihood ratios and area under the receiver operating characteristic curve (AUROC) in Tables 3 and 4; RETTS-A levels (orange and red) were combined to form a 2 × 2 table and regarded as a positive test when matching the predefined reference patient. In Table 3, under-triage and over-triage were defined as 1-Sensitivity (proportion of yellow/green triaged patients among all the ‘true emergencies’) and 1-Specificity (proportion of orange/red triaged patients among all the defined ‘non-emergencies’). In Table 4, the NPV was near 1.0, and the Clopper–Pearson CI was used instead. All tests are two-sided and p-values < 0.01 with the 99% CI are considered significant due to the number of tests. R Studio version 1.1.463 (RStudio Inc., Boston, MA) was used to perform the data processing and the statistical analysis.
Table 1.
Patient characteristics, EMS assessment and hospital assessment for patients transported to hospital
| Total | Red | Orange | Yellow | Green | P1 | |
|---|---|---|---|---|---|---|
| n = 4465 | n = 596 | n = 1588 | n = 1919 | n = 362 | ||
| Age – years (25th, 75th percentiles) | ||||||
| Median | 69 (45,83) | 71 (47,83) | 66 (42,81) | 71 (47,84) | 71 (48,85) | < 0.001 |
| Sex – n (%) | < 0.001 | |||||
| Female | 2319 (51.9) | 278 (46.6) | 783 (49.3) | 1040 (54.2) | 218 (60.2) | |
| Dispatcher priority – n (%) (2,5,9,2)2 | < 0.001 | |||||
| Priority 1 | 2218 (49.9) | 455 (76.6) | 926 (58.5) | 752 (39.4) | 85 (23.6) | |
| Priority 2 | 2059 (46.3) | 134 (22.6) | 628 (39.7) | 1067 (55.9) | 230 (63.9) | |
| Priority 3 | 170 (3.8) | 5 (0.8) | 29 (1.8) | 91 (4.8) | 45 (12.5) | |
| Dispatch medical index3 - n (%) (3,7,9,3)2 | ||||||
| Chest pain/cardiac disease | 651 (14.7) | 94 (15.9) | 250 (15.8) | 273 (14.3) | 34 (9.5) | 0.034 |
| Extremity/wound/minor trauma | 609 (13.7) | 12 (2.0) | 191 (12.1) | 337 (17.6) | 69 (19.2) | < 0.001 |
| Abdominal/urinary tract symptoms | 519 (11.7) | 40 (6.7) | 135 (8.5) | 312 (16.3) | 32 (8.9) | < 0.001 |
| Uncertain information/suspicion of severe illness | 476 (10.7) | 47 (7.9) | 168 (10.6) | 210 (11.0) | 51 (14.2) | 0.089 |
| Respiratory difficulties | 462 (10.4) | 152 (25.6) | 137 (8.7) | 136 (7.1) | 37 (10.3) | < 0.001 |
| Time of day – n (%) | 0.001 | |||||
| 08:00–16:00 | 2092 (46.9) | 274 (46.0) | 686 (43.2) | 930 (48.5) | 202 (55.8) | |
| 16:00–24:00 | 1547 (34.6) | 199 (33.4) | 606 (38.2) | 642 (33.5) | 100 (27.6) | |
| 24:00–08:00 | 826 (18.5) | 123 (20.6) | 296 (18.6) | 347 (18.1) | 60 (16.6) | |
| Time on scene – min (25th, 75th percentile) | ||||||
| Median | 22 (15,30) | 25 (18,33) | 22 (15,30) | 21 (14,29) | 21 (14,31) | < 0.001 |
| Medical history4 – n(%) | ||||||
| Diseases of the circulatory system I00-I99 | 4459 (30.2) | 715 (33.1) | 1593 (30.0) | 1846 (29.8) | 305 (27.9) | 0.008 |
| Mental and behavioural disorders F01-F99 | 2222 (15.1) | 333 (15.4) | 873 (16.4) | 861 (13.9) | 155 (14.2) | 0.002 |
| Endocrine, nutritional and metabolic diseases E00-E89 | 1338 (9.1) | 198 (9.2) | 447 (8.4) | 594 (9.6) | 99 (9.1) | 0.186 |
| Diseases of the musculoskeletal system and connective tissue M00-M99 | 1014 (6.9) | 101 (4.7) | 348 (6.6) | 457 (7.4) | 108 (9.9) | < 0.001 |
| Diseases of the digestive system K00-K95 | 878 (6.0) | 108 (5.0) | 299 (5.6) | 415 (6.7) | 56 (5.1) | 0.008 |
| No medical history | 569 (12.7) | 59 (9.9) | 211 (13.3) | 249 (13.0) | 50 (13.8) | 0.455 |
| Prehospital field assessment according to RETTS-A5 – n (%) | ||||||
| Abdominal/flank pain | 460 (10.3) | 36 (6.0) | 117 (7.4) | 289 (15.1) | 18 (5.0) | < 0.001 |
| Chest/thoracic pain | 456 (10.2) | 65 (10.9) | 189 (11.9) | 193 (10.1) | 9 (2.5) | 0.003 |
| Respiratory distress/dyspnoa/breathing difficulties | 382 (8.6) | 129 (21.6) | 117 (7.4) | 103 (5.4) | 33 (9.1) | < 0.001 |
| Unspecific condition | 309 (6.9) | 20 (3.4) | 42 (2.6) | 192 (10.0) | 55 (15.2) | < 0.001 |
| Injury head trauma | 274 (6.1) | 9 (1.5) | 142 (8.9) | 105 (5.5) | 18 (5.0) | 0.039 |
| Prehospital medication – n (%) | ||||||
| Any medication | 1565 (35.1) | 399 (66.9) | 653 (41.1) | 472 (24.6) | 41 (11.3) | < 0.001 |
| Intravenous medication | 889 (19.9) | 227 (38.1) | 367 (23.1) | 279 (14.5) | 16 (4.4) | < 0.001 |
| Management ED – n (%)6 | ||||||
| Admitted to in-patient care | 2287 (51.2) | 483 (81.0) | 866 (54.5) | 826 (43.0) | 112 (30.9) | < 0.001 |
| Extended examination – intervention ⟹ X-ray, CT, US, MR, LP | 687 (15.4) | 26 (4.4) | 300 (18.9) | 305 (15.9) | 56 (15.5) | 0.405 |
| Lab, drug administration, prescription | 1227 (27.5) | 80 (13.4) | 364 (22.9) | 657 (34.2) | 126 (34.8) | < 0.001 |
| Clinical exam observation only | 219 (4.9) | 7 (1.2) | 45 (2.8) | 112 (5.8) | 55 (15.2) | < 0.001 |
| Patient managed by ED nurse, referral to primary care | 45 (1.0) | 0 (0.0) | 13 (0.8) | 19 (1.0) | 13 (3.6) | 0.007 |
| Under the influence of substances (alcohol, drugs) – n (%) | 398 (8.9) | 67 (11.2) | 216 (13.6) | 91 (4.7) | 24 (6.6) | < 0.001 |
| Days of in-patient care – n | ||||||
| Mean (SD) | 7.97 (9.15) | 8.53 (10.35) | 7.48 (8.92) | 8.06 (8.76) | 8.74 (8.00) | 0.463 |
| Median (25th, 75th percentile) | 5 (2,11) | 6 (3,10) | 5 (2,10) | 6 (3,11) | 6 (3,13) | 0.042 |
| Final hospital assessment, ICD-10 codes – n (%) (8,59,62,24)2 | ||||||
| (R) Symptoms, signs and abnormal clinical and laboratory findings | 851 (19.7) | 57 (9.7) | 305 (19.9) | 436 (23.5) | 53 (15.7) | < 0.001 |
| (S,T) Injury, poisoning and certain other consequnces of external causes | 828 (19.2) | 53 (9.0) | 333 (21.8) | 379 (20.4) | 63 (18.6) | 0.113 |
| (I) Diseases of the circulatory system | 550 (12.8) | 148 (25.2) | 196 (12.8) | 178 (9.6) | 28 (8.3) | < 0.001 |
| (J) Diseases of the respiratory system | 351 (8.1) | 115 (19.6) | 133 (8.7) | 94 (5.1) | 9 (2.7) | < 0.001 |
| (F) Mental, behavioural and neurodevelopmental disorders | 301 (7.0) | 53 (9.0) | 110 (7.2) | 100 (5.4) | 38 (11.2) | 0.073 |
| All-cause mortality – n (%) | ||||||
| ≤ 7 days | 88 (2.0) | 39 (6.5) | 33 (2.1) | 16 (0.8) | 0 (0.0) | < 0.001 |
| ≤ 30 days | 209 (4.7) | 73 (12.2) | 80 (5.0) | 50 (2.6) | 6 (1.7) | < 0.001 |
| ≤ 365 days | 705 (15.8) | 155 (26.0) | 237 (14.9) | 264 (13.8) | 49 (13.5) | < 0.001 |
1 P-values calculated on red/orange and yellow/green groups respectively
2 Missing patients in each triage colour category respectively
3 The five most common dispatcher assessments according to dispatch medical index
4 Past medical history – one patient can have multiple diagnoses within the same category
5 The five most common prehospital assessments according to RETTS-A
6 One patient per category, descending order. CT computed tomography, US ultrasound, MR magnetic resonance, LP lumbar puncture
Table 2.
Risk prediction of outcomes in EMS RETTS-A triage levels
| High acuity | Low acuity | ||||
|---|---|---|---|---|---|
| Red | Orange | Yellow | Green | ||
| nt | 596 | 1588 | 1919 | 362 | |
| Time-sensitive condition | n | 153 | 193 | 116 | 11 |
| AR | 25.7 | 12.2 | 6.04 | 3.01 | |
| RR | 3.10 | 1.25 | 0.43 | 0.27 | |
| CI | [2.44,3.89] | [0.99,1.57] | [0.33,0.56] | [0.09,0.50] | |
| Deviating VS/complication within 48 h | n | 303 | 110 | 39 | 3 |
| AR | 50.8 | 6.92 | 2.03 | 0.83 | |
| RR | 12.94 | 0.58 | 0.12 | 0.08 | |
| CI | [10.5,16.5] | [0.43,0.75] | [0.08,0.18] | [0.02,0.33] | |
| Admission to in-patient care | n | 483 | 866 | 826 | 112 |
| AR | 81.0 | 54.5 | 43.0 | 30.9 | |
| RR | 1.74 | 1.10 | 0.75 | 0.58 | |
| CI | [1.62,1.86] | [1.02,1.19] | [0.69,0.81] | [0.47,0.71] | |
| 48 h mortality | n | 22 | 18 | 4 | 0 |
| AR | 3.69 | 1.13 | 0.21 | 0 | |
| RR | 6.49 | 1.25 | 0.13 | 0 | |
| CI | [2.93,14.75] | [0.52,2.81] | [0.03,0.50] | [0,0] | |
| 30-day mortality | n | 73 | 80 | 50 | 6 |
| AR | 12.8 | 5.04 | 2.6 | 1.66 | |
| RR | 3.48 | 1.12 | 0.42 | 0.34 | |
| CI | [2.41,4.88] | [0.78,1.60] | [0.27,0.62] | [0.05,0.74] | |
nt Total number of patients per triage level, n number of patients, AR absolute risk %, RR relative risk, CI Bootstrapped 99% confidence intervals
Table 3.
Prehospital triage according to RETTS-A compared with a reference patient
| All | Female | Male | ≤ 65 yrs | > 65 yrs | |
|---|---|---|---|---|---|
| n | 4465 | 2319 | 2146 | 2034 | 2431 |
| Sensitivity | 0.806 | 0.782 | 0.830 | 0.867 | 0.774 |
| [0.778,0.834]a | [0.740,0.824] | [0.791,0.867] | [0.826,0.906] | [0.737,0.810] | |
| Specificity | 0.644 | 0.668 | 0.616 | 0.585 | 0.702 |
| [0.621,0.666] | [0.638,0.699] | [0.583,0.648] | [0.553,0.618] | [0.672,0.732] | |
| PPV | 0.486 | 0.477 | 0.495 | 0.379 | 0.586 |
| [0.459,0.515] | [0.439,0.516] | [0.457,0.534] | [0.342,0.418] | [0.548,0.622] | |
| NPV | 0.888 | 0.888 | 0.889 | 0.938 | 0.851 |
| [0.871,0.905] | [0.864,0.910] | [0.863,0.913] | [0.918,0.956] | [0.825,0.876] | |
| LR + | 2.26 | 2.36 | 2.16 | 2.09 | 2.60 |
| [2.11,2.43] | [2.13,2.63] | [1.96,2.38] | [1.91,2.30] | [2.32,2.90] | |
| LR - | 0.30 | 0.33 | 0.28 | 0.23 | 0.32 |
| [0.26,0.35] | [0.26,0.39] | [0.22,0.34] | [0.16,0.30] | [0.27,0.38] | |
| AUROCb | 0.725 | 0.725 | 0.723 | 0.726 | 0.738 |
| [0.707,0.745] | [0.699,0.751] | [0.698,0.748] | [0.701,0.752] | [0.714,0.762] | |
| Accuracy | 0.692 | 0.700 | 0.683 | 0.649 | 0.727 |
| [0.674,0.710] | [0.675,0.725] | [0.656,0.709] | [0.621,0.676] | [0.704,0.751] | |
| Overtriage | 0.356 | 0.332 | 0.384 | 0.415 | 0.298 |
| [0.334,0.378] | [0.304,0.362] | [0.351,0.417] | [0.383,0.446] | [0.269,0.329] | |
| Undertriage | 0.194 | 0.218 | 0.170 | 0.133 | 0.226 |
| [0.167,0.222] | [0.176,0.261] | [0.133,0.209] | [0.094,0.176] | [0.189,0.263] |
PPV positive predictive value, NPV negative predictive value, LR Likelihood ratio, AUROC Area under the reciever operating characteristic curve
a Bootstrapped 99% confidence intervals
b Calculated on AUROC curve predefined sensitivity and specificity based on cutoffs in RETTS-A triage levels Red and Orange
Table 4.
Comparison of outcome measurements between RETTS-a combined Red/Orange level and NEWS medium/high and NEWS medium/high
| RETTS-A | NEWS | NEWS 2a | ||
|---|---|---|---|---|
| Time-sensitive condition | n = 473 | |||
| Se | 0.732 | 0.370 | 0.345 | |
| [0.679,0.782]b | [0.313,0.429] | [0.290,0.402] | ||
| Sp | 0.540 | 0.789 | 0.833 | |
| [0.519,0.560] | [0.772,0.805] | [0.818,0.848] | ||
| PPV | 0.158 | 0.172 | 0.197 | |
| [0.139,0.179] | [0.141,0.203] | [0.161,0.232] | ||
| NPV | 0.944 | 0.914 | 0.915 | |
| [0.932,0.956] | [0.900,0.926] | [0.902,0.927] | ||
| ACC | 0.560 | 0.744 | 0.781 | |
| [0.541,0.579] | [0.727,0.761] | [0.765,0.797] | ||
| AUROCc | 0.636 | 0.579 | 0.589 | |
| [0.607,0.662] | [0.549,0.609] | [0.560,0.619] | ||
| Deviating VS/complication within 48 h | n = 455 | |||
| Se | 0.908 | 0.774 | 0.636 | |
| [0.870,0.941] | [0.723,0.822] | [0.581,0.691] | ||
| Sp | 0.558 | 0.834 | 0.873 | |
| [0.538,0.578] | [0.818,0.849] | [0.860,0.887] | ||
| PPV | 0.189 | 0.345 | 0.389 | |
| [0.168,0.211] | [0.307,0.384] | [0.347,0.433] | ||
| NPV | 0.982 | 0.970 | 0.964 | |
| [0.974,0.989] | [0.962,0.977] | [0.955,0.972] | ||
| ACC | 0.594 | 0.828 | 0.857 | |
| [0.575,0.613] | [0.813,0.841] | [0.843,0.870] | ||
| AUROC | 0.733 | 0.804 | 0.792 | |
| [0.713,0.753] | [0.777,0.830] | [0.762,0.820] | ||
| Admission | n = 2287 | |||
| Se | 0.590 | 0.339 | 0.301 | |
| [0.564,0.616] | [0.314,0.365] | [0.276,0.327] | ||
| Sp | 0.617 | 0.888 | 0.935 | |
| [0.590,0.643] | [0.871,0.906] | [0.921,0.948] | ||
| PPV | 0.618 | 0.762 | 0.829 | |
| [0.590,0.644] | [0.727,0.795] | [0.794,0.862] | ||
| NPV | 0.589 | 0.562 | 0.560 | |
| [0.562,0.615] | [0.540,0.583] | [0.539,0.581] | ||
| ACC | 0.603 | 0.607 | 0.610 | |
| [0.585,0.622] | [0.588,0.626] | [0.591,0.629] | ||
| AUROC | 0.603 | 0.614 | 0.618 | |
| [0.585,0.622] | [0.598,0.629] | [0.604,0.632] | ||
| 48 h mortality | n = 44 | |||
| Se | 0.909 | 0.727 | 0.727 | |
| [0.795,1.000] | [0.545,0.886] | [0.545,0.886] | ||
| Sp | 0.515 | 0.777 | 0.819 | |
| [0.496,0.534] | [0.760,0.793] | [0.804,0.834] | ||
| PPV | 0.018 | 0.031 | 0.039 | |
| [0.011,0.026] | [0.019,0.046] | [0.023,0.057] | ||
| NPV | 0.998 | 0.997 | 0.997 | |
| [0.996,1.000] | [0.994,0.999] | [0.994,0.999] | ||
| ACC | 0.519 | 0.776 | 0.818 | |
| [0.499,0.538] | [0.760,0.792] | [0.804,0.833] | ||
| AUROC | 0.712 | 0.752 | 0.773 | |
| [0.646,0.759] | [0.661,0.831] | [0.681,0.855] | ||
| 30-day mortality | n = 209 | |||
| Se | 0.732 | 0.612 | 0.536 | |
| [0.651,0.809] | [0.526,0.699] | [0.445,0.627] | ||
| Sp | 0.523 | 0.791 | 0.831 | |
| [0.503,0.543] | [0.774,0.807] | [0.816,0.846] | ||
| PPV | 0.070 | 0.126 | 0.135 | |
| [0.056,0.084] | [0.099,0.152] | [0.106,0.166] | ||
| NPV | 0.975 | 0.976 | 0.973 | |
| [0.967,0.983] | [0.970,0.983] | [0.966,0.980] | ||
| ACC | 0.533 | 0.782 | 0.817 | |
| [0.513,0.553] | [0.766,0.798] | [0.802,0.832] | ||
| AUROC | 0.627 | 0.702 | 0.684 | |
| [0.586,0.667] | [0.657,0.746] | [0.639,0.728] |
Se sensitivity, Sp specificity, PPV positive predictive value, NPV negative predictive value, AUROC area under the recieving operator curve
a RETTS-A, NEWS and NEWS2 have been compared with the predefined cut off levels, RETTS-a (Orange/Red), NEWS (Medium/High), NEWS2 (Medium/HIgh)
b Bootstrapped 99% confidence intervals
c AUROC calculations based on predefined cut-off values for each system
Results
Patient characteristics and EMS-assigned RETTS-A triage levels
The median age of the 4465 patients included in the study was 69 years and 52% were females. The patients that were triaged to a lower acuity, tended to be older and more frequently, a female. Patients triaged to a high acuity were also dispatched with priority 1 in the majority of cases. Thus, the proportion of patients who were dispatched with priority 1 by the nurse at the scene were red, 77%; orange, 58%; yellow, 39%; and green, 24%. The most common DMI was ‘chest pain’, but with no significant difference between triage groups. The DMI of ‘respiratory difficulties’ was the most frequent (26%) among patients with the highest triage level (red). Being in contact with the EMS during office hours (08:00–16:00) was more common among patients who were triaged to a low acuity, whereas being in contact with the EMS during the night (24:00–08:00) was more common if triaged red (21%) and orange (19%), as compared with yellow and green. Time at the scene was longer for the high-acuity group, compared with the low acuity. For this reason, level red had a median time of 25 min, which was 4 min longer than that in levels green and yellow. The two most common medical history diagnosis groups were ‘diseases of the circulatory system’ and ‘mental and behavioural disorders’. The EMS nurse assessed a higher proportion of patients having abdominal/flank pain with levels yellow and green, compared with level orange/red, whereas, the opposite was found for ‘chest/thoracic pain’, with the highest percentage found in level orange (12%). ‘Chest/thoracic pain’ was more predisposed to high-acuity triage, together with ‘respiratory distress/dyspnoea/breathing difficulties’, which comprised 22% of red triage patients. In the ED, more patients in the high-acuity group were admitted to in-patient care than in the low-acuity group, with the red triage level having the highest frequency (81%). Circulatory and respiratory system-related hospital diagnosis were more common in the high-acuity group, whereas the hospital diagnosis of symptoms, signs and abnormal clinical and laboratory findings were more common in the low-acuity group. All-cause mortality at 7 days, 30 days and 365 days was associated with a higher acuity and thus increased with the higher triage levels (Table 1).
Probability of outcomes and EMS-assigned RETTS-A triage levels
Red-triaged patients had a three times higher probability of having a time-sensitive condition than the non-red-triaged patients. The occurrence of a deviating VS/complication was 13 times higher while the probability of dying within 48 h was 6 times higher for red-triaged patients. For the lowest triage level (green), there was a low to non-existent risk of developing any of the outcomes. However, there was no significant difference in risk ratios between the yellow and the green group in the occurrence of complications. Yellow- and green-triaged patients had a 79–100% lower risk of death within 48 h. Thirty-one per cent were admitted to in-patient care if triaged to green (Table 2).
Performance of pre-hospital RETTS-A triage
Overall, patients that were triaged to orange or red had an 81% sensitivity in detecting the predefined reference patient, and a 64% corresponding specificity, which implied an over-triage of 36% and under-triage of 19%. In the patients who were triaged to level green or yellow, the NPV was 89%, whereas the corresponding PPV was 49%. The accuracy and AUROC showed similar results, at 69 and 73%, respectively. Patients over 65 years of age had a lower sensitivity (77%) compared with those under 65 (87%), with a corresponding higher specificity of 70 and 59%, respectively. In the older group, the PPV was significantly higher (59%) than in the younger group (38%). The corresponding NPV was lower for those over 65 years of age, at 85%, compared with the younger group (94%). The diagnostic accuracy was higher in the older group (73%) than in the younger group (65%). The younger group had a higher over-triage (42%) than the older group (30%). The older patients had a higher under-triage (23%) than the younger group (13%) (Table 3).
Comparison of outcome measurements between RETTS-A, NEWS, and NEWS 2
The sensitivity of detecting a time-sensitive condition was higher with RETTS-A than with both NEWS and NEWS 2 (73, 37, and 35%), whereas the specificity was higher in NEWS 2 (83%) than RETTS (54%). The NPV was higher in RETTS-A (94%) than in both NEWS (91%) and NEWS 2 (92%). The accuracy was higher in NEWS 2 (78%), than in both NEWS (74%) and RETTS-A (56%). We found no significant difference between the three instruments when calculating the AUROC with the specified cut-offs for an emergent patient. In detecting 48-h mortality, NEWS 2 had higher specificity and accuracy (82%) than both NEWS (78%) and RETTS-A (52%). In terms of complications or deviating VS within 48 h, RETTS-A had higher sensitivity (91%) than NEWS (77%) and NEWS 2 (64%), whereas NEWS 2 had higher specificity (87%) than NEWS (83%) and RETTS-A (56%). For hospital admission, the sensitivity with RETTS-A was higher (59%) than with NEWS (34%) and NEWS 2 (30%); however, RETTS-A also had lower specificity and PPV than NEWS and NEWS 2. In predicting 30-day mortality, RETTS-A had a higher sensitivity (73%) than NEWS 2 (54%), albeit no significant difference compared to NEWS. RETTS-A had lower specificity than NEWS and NEWS 2, respectively. Both the PPV and the accuracy were lower in RETTS-A than in NEWS and NEWS 2 (Table 4).
Agreement between EMS nurse’s field diagnosis and physician’s hospital diagnosis
Of 4465 patients overall, 4168 received diagnosis according to the International Statistical Classification of Diseases and Related Health Problems - Tenth Revision, Swedish edition (ICD-10-SE). Categorising the hospital diagnosis (Table 5), 473 (11%) patients were classified as time-sensitive, 2646 (64%) were classified with a final diagnosis which was not time-sensitive, 808 (19%) were diagnosed with a symptom and 241 patients (6%) were diagnosed with a non-specific assessment. The EMS nurse’s field assessment was appropriate in 82% of the cases. In patients with a defined time-sensitive condition, the EMS nurse’s field assessment was considered appropriate in 395 cases (84%) (Table 5).
Table 5.
Agreement between the EMS nurse’s field assessment and the final hospital physician diagnosis
| EMS field assessment (n = 4168)a | |
|---|---|
| A. A defined final diagnosis classified as a time-sensitive condition | n = 473 |
| 1. The field diagnosis is in agreement with the final diagnosis | 195 (41.2) |
| 2. The field diagnosis is not in agreement with the final diagnosis | 49 (10.4) |
| 3. Typical symptoms related to the final diagnosis | 160 (33.8) |
| 4. Atypical symptoms related to the final diagnosis | 32 (6.8) |
| 5. More unusual symptoms related to the final diagnosis | 8 (1.7) |
| 6. The field assessment as a non-specified organ system | 29 (6.1) |
| B. A defined final diagnosis not classified as a time-sensitive condition | n = 2646 |
| 1. The field diagnosis is in agreement with the final diagnosis | 755 (28.5) |
| 2. The field diagnosis is not in agreement with the final diagnosis | 308 (11.6) |
| 3. Typical symptoms related to the final diagnosis | 1099 (41.5) |
| 4. Atypical symptoms related to the final diagnosis | 173 (6.5) |
| 5. More unusual symptoms related to the final diagnosis | 104 (3.9) |
| 6. The field assessment as a non-specified organ system | 207 (7.8) |
| C. The final diagnosis is expressed as a symptom | n = 808 |
| 1. The field diagnosis is in agreement with the final symptom | 24 (3.0) |
| 2. The field diagnosis is not in agreement with the final symptom | 48 (5.9) |
| 3. The field symptom and the final symptom are in agreement | 620 (76.7) |
| 4. The field symptom and the final symptom are not in agreement | 82 (10.1) |
| 5.The field assessment as a non-specified organ system | 34 (4.2) |
| D. The final diagnosis is described as a non-specific assessment | n = 241 |
| 1. The field diagnosis is in agreement with the symptom | 59 (24.5) |
| 2. The field diagnosis is not in agreement with the symptom | 42 (17.4) |
| 3. The field symptom is in agreement with the final assessment | 84 (34.9) |
| 4. The field symptom and final assessment are not in agreement | 20 (8.3) |
| 5. The field assessment is presented as a non-specified organ system | 36 (14.9) |
a Total number of final hospital diagnoses in 4465 patients
Discussion
In this study, among the population in contact with the EMS and assessed as needing to be sent to the hospital, the median age was 69 years and 87% had a medical history (circulatory and psychiatric diagnoses were the most common). The three most common field assessments according to RETTS-A were abdominal pain, chest pain, and dyspnoea. Our main findings regarding sensitivity and specificity were similar to those in other studies regarding major triage systems (Manchester triage system [MTS], Emergency severity index [ESI], South African triage scale [SATS]) in the EDs [18–21], albeit with a lower specificity indicating a higher rate of false positives in RETTS-A when used in a pre-hospital context. These systems have been reported in systematic reviews to have moderate to good validity with reasonable performance but high variability and different outcome measures [18, 22, 23]. Clinical competence plays a role in the patients’ assessment, and different triage levels may be reported depending on EMS nurse decisions based on the collected information on patient history and interpretation of the clinical presentation. In a systematic review, Considine and colleagues reported that factual knowledge is more important in triage decisions than triage or emergency nursing experience [24]. Furthermore, the EMS nurse has mostly only one patient to consider at-the-scene, whereas the triage nurse in the ED may be influenced by the current situation and triage in order to solve logistical problems [25]. This suggests that context plays a role. Zachariasse and colleagues reported that other factors influence the performance of triage systems such as infrastructure, nurse experience and epidemiology [22]. In a pre-hospital context, the patient population in contact with the EMS ranges from trivial problems to major traumas and severe medical diseases. This imposes greater demands on a triage system to aid in both these scenarios, however, no uniform guidelines exist within the Swedish ambulance organisations when it comes to referral to lower levels of care leading to local variations. In this study, 31% of the green-triaged patients were hospitalised, indicating that the EMS nurse’s actual competence is important in order to decide which green-triaged patient could remain at the scene and which needed to be transported. Furthermore, it is worth considering that among the EMS population, many patients with chronic diseases may not yield a ‘life-threatening’ triage level, though the patients may still be in need of in-patient care. Therefore, variations in the definition of outcomes for admission, based on severity, or the definition of a reference patient in other studies for example, makes comparisons difficult. In a systematic review of 57 studies of triage systems and their performance, a total of 33 different outcome measurements were used [26]. In order to better compare triage systems, suggestions for a consensus on uniform reporting as in the Utstein style [27], as well as a consensus on what constitutes a time-sensitive condition [28] are required.
Regarding over-triage, there appears to be a consensus that over-triage should be built into the system and perhaps even more so in a pre-hospital context. However, overly extensive over-triage may have implications not only in terms of the unnecessary allocation of resources in the ED but also for the EMS nurse. When the triage system indicates acuity (orange, red), there is no option other than ALS ambulance transportation to the hospital. This may cause an ‘alarm inflation’, together with high levels of over-triage at dispatch (5:1 ratio found in this study) and may induce a lack of trust in the system, with reduced adherence. Adherence to pre-hospital guidelines appears to be influenced by several factors, such as the relationship between patient outcomes and guideline evidence [29]. Pre-hospital over-triage is also associated with increased costs when transporting low-urgency patients to high-resource hospitals when these resources are not needed [30]. This may suggest that using the same triage systems both in the pre-hospital setting and, in the ED, could be favourable. Furthermore, in static triage systems based on expert opinion, when an adverse event occurs, interest often focuses on adjusting the system based on the single adverse event. This have previously been reported in accident investigations where the investigation often stops at the level “preventable causes”, because it is easy to resolve and practical to implement [31]. This may lead to a further increase in over-triage. Regarding under-triage, similar findings relating to under-triage in the ED have been reported in other studies of the MTS, ESI, and SATS (9–25%) [21, 32–35], with higher under-triage in the elderly [35, 36]. This was also found in this study (13.3% ≤ 65 yrs. vs 22.6% > 65 yrs.). This is a concern, as patients in contact with the EMS are often older and the three most frequent hospital diagnoses in the under-triage group were stroke/transient ischaemic attack, sepsis, and myocardial infarction. Patients with these diagnoses were commonly triaged to the RETTS-A category of non-specific complaints. Previous studies have shown that elderly patients with non-specific complaints in the ED have a higher short-term mortality, are to be triaged as less urgent and require resources and hospitalisation to a greater extent than patients with specific complaints [37, 38]. There is also a risk of patients being referred to other levels of care with the aid of a triage system, with older patients being more frequently triaged to non-specific complaints than younger patients [39]. A study of the RETTS-A in the ED reported increased 7-day and 30-day mortality in patients over 60 years of age who were triaged to green level [4]. However, we found that, regardless of age, the risk of death within 48 h was none to very low if triaged to level green or yellow, with a six-fold increase in risk if triaged to level red. This indicates that short-term mortality increases with increasing triage level when triaging with RETTS-A in the EMS and reflects the objective of the system. While many triage systems highlight frail patients, the alternative triage guidelines in trauma outperformed the existing guidelines in the elderly population in the EMS [40]. Furthermore, using the same VS definitions of severity levels regardless of age could miss critically ill elderly patients and thus jeopardise patient safety [41]. This suggests a different set-up regarding triage in the elderly, with specific cut-offs for the ageing patient.
The introduction of NEWS in the EMS, adding all VS to obtain a total score and thereby addressing the severity, may be one way of addressing the problem. In this study, NEWS 2 showed higher accuracy than RETTS-A for the prediction of a time-sensitive condition, 48-h mortality and deviating VS/complications within 48 h. On the other hand, RETTS-A showed greater sensitivity for a time-sensitive condition and deviating VS/complications. Furthermore, RETTS-A showed a higher NPV for time-sensitive conditions than both NEWS and NEWS 2, and combining ESS and VS to yield a triage level is a strength of RETTS-A, but at the expense of an increase in the number of false positives. The sensitivity was relatively low for both NEWS and NEWS 2, which also reflected the accuracy, indicating that NEWS is more capable of ruling out critical conditions but at the expense of more false negatives. However, despite the higher sensitivity, the under-triage of some patients (for example, those with sepsis), may indicate more difficulties when using RETTS-A to identify severe diseases if the symptoms are vague. Examples are proneness to falling (ESS yellow) or a slight deviation in VS, which may yield level yellow for single VS. Whereas, the NEWS, on the other hand, may have better capabilities to detect a deteriorating patient with a score of combined VS. This is also supported by other studies of sepsis, where NEWS was superior to both RETTS-A and qSOFA in the prediction of intensive care and 30-day mortality [42, 43].
Regarding at-the-scene time, the main purpose of the RETTS-A triage system in the ED is to assess patient urgency and time to physician in cases where red-triaged patients are thought to require immediate attention. In the EMS, the nurse is able to intervene at the scene when a condition is identified. The prolonged at-the-scene time at red-triaged level can therefore be explained by the examinations and interventions that were performed. Patients presenting with dyspnoea were common in these cases. In a Danish study on patients with dyspnoea, the 1-day mortality did not increase with a total transport time of > 30 min. However, for cerebrovascular conditions, a total at-the-scene time of > 60 min increased short-term mortality, suggesting faster transportation times in such cases [44]. A short at-the-scene time is essential for several conditions where definitive care is required in the hospital. In clear cases of stroke with the sudden onset of hemiplegia, symptom presentation may not be difficult to assess and manage. However, in the case of the older patient who is experiencing vertigo but has otherwise normal VS and is awake, it may be more complicated due to the vague symptom presentation. This indicates that support from a triage system in the field is valid and may aid in the decision-making process. The performance of triage systems in the EMS may also improve if they are linked to a decision-support system.
The EMS nurses’ field assessments, regardless of RETTS-A triage, were in agreement with the final hospital diagnosis in 82% of all the pre-hospital assessments and in 84% of all time-sensitive conditions. For the assessment of time-sensitive conditions in the field, our findings are similar to those of other studies of sepsis and MI, reporting 78–94% agreement between paramedics and hospital physicians assessments [45–47]. One explanation on the inaccurate field assessments of time-sensitive conditions is the inability to identify the condition as time-sensitive due to the lack of competence and lack of pre-hospital equipment, such as blood tests and instruments, to safely rule out time-sensitive conditions. Several diagnosis groups have been described as being more difficult to differentiate from others, as they have a symptomatic overlap. Such diagnoses include subarachnoid haemorrhage and migraine [48]. The EMS nurse may be influenced by factors that may bias the assessment, such as psychiatric illness; a medical history that was common among the patients in this study. In a study on experienced emergency physicians’ assessments in the ambulance, the agreement with hospital discharge diagnosis was 90%, but it decreased by almost 10% in patients with neurological diseases. The authors conclude that medical history at the scene is essential, together with a thorough patient examination, including laboratory tests such as glucose and electrocardiogram (ECG) recording [49]. In order to aid the EMS nurse in complex clinical decision-making, point-of-care tests and guidelines, together with a triage system that is more adapted for the elderly population, seems reasonable.
Strengths and limitations
The strength of this study was the relatively large patient cohort that was manually reviewed, together with the prospective aspect involving staff training in the triage system in order to minimise bias in patient records when retrospectively collected. Furthermore, most of the time, the EMS nurse had only one patient to deal with compared with the ED nurse, which may have reduced the risk of triaging on premises other than patient severity. The main limitation is that data collection took place in a single urban setting with short transportation times; thus, generalising the results may be problematic. Moreover, VS were collected from the registry and are reported by the EMS nurse. VS may be measured but never recorded. However, it is unlikely that abnormal VS are not recorded. In cases with a low level of triage, it is more likely that those unrecorded VS fall within the corresponding range of that colour, and MICE imputation therefore appears valid. The triage level may differ between the EMS nurse’s and the ED nurse’s assessments, as has been reported in a Canadian study [50]. However, a divergence in triage level may have several possible reasons and should be seen as continuous care where the patient may deteriorate or improve over time. Furthermore, this study was conducted on a RETTS-A version available in 2016, while RETTS-A is updated annually in order to reduce under-triage, and some ESS codes may have changed since then.
Conclusions
The median age of the population was 69 years and 87% had a previous medical history. Compared with a predefined reference patient, the sensitivity of RETTS-A was 81% and the specificity was 64%, with over-triage of 36% and under-triage of 19%. An increased risk of an adverse event was identified among the elderly. NEWS and NEWS 2 appeared to have performed better on the outcomes related to VS, mainly due to a higher specificity, while RETTS-A predicted time-sensitive conditions better than NEWS and NEWS 2, and showed a higher proportion of correctly classified low risk triaged patients. Despite that the EMS nurse’s competence play a role in the at-the-scene assessment, over-triage may be unavoidable with the current systems used in the EMS and point-of-care testing and increased medical consultation is proposed. Given the low risk of death in the green triage group, more patients could be diverted to the primary care physicians.
Supplementary information
Additional file 1. Definition of time-sensitive conditions.
Acknowledgements
The authors wish to acknowledge the Ambulance and Pre-Hospital Emergency Care, Sahlgrenska University Hospital in Gothenburg, for providing data on EMS assignments, and Thomas Karlsson, Health Metrics Unit, Sahlgrenska Academy, University of Gothenburg, for providing statistical input.
Abbreviations
- AR
Absolute risk
- ALS
Advanced lifesaving
- AUROC
Area under the receiver operating characteristic curve
- CI
Confidence interval
- DMI
Dispatcher medical index
- ECG
Electrocardiogram
- ED
Emergency department
- EMS
Emergency medical services
- EMT
Emergency medical technician
- ESI
Emergency severity index
- ESS
Emergency signs and symptoms
- ICD-10-SE
International Statistical Classification of Diseases and Related Health Problems – Tenth Revision, Swedish edition
- MTS
Manchester triage system
- MAR
Mostly at random
- MICE
Multivariate imputation via chained eqs.
- MI
Myocardial infarction
- NEWS
National early warning score
- NPV
Negative predictive value
- PPV
Positive predictive value
- qSOFA
Quick sequential organ failure assessment score
- RETTS-A
Rapid emergency triage and treatment system for adults
- RN
Registered nurse
- RR
Relative risk
- SATS
South African triage scale
- VS
Vital signs
Authors’ contributions
CM, CA, and JH developed the design and the research protocol. CM collected and reviewed the data. CM wrote the first draft. CA and JH supervised and analysed the data and their interpretation. All the authors have participated in the final draft of the manuscript and have read and approved the final manuscript.
Funding
CM received grants from the Swedish state under the agreement between the Swedish government and the county councils, the ALF agreement (ALFGBG883141). CM received funding from LÖF, a nationwide insurance Swedish insurance company, and the OLA Foundation for the patient safety project of which this study is part. The funding body had no influence over the results or any involvement in this study.
Availability of data and materials
The datasets analysed during the current study are available from the corresponding author in response to a reasonable request.
Ethics approval and consent to participate
The study was approved by the central ethical review board, Gothenburg, Sweden, approval no: 970–15. This is a study in which data were prospectively recorded by the health-care providers in the ambulance. All patients and/or relatives were asked whether their identity should remain confidential before being reported in the ambulance patient data records. The patients who asked for their data to remain confidential were not included in the retrospective analysis. However, informed consent was not obtained from the patients for our retrospective analysis of their data. This is generally not recommended by Ethical Review Boards in Sweden for the following reasons: 1) Individual patients could never be identified since their identification number was translated to a code (their integrity therefore remained intact); 2) some of the most serious patients could never be contacted retrospectively, as they had either died or were in very poor clinical condition (this would increase the risk of selection bias, thereby hampering the reliability of the data); and 3) approaching patients and/or relatives with these issues may create more anxiety than satisfaction and may therefore be regarded as unethical.
Consent for publication
Not applicable.
Competing interests
The authors declare that they have no competing interests.
Footnotes
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Contributor Information
Carl Magnusson, Email: carl.magnusson@vgregion.se.
Johan Herlitz, Email: johan.herlitz@hb.se.
Christer Axelsson, Email: christer.axelsson@hb.se.
Supplementary information
Supplementary information accompanies this paper at 10.1186/s13049-020-00766-1.
References
- 1.FitzGerald G, Jelinek GA, Scott D, Gerdtz MF. Emergency department triage revisited. Emerg Med J. 2010;27:86–92. doi: 10.1136/emj.2009.077081. [DOI] [PubMed] [Google Scholar]
- 2.Fernandes CMB, Tanabe P, Gilboy N, Johnson LA, McNair RS, Rosenau AM, et al. Five-level triage: a report from the acep/ena five-level triage task force. J Emerg Nurs. 2005;31:39–50. doi: 10.1016/j.jen.2004.11.002. [DOI] [PubMed] [Google Scholar]
- 3.Widgren BR, Jourak M. Medical emergency triage and treatment system (metts): a new protocol in primary triage and secondary priority decision in emergency medicine. J Emerg Med. 2011;40:623–628. doi: 10.1016/j.jemermed.2008.04.003. [DOI] [PubMed] [Google Scholar]
- 4.Ruge T, Malmer G, Wachtler C, Ekelund U, Westerlund E, Svensson P, et al. Age is associated with increased mortality in the retts-a triage scale. BMC Geriatr. 2019;19:139. [DOI] [PMC free article] [PubMed]
- 5.Lidal IB, Holte HH, Vist GE. Triage systems for pre-hospital emergency medical services - a systematic review. Scandinavian journal of trauma, resuscitation and. Emerg Med. 2013;21:28. doi: 10.1186/1757-7241-21-28. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Larsson G, Holmen A, Ziegert K. Early prehospital assessment of non-urgent patients and outcomes at the appropriate level of care: a prospective exploratory study. Int Emerg Nurs. 2017;32:45–49. doi: 10.1016/j.ienj.2017.02.003. [DOI] [PubMed] [Google Scholar]
- 7.Ng C-J, Chien C-Y, Seak J-J, Tsai S-L, Weng Y-M, Chaou C-H, et al. Validation of the five-tier Taiwan triage and acuity scale for prehospital use by emergency medical technicians. Emerg Med J. 2019;36:472–478. doi: 10.1136/emermed-2018-207509. [DOI] [PubMed] [Google Scholar]
- 8.Askim Å, Moser F, Gustad LT, Stene H, Gundersen M, Åsvold BO, et al. Poor performance of quick-sofa (qSOFA) score in predicting severe sepsis and mortality –a prospective study of patients admitted with infection to the emergency department. Scand J Trauma, Resusc Emerg Med. 2017;25:56. doi: 10.1186/s13049-017-0399-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Hoikka et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine. 2018;26:48. 10.1186/s13049-018-0514-1. [DOI] [PMC free article] [PubMed]
- 10.Silcock D, Corfield A, Gowens P, Rooney K. Validation of the national early warning score in the prehospital setting. Resuscitation. 2015;89:31–35. doi: 10.1016/j.resuscitation.2014.12.029. [DOI] [PubMed] [Google Scholar]
- 11.Pirneskoski J, Kuisma M, Olkkola K, Nurmi J. Prehospital national early warning score predicts early mortality. Acta Anaesthesiol Scand. 2019;63:676–683. doi: 10.1111/aas.13310. [DOI] [PubMed] [Google Scholar]
- 12.Lane DJ, Wunsch H, Saskin R, et al. Assessing Severity of Illness in Patients Transported to Hospital by Paramedics: External Validation of 3 Prognostic Scores. Prehosp Emerg Care. 2020;24(2):273–81. 10.1080/10903127.2019.1632998. [DOI] [PubMed]
- 13.Martín-Rodríguez F, López-Izquierdo R, del Pozo Vegas C, Delgado Benito JF, Carbajosa Rodríguez V, Diego Rasilla MN, et al. Accuracy of national early warning score 2 (news2) in prehospital triage on in-hospital early mortality: a multi-center observational prospective cohort study. Prehosp Disaster Med. 2019;34:610–618. doi: 10.1017/S1049023X19005041. [DOI] [PubMed] [Google Scholar]
- 14.Smith G, Prytherch D, Meredith P, Schmidt P, Featherstone P. The ability of the national early warning score (news) to discriminate patients at risk of early cardiac arrest, unanticipated intensive care unit admission, and death. Resuscitation. 2013;84:465–470. doi: 10.1016/j.resuscitation.2012.12.016. [DOI] [PubMed] [Google Scholar]
- 15.Williams B. The national early warning score 2 (news2) in patients with hypercapnic respiratory failure. Clin Med (Lond) 2019;19:94–95. doi: 10.7861/clinmedicine.19-1-94. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Hagiwara MA, Nilsson L, Strömsöe A, Axelsson C, Kängström A, Herlitz J. Patient safety and patient assessment in pre-hospital care: a study protocol. Scand J Trauma Resusc Emerg Med. 2016;24:14. doi: 10.1186/s13049-016-0206-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Tärnqvist J, Dahlén E, Norberg G, Magnusson C, Herlitz J, Strömsöe A, et al. On-scene and final assessments and their interrelationship among patients who use the ems on multiple occasions. Prehosp Disaster Med. 2017;32:528–535. doi: 10.1017/S1049023X17006458. [DOI] [PubMed] [Google Scholar]
- 18.Zachariasse JM, Seiger N, Rood PPM, Alves CF, Freitas P, Smit FJ, et al. Validity of the Manchester triage system in emergency care: a prospective observational study. PLoS One. 2017;12:e0170811. doi: 10.1371/journal.pone.0170811. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Travers DA, Waller AE, Bowling JM, Flowers D, Tintinalli J. Five-level triage system more effective than three-level in tertiary emergency department. J Emerg Nurs. 2002;28:395–400. doi: 10.1067/men.2002.127184. [DOI] [PubMed] [Google Scholar]
- 20.Dalwai MK, Twomey M, Maikere J, Said S, Wakeel M, Jemmy J-P, et al. Reliability and accuracy of the south african triage scale when used by nurses in the emergency department of timergara hospital, Pakistan. S Afr Med J. 2014;104:372–375. doi: 10.7196/samj.7604. [DOI] [PubMed] [Google Scholar]
- 21.Meyer GD, Meyer TN, Gaunt CB. Validity of the south african triage scale in a rural district hospital. Afr J Emerg Med. 2018;8:145–149. doi: 10.1016/j.afjem.2018.07.004. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Zachariasse JM, van der Hagen V, Seiger N, Mackway-Jones K, van Veen M, Moll HA. Performance of triage systems in emergency care: a systematicreview and meta-analysis. BMJ Open. 2019;9(5):e026471. 10.1136/bmjopen-2018-026471. [DOI] [PMC free article] [PubMed]
- 23.Hinson JS, Martinez DA, Cabral S, George K, Whalen M, Hansoti B, et al. Triage performance in emergency medicine: a systematic review. Ann Emerg Med. 2019;74:140–152. doi: 10.1016/j.annemergmed.2018.09.022. [DOI] [PubMed] [Google Scholar]
- 24.Considine J, Botti M, Thomas S. Do knowledge and experience have specific roles in triage decision-making? Acad Emerg Med. 2007;14:722–726. doi: 10.1197/j.aem.2007.04.015. [DOI] [PubMed] [Google Scholar]
- 25.Wolf LA, Delao AM, Perhats C, Moon MD, Zavotsky KE. Triaging the emergency department, not the patient: United States emergency nurses’ experience of the triage process. J Emerg Nurs. 2018;44:258–266. doi: 10.1016/j.jen.2017.06.010. [DOI] [PubMed] [Google Scholar]
- 26.Kuriyama A, Urushidani S, Nakayama T. Five-level emergency triage systems: variation in assessment of validity. Emerg Med J. 2017;34:703–710. doi: 10.1136/emermed-2016-206295. [DOI] [PubMed] [Google Scholar]
- 27.Idris AH, Bierens JJLM, Perkins GD, Wenzel V, Nadkarni V, Morley P, et al. 2015 revised utstein-style recommended guidelines for uniform reporting of data from drowning-related resuscitation: an ilcor advisory statement. Resuscitation. 2017;118:147–158. doi: 10.1016/j.resuscitation.2017.05.028. [DOI] [PubMed] [Google Scholar]
- 28.Wibring K, Magnusson C, Axelsson C, Lundgren P, Herlitz J, Andersson HM. Towards definitions of time-sensitive conditions in prehospital care. Scand J Trauma Resusc Emerg Med. 2020;28:7. doi: 10.1186/s13049-020-0706-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Ebben RHA, Vloet LCM, van Grunsven PM, Breeman W, Goosselink B, Lichtveld RA, et al. Factors influencing ambulance nurses’ adherence to a national protocol ambulance care: an implementation study in the Netherlands. Eur J Emerg Med. 2015;22:199–205. doi: 10.1097/MEJ.0000000000000133. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Newgard CD, Staudenmayer K, Hsia RY, Mann NC, Bulger EM, Holmes JF, et al. The cost of overtriage: More than one-third of low-risk injured patients were taken to major trauma centers. Health Aff (Millwood) 2013;32:1591–1599. doi: 10.1377/hlthaff.2012.1142. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Lundberg J, Rollenhagen C, Hollnagel E. What you find is not always what you fix–how other aspects than causes of accidents decide recommendations for remedial actions. Accid Anal Prev. 2010;42:2132–2139. doi: 10.1016/j.aap.2010.07.003. [DOI] [PubMed] [Google Scholar]
- 32.Storm-Versloot MN, Ubbink DT, Kappelhof J, Luitse JSK. Comparison of an informally structured triage system, the emergency severity index, and the Manchester triage system to distinguish patient priority in the emergency department. Acad Emerg Med. 2011;18:822–829. doi: 10.1111/j.1553-2712.2011.01122.x. [DOI] [PubMed] [Google Scholar]
- 33.Olofsson P, Gellerstedt M, Carlström ED. Manchester triage in Sweden - interrater reliability and accuracy. Int Emerg Nurs. 2009;17:143–148. doi: 10.1016/j.ienj.2008.11.008. [DOI] [PubMed] [Google Scholar]
- 34.van der Wulp I, van Baar ME, Schrijvers AJP. Reliability and validity of the Manchester triage system in a general emergency department patient population in the Netherlands: results of a simulation study. Emerg Med J. 2008;25:431–434. doi: 10.1136/emj.2007.055228. [DOI] [PubMed] [Google Scholar]
- 35.van der Wulp I, Schrijvers AJP, van Stel HF. Predicting admission and mortality with the emergency severity index and the Manchester triage system: a retrospective observational study. Emerg Med J. 2009;26:506–509. doi: 10.1136/emj.2008.063768. [DOI] [PubMed] [Google Scholar]
- 36.Brouns SHA, Mignot-Evers L, Derkx F, Lambooij SL, Dieleman JP, Haak HR. Performance of the Manchester triage system in older emergency department patients: a retrospective cohort study. BMC Emerg Med. 2019;19:3. doi: 10.1186/s12873-018-0217-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37.Wachelder JJH, Stassen PM, Hubens LPAM, Brouns SHA, Lambooij SLE, Dieleman JP, et al. Elderly emergency patients presenting with non-specific complaints: characteristics and outcomes. PLoS One. 2017;12:e0188954. doi: 10.1371/journal.pone.0188954. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Kemp K, Mertanen R, Lääperi M, Niemi-Murola L, Lehtonen L, Castren M. Nonspecific complaints in the emergency department - a systematic review. Scand J Trauma Resusc Emerg Med. 2020;28:6. doi: 10.1186/s13049-020-0699-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 39.Magnusson C, Herlitz J, Axelsson C. Patient characteristics, triage utilisation, level of care, and outcomes in an unselected adult patient population seen by the emergency medical services: a prospective observational study. BMC Emerg Med. 2020;20:7. doi: 10.1186/s12873-020-0302-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Newgard CD, Holmes JF, Haukoos JS, Bulger EM, Staudenmayer K, Wittwer L, et al. Improving early identification of the high-risk elderly trauma patient by emergency medical services. Injury. 2016;47:19–25. doi: 10.1016/j.injury.2015.09.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Lamantia MA, Stewart PW, Platts-Mills TF, Biese KJ, Forbach C, Zamora E, et al. Predictive value of initial triage vital signs for critically ill older adults. West J Emerg Med. 2013;14:453–460. doi: 10.5811/westjem.2013.5.13411. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 42.Mellhammar L, Linder A, Tverring J, Christensson B, Boyd JH, Åkesson P, et al. Scores for sepsis detection and risk stratification - construction of a novel score using a statistical approach and validation of retts. PLoS One. 2020;15:e0229210. doi: 10.1371/journal.pone.0229210. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Silcock DJ, Corfield AR, Staines H, Rooney KD. Superior performance of National Early Warning Score compared with quick Sepsis-related Organ Failure Assessment Score in predicting adverse outcomes: a retrospective observational study of patients in the prehospital setting. Eur J Emerg Med. 2019;26(6):433–9. 10.1097/MEJ.0000000000000589. [DOI] [PubMed]
- 44.Mills EHA, Aasbjerg PMD, Hansen SM, Ringgren KBMB, Dahl PMD, Rasmussen BS, et al. Prehospital time and mortality in patients requiring a highest priority emergency medical response: a danish registry-based cohort study. BMJ Open. 2019;9:e023049. doi: 10.1136/bmjopen-2018-023049. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 45.Feldman JA, Brinsfield K, Bernard S, White D, Maciejko T. Real-time paramedic compared with blinded physician identification of st-segment elevation myocardial infarction: results of an observational study. Am J Emerg Med. 2005;23:443–448. doi: 10.1016/j.ajem.2004.10.011. [DOI] [PubMed] [Google Scholar]
- 46.Green RS, Travers AH, Cain E, Campbell SG, Jensen JL, Petrie DA, et al. Paramedic recognition of sepsis in the prehospital setting: a prospective observational study. Emerg Med Int. 2016;2016:6717261. doi: 10.1155/2016/6717261. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 47.Ducas RA, Wassef AW, Jassal DS, Weldon E, Schmidt C, Grierson R, et al. To transmit or not to transmit: how good are emergency medical personnel in detecting stemi in patients with chest pain? Can J Cardiol. 2012;28:432–437. doi: 10.1016/j.cjca.2012.04.008. [DOI] [PubMed] [Google Scholar]
- 48.Croskerry P. Context is everything or how could i have been that stupid? Healthc Q. 2009;12:e171–e176. doi: 10.12927/hcq.2009.20945. [DOI] [PubMed] [Google Scholar]
- 49.Heuer JF, Gruschka D, Crozier TA, Bleckmann A, Plock E, Moerer O, et al. Accuracy of prehospital diagnoses by emergency physicians: comparison with discharge diagnosis. Eur J Emerg Med. 2012;19:292–296. doi: 10.1097/MEJ.0b013e32834ce104. [DOI] [PubMed] [Google Scholar]
- 50.Leeies M, Ffrench C, Strome T, Weldon E, Bullard M, Grierson R. Prehospital application of the Canadian triage and acuity scale by emergency medical services. CJEM. 2017;19:26–31. doi: 10.1017/cem.2016.345. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Additional file 1. Definition of time-sensitive conditions.
Data Availability Statement
The datasets analysed during the current study are available from the corresponding author in response to a reasonable request.