Abstract
Background
Patients with major trauma might benefit from treatment in a trauma centre, but early identification of major trauma (Injury Severity Score (ISS) over 15) remains difficult. The aim of this study was to undertake an external validation of existing prognostic models for injured patients to assess their ability to predict mortality and major trauma in the prehospital setting.
Methods
Prognostic models were identified through a systematic literature search up to October 2017. Injured patients transported by Emergency Medical Services to an English hospital from the Trauma Audit and Research Network between 2013 and 2016 were included. Outcome measures were major trauma (ISS over 15) and in‐hospital mortality. The performance of the models was assessed in terms of discrimination (concordance index, C‐statistic) and net benefit to assess the clinical usefulness.
Results
A total of 154 476 patients were included to validate six previously proposed prediction models. Discriminative ability ranged from a C‐statistic value of 0·602 (95 per cent c.i. 0·596 to 0·608) for the Mechanism, Glasgow Coma Scale, Age and Arterial Pressure model to 0·793 (0·789 to 0·797) for the modified Rapid Emergency Medicine Score (mREMS) in predicting in‐hospital mortality (11 882 patients). Major trauma was identified in 52 818 patients, with discrimination from a C‐statistic value of 0·589 (0·586 to 0·592) for mREMS to 0·735 (0·733 to 0·737) for the Kampala Trauma Score in predicting major trauma. None of the prediction models met acceptable undertriage and overtriage rates.
Conclusion
Currently available prehospital trauma models perform reasonably in predicting in‐hospital mortality, but are inadequate in identifying patients with major trauma. Future research should focus on which patients would benefit from treatment in a major trauma centre.
Currently available prehospital trauma models perform reasonably in predicting in‐hospital mortality, but are inadequate at identifying patients who have experienced major trauma. Future research should focus on which patients would benefit from treatment in a major trauma centre. TP, true‐positive; FP, false‐positive; MGAP, Mechanism, Glasgow Coma Scale, Age and Arterial Pressure; PSS, Physiologic Severity Score; T‐RTS, Triage Revised Trauma Score; PHI, Prehospital Index; mREMS, modified Rapid Emergency Medicine Score; KTS, Kampala Trauma Score.
Useful in predicting mortality but not major trauma
Antecedentes
Los pacientes con traumatismo mayor pueden beneficiarse del tratamiento en un centro de trauma, pero la identificación precoz del traumatismo mayor (Injury Severity Score, ISS > 15) sigue siendo difícil. El objetivo de este estudio fue validar externamente los modelos pronósticos existentes para los pacientes con traumatismos con el fin de evaluar su capacidad para predecir el traumatismo mayor y la mortalidad en el entorno pre‐hospitalario.
Métodos
Los modelos pronóstico se identificaron mediante una búsqueda sistemática de la literatura hasta octubre de 2017. Los pacientes incluidos fueron pacientes con traumatismos que fueron trasladados mediante los servicios de emergencia médica (emergency medical services, EMS) a un hospital inglés perteneciente a Trauma Audit and Research Network (TARN) entre 2013 y 2016. Las variables evaluadas fueron los traumatismos graves (ISS > 15) y la mortalidad hospitalaria. El rendimiento de los modelos se analizó en términos de discriminación (índice de concordancia, c) y de beneficio neto para evaluar la utilidad clínica.
Resultados
Se incluyeron un total de 154.476 pacientes para validar los seis modelos de predicción propuestos previamente. La capacidad discriminatoria osciló entre c = 0,602 (i.c. del 95%: 0,596‐0,608) para el modelo que incluye mecanismo, escala de coma de Glasgow, edad y presión arterial (MGAP) hasta c = 0,793 (0,789‐0,797) para la puntuación de medicina de emergencia rápida modificada (mREMS) en la predicción de la mortalidad hospitalaria (n = 11.882). Se identificó un traumatismo mayor en 52.818 pacientes, con una discriminación de c = 0,589 (0,586‐0,592) para mREMS a c = 0,735 (0,733‐0,737) para la puntuación de trauma de Kampala en la predicción de traumatismo mayor. Ninguno de los modelos de predicción cumplió con las tasas aceptables de subtriaje (undertriage) y sobretriaje (overtriage).
Conclusión
Los modelos de trauma pre‐hospitalarios actualmente disponibles tienen un rendimiento razonable para predecir la mortalidad hospitalaria, pero son inadecuados para identificar a los pacientes con traumatismo mayor. En el futuro, las investigaciones deberían centrarse en identificar a los pacientes que se podrían beneficiar del tratamiento en un centro de trauma especializado.
Introduction
Trauma remains one of the major causes of premature death and disability worldwide1. Studies suggest that patients with major trauma, defined as life‐threatening or potentially life‐changing injury, benefit from treatment in a level 1 trauma centre2. Identification of patients with major trauma in the prehospital setting is therefore important, as certain patients could potentially benefit from bypassing the nearest hospital and a longer primary journey to a more distant level 1 trauma centre.
However, early identification of major trauma remains difficult. Undertriage, whereby patients with major trauma are treated at a non‐trauma centre, could cause avoidable mortality and morbidity3. On the other hand, overtriage, whereby patients without major trauma are transported to a trauma centre, could overwhelm level 1 trauma centres, adversely affecting patient outcomes, and transporting patients longer distances for no benefit. The American College of Surgeons – Committee on Trauma4 stipulates that an undertriage rate of 5 per cent and an overtriage rate of 25–35 per cent is acceptable.
Major trauma is usually defined in anatomical terms by an Injury Severity Score (ISS) above 15. The ISS is based on postimaging anatomical and clinical findings, and cannot be calculated in the prehospital setting5. Currently, there are no consensual criteria for prehospital identification of major trauma. A previous systematic review6 found that prognostic models developed to predict mortality could be useful for identifying major trauma. However, a direct comparison of the performance of existing models in the same validation cohort is needed to assess which models are useful in the prehospital setting to predict major trauma (ISS over 15) and mortality7.
The aim of this study was to undertake an external validation of existing prognostic models for injured patients, and to assess their discriminative ability to predict mortality and major trauma in the prehospital setting.
Methods
Identification of prognostic models
A systematic search was undertaken to identify existing prognostic models aimed at improving early trauma care, based on a systematic review published in 20116. Five prognostic models that were included in the original study were included in this study, and the search was updated using the original published search strategy in the MEDLINE database (Appendix S1 , supporting information). Each study was assessed against the original inclusion criteria of the systematic review, which comprised: a tool for clinicians that includes two or more predictors obtained from the history and physical examination of a suspected trauma victim; predictors collected in the field or in the emergency department up to 12 h after injury; prognostic models developed for adult patients defined, for the purpose of this review, as older than 15 years of age or if the patients were described by the authors as adults; and studies published within the last 20 years. The original exclusion criteria of the systematic review were: models that required complex information such as paraclinical diagnostic tests; models pertaining to burns, drowning, strangulation, isolated proximal femur fractures, isolated traumatic brain injury, pregnancy or medical conditions; and studies not written in English. Prognostic models identified from the systematic review and the updated search were validated externally when the variables used could be retrieved in the validation cohort.
Validation cohort
To validate the selected prognostic models externally, retrospectively collected data from the Trauma Audit and Research Network (TARN) were used. The TARN database is the national trauma registry of England, Wales, Northern Ireland and the Republic of Ireland, with some members in continental Europe. TARN includes patients with significant injury presenting to hospital who were subsequently either admitted for at least 72 h or to a critical care area, or required interhospital transfer for acute care, or who died in hospital. Isolated fractures of the hip/pubic ramus in patients aged 65 years or more and isolated closed limb injury (except for femoral shaft) were excluded. Injured adult patients (aged over 15 years) who were transported by the Emergency Medical Services (EMS) or Helicopter Emergency Medical Services (HEMS) and admitted to hospitals in England between 2013 and 2016 were included; secondary referrals of patients for whom first hospital details were not available were excluded. TARN has UK Health Research Authority Approval (PIAG Section 251) for research on anonymized patient data (TARN extraction date: 23 May 2017).
Statistical analysis
Baseline characteristics of patients in the validation cohort are presented as median (i.q.r.) for continuous variables and as percentages for categorical variables. To account for missing data, multiple imputation was performed8, 9. The pattern of missingness was assumed to be missing at random. This means that the missing values correlate with other patient characteristics, which makes it possible to use multiple imputation. The imputation model contained all relevant predictor and outcome variables: the model predictors (systolic BP, respiratory rate, prehospital intubation, pulse, oxygen saturation, consciousness or Glasgow Coma Scale (GCS) score, mechanism of injury (blunt versus other), number of serious injuries and age), combined with extra predictor variables that could add information to predict missing values (in‐hospital intubation, ISS, New Injury Severity Score, duration of hospital stay, duration of critical care stay, trauma team activation, head injury, Charlson Co‐morbidity Index, GCS components at emergency department, sex and in‐hospital mortality).
The models were first validated to assess their ability to predict in‐hospital mortality, and then to assess their ability to predict injury severity (ISS over 15). Patients were scored retrospectively using the published score chart for each model. The published score charts were used instead of the regression formulas, as the performance of the models in the prehospital setting was assessed and ambulance personnel would use the score charts. When a variable was not available in the TARN cohort, a proxy was used if possible. The intercept was refitted on the TARN registry, which resulted in a recalibrated model that was slightly adjusted to the TARN population. The performance of the models was assessed in terms of discrimination, which is the ability of the model to distinguish between low‐ and high‐risk groups of patients. Discriminative ability was expressed as the C‐statistic, which is equivalent to the area under the receiving operating characteristic (ROC) curve (AUC). A C‐statistic of 1 implies perfect discrimination, whereas a value of 0·5 implies that the performance of the model is equal to chance10. Additionally, overtriage and undertriage rates were calculated. Undertriage is the proportion of patients with major trauma who were not classified by the model as having major trauma, and was calculated as 1 – sensitivity, where sensitivity is defined as the proportion of patients actually with major trauma who were identified as having major trauma by the model. Overtriage is the proportion of patients with minor trauma who were classified by the prediction model as having major trauma, and was calculated as 1 – specificity, where specificity was defined as the proportion of patients actually with major trauma who were identified as having minor trauma by the model.
Finally, the net benefit was calculated for all models at different thresholds. The net benefit represents the potential gain of using the prediction models under study for triage of injured patients compared with sending all patients to a major trauma centre. Net benefit is defined as the proportion of true‐positives – proportion of false‐positives × weight. For example, a threshold of 0·2 means that a trauma centre would accept four patients wrongly classified as having major trauma (false‐positives) to identify one correctly classified as having major trauma (true‐positive, defined as ISS over 15). The weight is defined as the odds of the threshold (maximum number of patients wrongly classified as having major trauma (false‐positives) to correctly classify 1 patient with major trauma (true‐positive)). For the threshold of 0·2, the weight is 1 : 411.
Data were analysed using R software version 3.2.2 or higher (R Foundation for Statistical Computing, Vienna, Austria).
Results
Selection of risk prediction models
The study by Rehn and colleagues6 identified five prognostic models: Circulation, Respiration, Abdomen, Motor, Speech (CRAMS)12, Prehospital Index (PHI)13, Triage Revised Trauma Score (T‐RTS)14, Physiologic Severity Score (PSS)15 and Mechanism, Glasgow Coma Scale, Age and Arterial Pressure (MGAP)16; all were developed for predicting mortality in injured patients (Table S1 , supporting information).
The updated electronic search identified 6048 new articles until October 2017 (Fig. S1 , supporting information). After screening of titles and abstracts, 106 potentially eligible articles were identified for full‐text review. After full‐text assessment, 104 articles were excluded because they did not develop a prediction model or the prediction model included predictors that could not be measured in the prehospital setting. Finally, two extra prediction models were identified: the modified Rapid Emergency Medicine Score (mREMS)17 and the Kampala Trauma Score (KTS)18, also developed for predicting mortality in injured patients (Table S1 , supporting information).
The CRAMS model included a variable that needs prehospital assessment of abdominal tenderness and the presence of rigid or flail chest. There was no proxy available for this predictor in the validation data, so this model was excluded. All other models could be validated in the TARN data set. In total, six prehospital models were included for validation (Table S1 , supporting information).
For KTS, the variable ‘number of serious injuries’ was not directly available in the present validation cohort. Therefore, the number of serious injuries was based on the number of reported Abbreviated Injury Scale (AIS) codes greater than 2.
Validation cohort
Some 246 301 patients were included in the TARN registry between 2013 and 2016. A total of 154 476 adult injured patients who were transported by the EMS or HEMS to a hospital were included in the validation cohort (Fig. 1). Overall, 101 658 patients (65·8 per cent) had an ISS of 15 or below. The median age of these patients was 68 (i.q.r. 50–84) years, 48 511 patients (47·7 per cent) were male, median ISS was 9 (5–9), 0·9 per cent of these patients were comatose when presenting at the emergency department, and the in‐hospital mortality rate was 3·7 per cent (Table 1). Among patients who had major trauma (ISS over 15), the median age was 61 (39–81) years, 65·3 per cent were male, the median ISS was 25 (17–27), 16·3 per cent were comatose when presenting at the emergency department, and the in‐hospital mortality rate was 15·5 per cent. The proportions of missing values were low: 0·1 per cent for in‐hospital variables, 6·6 per cent for prehospital variables and below 0·01 per cent for outcome variables in the group with an ISS of 15 or less. Respective proportions in the group with an ISS above 15 were 0·1, 9·1 and less than 0·01 per cent.
Figure 1.
STROBE flow diagram of included and excluded patients TARN, Trauma, Audit and Research Network; EMS, Emergency Medical Services; HEMS, Helicopter Emergency Medical Services.
Table 1.
Baseline characteristics according to severity of trauma and survival status
| ISS ≤15 (n = 101 658) | ISS > 15 (n = 52 818) | Alive at discharge (n = 142 594) | Died in hospital (n = 11 882) | Total (n = 154 476) | |
|---|---|---|---|---|---|
| Patient characteristics | |||||
| Age (years)* | 68 (50–84) | 61 (39–81) | 64 (46–82) | 81 (62–88) | 66 (47–83) |
| Male | 48 511 (47·7) | 34 468 (65·3) | 76 272 (53·5) | 6707 (56·4) | 82 979 (53·7) |
| ISS* | 9 (5–9) | 25 (17–27) | 9 (9–17) | 25 (10–26) | 9 (9–17) |
| GCS score in emergency department* | 15 (15–15) | 15 (12–15) | 15 (15–15) | 13 (4–15) | 15 (15–15) |
| GCS score < 9† | 944 (0·9) | 8516 (16·3) | 5087 (3·6) | 4373 (37·5) | 9460 (6·2) |
| Penetrating injury | 2867 (2·8) | 1392 (2·6) | 3970 (2·8) | 289 (2·4) | 4259 (2·8) |
| Intubated in emergency department | 2030 (2·0) | 12 357 (23·4) | 9819 (6·9) | 4568 (38·4) | 14 387 (9·3) |
| Shock (systolic BP < 90 mmHg) in emergency department | 17 246 (17·0) | 12 445 (23·6) | 26 189 (18·4) | 3502 (29·5) | 29 691 (19·2) |
| Prehospital measurements | |||||
| Systolic BP (mmHg)* | 139 (122–158) | 136 (118–157) | 138 (121–157) | 139 (113–163) | 138 (121–158) |
| Respiratory rate (per min)* | 18 (16–20) | 18 (16–22) | 18 (16–20) | 18 (16–22) | 18 (16–20) |
| Pulse (per min)* | 83 (72–96) | 84 (71–100) | 84 (72–97) | 82 (68–100) | 84 (72–97) |
| Oxygen saturation (%)* | 97 (95–98) | 97 (94–98) | 97 (95–99) | 95 (91–98) | 97 (95–98) |
| No. of serious injuries | |||||
| 0 | 961 (0·9) | 0 (0) | 914 (0·6) | 47 (0·4) | 961 (0·6) |
| 1 | 70 989 (69·8) | 12 076 (22·9) | 78 587 (55·1) | 4478 (37·7) | 83 065 (53·8) |
| ≥ 2 | 29 708 (29·2) | 40 742 (77·1) | 63 093 (44·2) | 7357 (61·9) | 70 450 (45·6) |
| Outcomes | |||||
| Duration of hospital stay (days)* | 10 (5–18) | 9 (4–20) | 10 (5–19) | 4 (1–10) | 9 (5–19) |
| Duration of critical care stay (days)* | 0 (0–0) | 0 (0–1) | 0 (0–0) | 0 (0–1) | 0 (0–0) |
| Primary referral to MTC | 33 349 (32·8) | 27 220 (51·5) | 55 298 (38·8) | 5271 (44·4) | 60 569 (39·2) |
| In‐hospital mortality | 3711 (3·7) | 8171 (15·5) | – | – | 11 882 (7·7) |
Values in parentheses are percentages unless indicated otherwise;
values are median (i.q.r.).
Missing values. ISS, Injury Severity Score; GCS, Glasgow Coma Scale; MTC, major trauma centre.
External validation
First, the discriminative ability of the prediction models to predict in‐hospital mortality was assessed. The discriminative performance (C‐statistic) for the models varied between 0·602 (95 per cent c.i. 0·596 to 0·608) for the MGAP model and 0·793 (0·789 to 0·797) for the mREMS model (Table 2). For comparison, the C‐statistic of the ISS for predicting in‐hospital mortality was 0·728 (0·723 to 0·733).
Table 2.
Discriminative ability (C‐statistic) for in‐hospital mortality (11 882 patients) and major trauma (52 818) among 154 476 patients registered in TARN, 2013–2016, with overtriage and undertriage rates for predicting major trauma using the optimal cut‐off for models in the validation data set
| Model | C‐statistic for in‐hospital mortality | C‐statistic for ISS > 15 | Cut‐off indicating major trauma of total model score | 1 – sensitivity (undertriage) | 1 – specificity (overtriage) |
|---|---|---|---|---|---|
| PHI | 0·734 (0·729, 0·739) | 0·708 (0·706, 0·711) | ≥ 1 of 20 | 38·9 | 23·7 |
| T‐RTS | 0·706 (0·702, 0·711) | 0·630 (0·628, 0·632) | ≤ 11 of 12 | 66·8 | 8·1 |
| PSS | 0·741 (0·736, 0·746) | 0·710 (0·708, 0·713) | ≤ 11 of 12 | 40·5 | 21·3 |
| MGAP | 0·602 (0·596, 0·608) | 0·659 (0·657, 0·662) | ≤ 28 of 29 | 31·0 | 51·2 |
| mREMS | 0·793 (0·789, 0·797) | 0·589 (0·586, 0·592) | > 3 of 26 | 23·1 | 72·4 |
| KTS | 0·769 (0·764, 0·773) | 0·735 (0·733, 0·737) | ≤ 15 of 16 | 3·6 | 82·8 |
| ISS | 0·728 (0·723, 0·733) | – | > 15 | – | – |
Values in parentheses are 95 per cent confidence intervals. An Injury Severity Score (ISS) above 15 indicates major trauma. According to American College of Surgeons – Committee on Trauma guidelines, an undertriage rate of 5 per cent and an overtriage rate of 25–35 per cent is acceptable. TARN, Trauma and Audit Research Network; PHI, Prehospital Index; T‐RTS, Triage Revised Trauma Score; PSS, Physiologic Severity Score; MGAP, Mechanism, Glasgow Coma Scale, Age and Arterial Pressure; mREMS, modified Rapid Emergency Medicine Score; KTS, Kampala Trauma Score.
The ability of the prehospital prediction models to predict major trauma (ISS over 15) was then assessed. The C‐statistic the models varied between 0·589 (0·586 to 0·592) for the mREMS model and 0·735 (0·733 to 0·737) for the KTS model (Table 2).
Overtriage and undertriage rates were calculated. The optimal cut‐off values were tested and the undertriage rate (1 – sensitivity) varied from 3·6 per cent for KTS to 66·8 per cent for T‐RTS. The corresponding overtriage rate (1 – specificity) varied from 82·8 per cent for KTS to 8·1 per cent for T‐RTS (Table 2).
Clinical usefulness
Compared with treating all patients as having major trauma, the prediction models showed benefit with a threshold above 0·2, corresponding to a weight of at least 1: 4 (Fig. 2). Of all models, KTS showed the greatest benefit. For thresholds below 0·2, it was most beneficial to treat all patients as having major trauma (so transporting all patients to a major trauma centre).
Figure 2.
Net benefit curves for different prehospital strategies for patients with suspected major trauma The graph shows the net benefit of using specific prehospital models to detect major trauma, based their optimal cut‐off in the validation data. Plots for the strategy of treating none of the patients as having major trauma (transporting no patients to a major trauma centre) or all patients as having major trauma are also shown. The x‐axis shows the threshold, defined as the ratio between the number of true‐positive (TP) and false‐positive (FP) patients (for example, a threshold of 0·2 means one is willing to accept 4 patients wrongly classified as having major trauma to identify 1 patient with major trauma). The number of FPs decreases as the threshold increases. The y‐axis shows the net benefit, defined as the proportion of TPs minus the proportion of FPs corrected for the weight (odds of threshold). MGAP, Mechanism, Glasgow Coma Scale, Age and Arterial Pressure; PSS, Physiologic Severity Score; T‐RTS, Triage Revised Trauma Score; PHI, Prehospital Index; mREMS, modified Rapid Emergency Medicine Score; KTS, Kampala Trauma Score.
Discussion
This study comprised an external validation of models developed to predict mortality and major trauma in injured patients in the prehospital setting. Most models were found to perform reasonably in predicting in‐hospital mortality but less well in identifying major trauma. None of the prediction models was able to obtain overtriage and undertriage rates conforming to the guidelines of the American College of Surgeons4. The present results are consistent with those for paediatric triage tools for injured children19, which were also unable to meet the recommended criteria.
Net benefit analysis showed that for thresholds between 0·2 and 0·5, meaning that trauma centres would be willing to accept a maximum of four patients wrongly classified as having major trauma for one correctly classified as having major trauma, KTS was the best model of those analysed and compared with treating all patients within the TARN registry as having major trauma. For thresholds below 0·2 (accepting more than 4 patients wrongly classified as having major trauma for 1 correctly classified as having major trauma), treating all patients as having major trauma seemed most beneficial.
All models, except MGAP, had a more than reasonable score (C‐statistic at least 0·70) in predicting in‐hospital mortality. The three best prediction models for in‐hospital mortality were mREMS, KTS and PSS. These models included the following predictors that seem to be most important for in‐hospital mortality: systolic BP, respiratory rate and level of consciousness or GCS score. The models also performed better in predicting in‐hospital mortality than the ISS. The three best prediction models for predicting major trauma were KTS, PSS and PHI. These models have the predictors systolic BP, respiratory rate and consciousness in common. KTS contains the number of serious injuries, an apparently important predictor that is lacking in all other models. The preferred model to implement would depend on the intended use of the models and the available predictors. The mREMS model showed the best discriminative ability in predicting in‐hospital mortality, whereas the KTS model was the best in distinguishing patients with major trauma from those without.
Most prediction models performed less well in this external validation than in their derivation population and other external validations for the outcome mortality. MGAP had an AUC of 0·90 in its derivation cohort16, compared with 0·602 in the present study; mREMS had an AUC value of 0·9717 compared with 0·793 here; and PSS had an AUC of 0·9315 compared with 0·741 in this study. These AUC values above 0·90 in the development setting imply overfitting of the model and explain the poorer performance in the present study population. External validations of T‐RTS varied between 0·83 and 0·8414, 16, 20, 21 compared with 0·706 in the present study. An external validation of PHI showed an AUC of 0·6613, 22 compared with 0·734 in this study. An external validation of KTS showed an AUC of 0·77 previously18, 23 compared with 0·769 here. The performance of the prehospital trauma scores seems very population‐ and setting‐dependent. Therefore, it might be important to validate models externally before using them in specific settings.
All models were identified through a systematic literature search and it was possible to include most existing prediction models in the validation study. The present validation cohort was very large with wide heterogeneity in patient characteristics, mechanism of injury and outcomes. By evaluating the different models in the same validation cohort, this study provides more information than separate validation studies. The PSS model has never been validated externally before6. The net benefit analysis has shown the importance of looking at different weights as they influence the potential benefit of the models.
One limitation of this study is the use of retrospective data restricted to injured patients who were admitted to hospital. Patients with minor trauma who were discharged immediately from the emergency department were not included in the validation cohort. This could have led to underestimation of the specificity and overestimated overtriage rates of all models, because such patients would have been true‐negatives. This makes the net benefit analyses applicable only to injured patients with higher chances of having major trauma because those who had experienced minor trauma were not included in TARN. A (clustered) RCT of different triage tools might be considered to examine the effect of such tools, for example to assess the effect of KTS in a high‐income country. However, a recent trial24, 25 that investigated the impact of trying to transport patients with head injury straight to neurosurgery showed that it remains difficult to study the effect of certain interventions.
It was not possible to validate the CRAMS model, which had high sensitivity (92 per cent) and specificity (98 per cent) in the original article12, owing to lack of variables and proxy variables. Nor was it possible to assess calibration, because recalibration of the models was necessary owing to missing information about the intercept. Furthermore, there was a lack of cut‐offs in predicted probabilities proposed in the original studies. Some prediction models mentioned only two categories with the corresponding predicted probabilities in their main article, which meant that the intercept and slope retrieved from the validation cohort were not very informative. In addition, there were some missing values in the prehospital variables, with larger percentages of missingness for the severely injured patients. This could have introduced bias, although missing values were handled using multiple imputation with a rich imputation model that included essentially all available information26.
Another limitation of this study was that all included prediction models were developed initially to predict mortality. However, as these prediction models are used clinically for triage in the prehospital setting, their discriminative abilities for predicting both in‐hospital mortality and major trauma were assessed. The model that performed most solidly in predicting both in‐hospital mortality and major trauma was the KTS, which was developed in Uganda23. As this triage model had never been tested in a high‐income environment before, it requires further validation in high‐income countries. As a result of the strict inclusion criteria of TARN, the large number of serious injuries created overtriage rates in KTS. Future studies considering KTS should include the prehospital opinion about the number of serious injuries.
This study has shown that most prehospital trauma models are inadequate in identifying patients with major trauma when this is defined by an ISS of more 15. None of these prediction models met the required undertriage and overtriage rates as defined by the American College of Surgeons (5 per cent undertriage and 25–35 per cent overtriage). These overtriage and undertriage rates imply that avoiding undertriage is considered five to seven times more important, corresponding to a threshold of 0·14. For this threshold, the preferred strategy would be to transport all injured patients to major trauma centres. However, this could cause overcrowding of emergency departments in such centres. The optimal threshold, and thus the cut‐offs used for triage decisions, might differ according to the individual trauma system; different hospitals might be able to tolerate different amounts of overtriage.
However, a universal definition of major trauma is still lacking. The injury severity measure used throughout trauma registries and research is the ISS, with a score of more than 15 defining severely injured or major trauma. Questions about the accuracy of the ISS have been raised. First, an equal AIS in different body regions is assumed to indicate similar injury severity27, 28. Second, the ISS does not account for multiple injuries in the same body region28, 29. This implies that patients with equal ISS scores could benefit differently from treatment in designated trauma centres. Future research should focus on which groups of injured patients could really benefit from designated trauma centres in terms of mortality and quality of life. This is an essential step before new triage tools can be developed.
Future studies should focus on the development of prehospital tools to identify patients who would benefit from treatment in a major trauma centre instead of focusing on the development of new prehospital tools to identify patients with an ISS exceeding 15.
Supporting information
Appendix S1. Search strategy
Fig. S1. Flow chart search strategy
Table S1. Prehospital prediction models
Acknowledgements
The authors thank T. Lawrence and O. Bouamra for help with extracting data from TARN. No preregistration exists for the studies reported in this article. C.A.S., E.J.A.W. and E.V. had full access to all data in the study, and take responsibility for the integrity of the data and the accuracy of the data analysis.
Disclosure: The authors declare no conflict of interest.
References
- 1. Polinder S, Haagsma JA, Toet H, van Beeck EF. Epidemiological burden of minor, major and fatal trauma in a national injury pyramid. Br J Surg 2012; 99(Suppl 1): 114–121. [DOI] [PubMed] [Google Scholar]
- 2. MacKenzie EJ, Rivara FP, Jurkovich GJ, Nathens AB, Frey KP, Egleston BL et al A national evaluation of the effect of trauma‐center care on mortality. N Engl J Med 2006; 354: 366–378. [DOI] [PubMed] [Google Scholar]
- 3. Demetriades D, Martin M, Salim A, Rhee P, Brown C, Chan L. The effect of trauma center designation and trauma volume on outcome in specific severe injuries. Ann Surg 2005; 242: 512–519. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Rotondo M, Cribari C, Smith R, American College of Surgeons - Committee on Trauma. Resources for Optimal Care of the Injured Patient. American College of Surgeons: Chicago, 2014. [Google Scholar]
- 5. Baker SP, O'Neill B. The injury severity score: an update. J Trauma 1976; 16: 882–885. [DOI] [PubMed] [Google Scholar]
- 6. Rehn M, Perel P, Blackhall K, Lossius HM. Prognostic models for the early care of trauma patients: a systematic review. Scand J Trauma Resusc Emerg Med 2011; 19: 17. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Collins GS, Moons KG. Comparing risk prediction models. BMJ 2012; 344: e3186. [DOI] [PubMed] [Google Scholar]
- 8. Moons KG, Donders RA, Stijnen T, Harrell FE Jr. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol 2006; 59: 1092–1101. [DOI] [PubMed] [Google Scholar]
- 9. Sterne JA, White IR, Carlin JB, Spratt M, Royston P, Kenward MG et al Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ 2009; 338: b2393. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Bamber D. The area above the ordinal dominance graph and the area below the receiver operating characteristic graph. J Math Psychol 1975; 12: 387–415. [Google Scholar]
- 11. Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ 2016; 352: i6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Gormican SP. CRAMS scale: field triage of trauma victims. Ann Emerg Med 1982; 11: 132–135. [DOI] [PubMed] [Google Scholar]
- 13. Koehler JJ, Baer LJ, Malafa SA, Meindertsma MS, Navitskas NR, Huizenga JE. Prehospital Index: a scoring system for field triage of trauma victims. Ann Emerg Med 1986; 15: 178–182. [DOI] [PubMed] [Google Scholar]
- 14. Champion HR, Sacco WJ, Copes WS, Gann DS, Gennarelli TA, Flanagan ME. A revision of the Trauma Score. J Trauma 1989; 29: 623–629. [DOI] [PubMed] [Google Scholar]
- 15. Husum H, Gilbert M, Wisborg T, Van Heng Y, Murad M. Respiratory rate as a prehospital triage tool in rural trauma. J Trauma 2003; 55: 466–470. [DOI] [PubMed] [Google Scholar]
- 16. Sartorius D, Le Manach Y, David JS, Rancurel E, Smail N, Thicoïpé M et al Mechanism, Glasgow Coma Scale, Age, and Arterial Pressure (MGAP): a new simple prehospital triage score to predict mortality in trauma patients. Crit Care Med 2010; 38: 831–837. [DOI] [PubMed] [Google Scholar]
- 17. Miller RT, Nazir N, McDonald T, Cannon CM. The modified rapid emergency medicine score: a novel trauma triage tool to predict in‐hospital mortality. Injury 2017; 48: 1870–1877. [DOI] [PubMed] [Google Scholar]
- 18. Kobusingye OC, Lett RR. Hospital‐based trauma registries in Uganda. J Trauma 2000; 48: 498–502. [DOI] [PubMed] [Google Scholar]
- 19. Cheung R, Ardolino A, Lawrence T, Bouamra O, Lecky F, Berry K et al The accuracy of existing prehospital triage tools for injured children in England – an analysis using trauma registry data. Emerg Med J 2013; 30: 476–479. [DOI] [PubMed] [Google Scholar]
- 20. Al‐Salamah MA, McDowell I, Stiell IG, Wells GA, Perry J, Al‐Sultan M et al; OPALS Study Group . Initial emergency department trauma scores from the OPALS study: the case for the motor score in blunt trauma. Acad Emerg Med 2004; 11: 834–842. [DOI] [PubMed] [Google Scholar]
- 21. Moore L, Lavoie A, Abdous B, Le Sage N, Liberman M, Bergeron E, Emond M. Unification of the revised trauma score. J Trauma 2006; 61: 718–722. [DOI] [PubMed] [Google Scholar]
- 22. Tamim H, Joseph L, Mulder D, Battista RN, Lavoie A, Sampalis JS. Field triage of trauma patients: improving on the Prehospital Index. Am J Emerg Med 2002; 20: 170–176. [DOI] [PubMed] [Google Scholar]
- 23. Weeks SR, Juillard CJ, Monono ME, Etoundi GA, Ngamby MK, Hyder AA et al Is the Kampala trauma score an effective predictor of mortality in low‐resource settings? A comparison of multiple trauma severity scores. World J Surg 2014; 38: 1905–1911. [DOI] [PubMed] [Google Scholar]
- 24. Lecky FE, Russell W, McClelland G, Pennington E, Fuller G, Goodacre S et al Bypassing nearest hospital for more distant neuroscience care in head‐injured adults with suspected traumatic brain injury: findings of the head injury transportation straight to neurosurgery (HITS‐NS) pilot cluster randomised trial. BMJ Open 2017; 7: e016355. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Lecky F, Russell W, Fuller G, McClelland G, Pennington E, Goodacre S et al The Head Injury Transportation Straight to Neurosurgery (HITS‐NS) randomised trial: a feasibility study. Health Technol Assess 2016; 20: 1–198. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Newgard CD. The validity of using multiple imputation for missing out‐of‐hospital data in a state trauma registry. Acad Emerg Med 2006; 13: 314–324. [DOI] [PubMed] [Google Scholar]
- 27. Chawda MN, Hildebrand F, Pape HC, Giannoudis PV. Predicting outcome after multiple trauma: which scoring system? Injury 2004; 35: 347–358. [DOI] [PubMed] [Google Scholar]
- 28. Köksal O, Ozdemir F, Bulut M, Aydin S, Almacioğlu ML, Ozgüç H. Comparison of trauma scoring systems for predicting mortality in firearm injuries. Ulus Travma Acil Cerrahi Derg 2009; 15: 559–564. [PubMed] [Google Scholar]
- 29. Tamim H, Al Hazzouri AZ, Mahfoud Z, Atoui M, El‐Chemaly S. The injury severity score or the new injury severity score for predicting mortality, intensive care unit admission and length of hospital stay: experience from a university hospital in a developing country. Injury 2008; 39: 115–120. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Appendix S1. Search strategy
Fig. S1. Flow chart search strategy
Table S1. Prehospital prediction models