Abstract
Objective
To develop and validate a prognostic model and a simple model to predict death due to bleeding in trauma patients.
Design
Cross-sectional study with multivariable logistic regression using data from two large trauma cohorts.
Setting
274 hospitals from 40 countries in the Clinical Randomisation of Anti-fibrinolytic in Significant Haemorrhage (CRASH-2) trial and 24 hospitals in the Northern French Alps Trauma registry.
Participants
13 485 trauma patients in the CRASH-2 trial and 9945 patients in the Northern French Alps Trauma registry who were admitted to hospital within 3 hours of injury.
Main outcome measure
In-hospital death due to bleeding within 28 days.
Results
There were 815 (6%) deaths from bleeding in the CRASH-2 trial and 102 (1%) in the Northern French Alps Trauma registry. The full model included age, systolic blood pressure (SBP), Glasgow Coma Scale (GCS), heart rate, respiratory rate and type of injury (penetrating). The simple model included age, SBP and GCS. In a cross-validation procedure by country, discrimination and calibration were adequate (pooled C-statistic 0.85 (95% CI 0.81 to 0.88) for the full model and 0.84 (95% CI 0.80 to 0.88) for the simple model).
Conclusion
This prognostic model can identify trauma patients at risk of death due to bleeding in a wide range of settings and can support prehospital triage and trauma audit, including audit of tranexamic acid use.
Keywords: trauma, injury, bleeding, haemorrhage, prognostic, coagulopathy, death, score, audit, human
Strengths and limitations of this study.
While there are models that predict all-cause mortality for trauma, this prognostic model is the first to identify trauma patients from a wide range of settings at risk of death due to bleeding.
We used a rigorous innovative method to develop and validate this prognostic model with an internal–external cross-validation method based on data from 41 countries to ensure that the result is widely applicable.
This model can support clinical decision-making for prehospital triage and for identifying population to audit to help implementation of effective intervention such as tranexamic acid.
As the objective of this model was to identify the population at risk of death due to bleeding, discrimination showed a good ability and homogenous results across countries.
Due to narrow range in the case-mix of some countries, we observed statistical heterogeneity in terms of calibration.
Introduction
Traumatic haemorrhage is responsible for about 2 million deaths each year and is a leading cause of preventable death in trauma.1–3 Early administration of tranexamic acid given within 3 hours of injury reduces death due to bleeding by about one-third.4 Tranexamic acid is widely included in trauma care guidelines.5 6
Nevertheless, many trauma patients who might benefit from tranexamic acid are not treated or are not treated soon enough.7 Despite an increase in tranexamic acid use after regionalisation of trauma services in England, 42% of bleeding trauma patients did not receive it.8 There are many ways to increase adherence to guidelines, ranging from education to financial incentives and regulation.9 Audit and feedback are particularly effective and have helped increase the timely use of reperfusion therapies in patients with myocardial infarction and stroke.10–13 Audit and feedback are also important in trauma care.14–16 Prognostic models estimate the risk of death for each patient and allow us to target the population to audit. We can also calculate a clinical score that can be used for initial triage. To allow the audit of tranexamic acid use in trauma patients, we developed and validated a prognostic model to predict the risk of death due to bleeding based on information available at the first clinical assessment.
Methods
Study population
We used data from two large multicentre studies to develop a widely applicable prognostic model for death due to bleeding in trauma patients: an international randomised control trial (The Clinical Randomisation of Anti-fibrinolytic in Significant Haemorrhage [CRASH-2] trial) and the Northern French Alps Trauma registry.17 18
The CRASH-2 trial included patients from 274 hospitals in 40 countries from 2005 to 2010. Patients with or at risk of significant bleeding within 8 hours of injury were included. Since tranexamic acid is effective only within 3 hours of injury, we excluded patients treated beyond 3 hours.
The Northern French Alps trauma registry, part of the Northern French Alps trauma system (TRENAU), includes 24 hospitals and 16 prehospital mobile intensive care units from 3 emergency medical service systems. Patients, from 2009 to2016, with major trauma according to the triage rules of the American College of Surgeons were included.19 We excluded patients with cardiac arrest at the scene of the injury.
Outcome and variable selection
The primary outcome was in-hospital death due to bleeding within 28 days. In the CRASH-2 trial, the clinician responsible recorded the cause of death. In the Northern French Alps registry, two trauma surgeons and two emergency physicians reviewed the records of all patients who died to determine the cause of death. We selected potential predictors from the CRASH-2 trial data collected before randomisation. We focused on the data available in the prehospital setting or on hospital admission in the Northern French Alps Trauma registry. These data included demographic characteristics (age, sex), physiological parameters (systolic blood pressure (SBP), heart rate (HR), respiratory rate (RR), Glasgow coma scale (GCS)) and the mechanism of injury (blunt or penetrating). All variables could be assessed at the first clinical assessment and were available in hospital records. Physiological variables were the first measures recorded. We also included treatment by tranexamic acid and country income level (high, middle or low income). Treatment by tranexamic acid was included in the equation for statistical adjustment. The coefficient for tranexamic acid treatment was constrained in the model equation to obtain a prediction before treatment at the first clinical assessment. Therefore, we used the entire dataset and not just the placebo arm of the CRASH-2 randomised trial. We assessed the importance of each predictor with the partial R2 statistic that estimates the variability of the outcome explained by the predictor. We developed two models. A full model that included all potential predictors and a simple model.
Model development
We used multivariable logistic regression with random effects by country to identify predictors of death due to bleeding. Continuous variables were included in the model as linear terms. We assessed departures from linearity by plotting the risk of death against continuous variables and added quadratic and cubic terms to the model for all continuous variables that showed a non-linear relationship graphically. The GCS was used as a continuous variable. We used a backward stepwise method by including all variables, quadratic and cubic terms and plausible interactions between the mechanism of injury and SBP, between the mechanism of injury and GCS, and between age and SBP. We then removed, one at a time, variables for which there was no evidence of association (p>0.05) from the Wald test. We also used the Least Absolute Shrinkage and Selecting Operator (LASSO) method to check that variable selection obtained by the ordinary least squares method was similar.20
Model performance
We assessed the model performance in terms of discrimination and calibration. Discrimination was assessed with the C-statistic and the receiving operating characteristic curve.21 Calibration was assessed as the difference between mean observed and predicted probabilities (calibration in the large) and by plotting observed outcome and predicted probabilities by decile of the predicted risk of death and with a non-parametric smooth function.22 We estimated the calibration slope based on the linear predictor of each model. A calibration slope of 1 and an intercept of 0 indicates perfect calibration. The overall calibration was summarised by the ratio of expected and observed number of events (E/O) with an ideal value of 1.23 A value <1 indicates an underprediction and a value > 1 indicates an overprediction.
Model validation
We performed internal validation to estimate the statistical optimism of the final model. We drew 200 bootstrapped samples of 23 402 patients. We developed a model in each bootstrapped sample including variable selection. We estimated the C-statistic in each bootstrapped sample and assessed the performance of each model in the original sample. Optimism was estimated as the mean of the difference between the C-statistic of the bootstrap sample and the C-statistic in the original sample. We subtracted optimism from the C-statistic of the model developed in the original sample to obtain the optimism-corrected C-statistic.
We also conducted an internal–external validation.24–26 We performed a cross-validation procedure where we selected countries with a sample size > 300.25 27 We left out one country in turn and developed models using the same predictors in the remaining countries and estimated the discrimination and calibration in the omitted country. C-statistics, calibration slope and overall calibration for each country were pooled with random effects. We assessed heterogeneity with I2 statistics and by testing interaction between calibration slope and country.
Missing data
There was no loss to follow-up in the CRASH-2 trial and <0.3% in the Northern French Alps Trauma registry. There were 0% to 2% missing values for predictors in the CRASH-2 trial and 0% to 5% in the Northern French Alps Trauma registry. We performed multiple imputation by chained equations to fill in the missing values of predictors.28 We generated 20 imputed datasets. We imputed 2253 missing values (1.6%) for 1317 incomplete observations.
Patient and public involvement statement
Patients were not involved in the research question and in the design of the study.
All analyses were performed using STATA software V.14.0; and R software V.3.4.3 (R foundation for statistical computing).
Results
We included 23 430 trauma patients in the study (13 485 in the CRASH-2 trial and 9945 in the Northern French Alps registry, tables 1 and 2). In both the CRASH-2 and Northern French Alps cohorts, the patients were mostly men with a median age of 30 and 35 years respectively. Patients who died from bleeding had lower SBP, lower GCS scores and higher HRs. Penetrating injury was more frequent in the CRASH-2 trial patients (51%) than in the Northern French Alps (5%). Eight hundred and fifteen patients (6%) died from bleeding in the CRASH-2 trial and 102 (1%) in the Northern French Alps cohorts (table 3). Half of the Northern French Alps patients had an Injury Severity Score (ISS) of 16 or more and three quarters had an ISS of 9 or more.
Table 1.
Characteristics of the CRASH-2 trial patients
| Missing (%) | All patients n=13 485 | Alive n=11 404 |
All causes of death n=2081 | Death due to bleeding n=815 | |
| Age, median (IQR) | 0 | 30 (24–42) | 30 (23–41) | 34 (25–46) | 32 (25–45) |
| SBP, median (IQR) | 2 | 90 (80–110) | 95 (80–110) | 80 (70–100) | 77 (60–90) |
| HR, median (IQR) | 1 | 106 (92–120) | 105 (90–120) | 112 (98–128) | 116 (100–130) |
| RR, median (IQR) | 1 | 22 (20–26) | 22 (20–26) | 24 (20–30) | 24 (20–30) |
| GCS, n (%) | 0 | ||||
| 3–8 | 2125 (16%) | 1030 (9%) | 1094 (53%) | 360 (35%) | |
| 9–12 | 1784 (13%) | 1451 (13%) | 332 (16%) | 171 (21%) | |
| 13–15 | 9578 (71%) | 8918 (78%) | 654 (31%) | 360 (44%) | |
| Penetrating Injury, n (%) | 0 | 6874 (51%) | 5958 (52%) | 916 (44%) | 485 (60%) |
CRASH-2, Clinical Randomisation of Anti-fibrinolytic in Significant Haemorrhage; GCS, Glasgow Coma Scale; HR, heart rate (bpm); RR, respiratory rate (bpm); SBP, systolic blood pressure (mm Hg).
Table 2.
Characteristics of the Northern French Alps registry
| Missing (%) | All patients n=9945 | Alive n=9256 | All causes of death n=661 | Death due to bleeding n=102 | |
| Age, median (IQR) | <1 | 36 (22–53) | 35 (22–51) | 58 (31–73) | 51 (31–68) |
| SBP, median (IQR) | 3 | 124 (110–140) | 125 (111–140) | 116 (80–140) | 83 (60–110) |
| HR, median (IQR) | 4 | 84 (74–100) | 85 (75–100) | 84 (60–110) | 97 (60–120) |
| RR, median (IQR) | 4 | 16 (15–20) | 16 [15-–20] | 15 (14–20) | 17 (11–25) |
| GCS, n (%) | 3 | ||||
| 3–8 | 1170 (12) | 718 (8) | 449 (70) | 51 (52) | |
| 9–12 | 500 (5) | 452 (5) | 48 (7) | 10 (10) | |
| 13–15 | 7984 (83) | 7813 (87) | 148 (23) | 37 (38) | |
| Penetrating injury | <1 | 554 (6) | 508 (6) | 45 (7) | 16 (16) |
| Injury severity Score, n (%) | 2 | ||||
| 0–8 | 2738 (28) | 2723 (30) | 14 (2) | 1 (1) | |
| 9–15 | 2480 (26) | 2450 (27) | 26 (4) | 6 (6) | |
| 16–24 | 2081 (21) | 2008 (22) | 68 (11) | 15 (15) | |
| 25–34 | 1778 (18) | 1453 (16) | 316 (49) | 36 (36) | |
| >35 | 686 (7) | 465 (5) | 221 (34) | 41 (41) |
GCS, Glasgow Coma Scale; HR, heart rate (bpm); RR, respiratory rate (bpm); SBP, systolic blood pressure (mm Hg).
Table 3.
Risk of death and intervention
| CRASH-2 n (%) |
Northern French Alps Trauma registry n (%) | |
| Death due to bleeding | 815 (6) | 102 (1) |
| Overall death | 2081 (15) | 661 (7) |
| Admission in ICU | 5354 (40) | 4205 (42) |
| Surgical procedure | 6608 (49) | 2691 (27) |
| Surgical procedure for bleeding | 916 (7) | 1251 (12)* |
| Blood transfusion, | 6506 (48) | 1054 (11) |
| ICU median day (IQR) | 3 (1–7) | 4 (2–10) |
*including embolisation
CRASH-2, Clinical Randomisation of Anti-fibrinolytic in Significant Haemorrhage; ICU, intensive care unit.
Figure 1 shows the relationships between the potential predictors and death due to bleeding. The risk of death due to bleeding was higher with higher age, lower SBP and lower GCS. HR and RR showed U-shaped relations. The predictors included in the full model were age, SBP, GCS, HR, RR and the mechanism of injury. Sex and country income were not associated with death due to bleeding in multivariable analysis (online supplementary 1 and 2). The LASSO method gave similar results. Age, SBP and GCS had the strongest prognostic value according to partial R2. The performance of the model development showed good discrimination with C-statistics of 0.88 (0.87 to 0.89) and 0.87 (0.86 to 0.88) for the full and simple models respectively (table 4). Calibration was good with no differences between observed and predicted deaths due to bleeding, except for high-risk patients (n=138) in whom the risk was over-estimated above a predicted probability of 0.5 (figure 2). Bootstrap resampling showed negligible model optimism of 0.0023 and gave an optimism-corrected performance that was unchanged with a C-statistic of 0.88 and 0.87 for the full and simple models. At internal–external cross-validation, the C-statistic ranged from 0.80 to 0.94, except for India with a C-statistic of 0.72 (figure 3). The pooled C-statistics were 0.85 (0.81 to 0.88) and 0.84 (0.80 to 0.88) for the full and simple models respectively (table 4). Pooled calibration slope was 1.07 (0.91 to 1.24) and 1.12 (0.95 to 1.29). Calibration slope and overall calibration showed heterogeneity, especially for Iraq, Georgia and Indonesia (figures 4 and 5). We found a significant interaction between calibration slope and country (p<0.001).
Figure 1.
Relationship between death due to bleeding and potential predictors.
Table 4.
Model performance, internal and internal–external validation.
| Full model | Simple model | |||
| Development | Internal–external validation* | Development | Internal–external validation* | |
| n=23 402 | n=22 422 | n=23 402 | n=22 422 | |
| C- statistic (AUC) | 0.88 (0.87–0.89) | 0.85 (0.81–0.88) | 0.87 (0.86–0.88) | 0.84 (0.80–0.88) |
| Calibration-in-the-large† | <0.1 | <0.1 | <0.1 | 0.3 (0.1–0.6) |
| Calibration slope | 1.01 (0.96–1.07) | 1.07 (0.91–1.14) | 1.04 (0.98–1.09) | 1.12 (0.95–1.29) |
| E/O | 1.02 (0.96–1.08) | 0.93 (0.71–1.15) | 0.98 (0.92–1.04) | 0.91 (0.82–0.99) |
*Internal–external validation based on pooled data with random effect obtained by cross validation from 13 countries (each with n≥300). Every country is left out once for validation of a model based on the remaining countries.
†Calibration-in-the-large showed difference between observed and predicted death due to bleeding.
AUC, area under the curve (C-statistic); E/O: expected/observed number of deaths due to bleeding.
Figure 2.
Calibration curves for model development. AUC, area under the curve.
Figure 3.
Internal–external cross-validation C-statistics by countries. AUC, area under the curve.
Figure 4.
Internal–external cross-validation of calibration slope by countries.
Figure 5.
Internal–external cross-validation overall calibration expected and observed number of deaths due to bleeding (E/O) by countries.
bmjopen-2018-026823supp001.pdf (49.4KB, pdf)
Discussion
Main findings
We developed and internationally validated a prognostic model to predict death due to bleeding in trauma patients. The model showed good discrimination and calibration in a wide range of settings. By using clinical parameters that can be assessed at the site of injury and available in hospital records, we can accurately estimate the risk of death due to bleeding in a population with major trauma.
Strengths and limitations
This study has several strengths. We used data from well-described inception cohorts of bleeding trauma patients with or at risk of significant haemorrhage. Prognostic factors collected correspond to the first measure recorded after injury. Unlike previous studies, loss to follow-up was minimal.29 We used a well-defined outcome at a fixed time point after injury. These strengths help ensure the internal validity of the model.
We developed our model in a large international cohort with patients from 40 countries and a large trauma registry. This helps to ensure that our results are widely applicable. We did not split the data randomly or use separate derivation and validation cohorts. Because the number of outcome events is the limiting factor in prognostic studies, we used the full dataset with more than 900 traumatic deaths due to bleeding to ensure accurate prediction and strengthen internal validity. Splitting the data could have led to a pessimistic and unstable estimate of performance.30 For this reason, we did not perform split-sample validation and preferred to perform internal-external cross-validation that has been recommended for assessing generalisability.24 We also performed boostraping that helps to estimate the model optimism. However, we welcome further external validation in different trauma cohorts by different authors.31
Our study also has limitations. We cannot rule out misclassification of the outcome. The cause of death can be difficult to determine, especially in late bleeding deaths that could be confused with thrombotic disseminated intravascular coagulation (DIC).6 If deaths due to DIC were misclassified as deaths due to bleeding, this might underestimate the effect of SBP, HR or RR in this model.
Another limitation was the potential for measurement error in prognostic factors. The use of a single measurement for blood pressure rather than the average of several measurements could lead to error and regression dilution bias.32 The regression line between outcome and predictor is fitted in order to minimise the distance between each point and the line. The random error of the predictor increases the distance to the regression line and underestimates the effect of the predictor by flattening the regression line.33 This may explain the over-prediction in high risk patients. Patients with haemorrhagic shock and haemodynamic instability are more likely to have blood pressure variation and, hence, measurement error. This over-prediction occurred only for trauma patients with a very high predicted risk of death due to bleeding (above 0.45), representing <0.6% of the study population (n=138). In these very high-risk patients, precise quantification of the risk of death is unlikely to influence clinical decisions. On the other hand, accurate prediction is clinically important in low-risk patients, as, for example, it may determine who receives tranexamic acid.
Finally, we observed heterogeneity of performance across countries. We note that the discriminative ability is affected by miscalibration and case-mix.26 The relatively poor C-statistic in India could be explained by the combination of calibration slope below 1 and a relatively homogenous case-mix. On the other hand, the high C-statistic in France reflected that the Northern French Alps trauma registry selected a more heterogeneous case-mix population with major trauma. We acknowledge that this model is suitable for a population similar to that used in this study, such as a population with major trauma.
Implications of study
Our prognostic model provides a way of identifying trauma patients with or at risk of significant haemorrhage based on predicted probabilities of death due to bleeding. Quality improvement programmes could use this model to estimate the individual risks of death due to bleeding in a trauma population. Based on these predictions, a trauma audit could determine a threshold for patients with ‘significant haemorrhage’ who should be treated with tranexamic acid. The threshold used may depend on effectiveness, cost and safety considerations. According to European guidelines for the management of traumatic bleeding, tranexamic acid is supported by the highest level of evidence (grade 1A).5 Tranexamic acid costs about one pound per patient and has no serious adverse effects. For these reasons, a low predicted risk of bleeding death might be used in trauma audit.
An internet application has been prepared using our simple model for use in the prehospital setting (www.evidencio.com). This could help paramedics decide who should receive tranexamic acid at the scene of injury. It could also be useful in prehospital triage. Some previously proposed trauma scores predict all-cause mortality or massive transfusion.29 34 35 Ours is the only model that predicts death due to bleeding. Because bleeding is the leading cause of preventable death, the model might become an essential tool for identifying patients needing urgent interventions such as damage control surgery and multispeciality critical care. It could also help identify patients who need to be transported directly to a regional trauma centre or for whom massive transfusion protocol needs to be activated before they arrive at the hospital.
A prognostic model predicting all-cause mortality was developed previously using CRASH-2 data.36 However, traumatic deaths can result from many different pathophysiological mechanisms. For example, both high and low SBP predict death from all causes but only low blood pressure predicts death due to bleeding. The association of high blood pressure with all-cause mortality is likely to reflect deaths from traumatic brain injury. By combining different mechanisms of death, predictions based on all-cause mortality could misclassify the risk of death from bleeding.
Future studies
Our models may facilitate stratification of clinical trial populations into risk categories at baseline. Future studies may examine if and how the effect of tranexamic acid varies with baseline risk and model the health impact of different treatment strategies.
Supplementary Material
Footnotes
Contributors: FXA, IR designed the study. FXA, PB, IR designed and monitored the data collection from which this paper was developed. FXA, AGA analysed the data. PB gave feedback about the clinical use. ES gave feedback and statistical advice. FXA, IR wrote the first draft. FXA, AGA, ES, PB, IR contributed to write and revised the paper.
Funding: The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.
Competing interests: None declared.
Ethics approval: The CRASH-2 trial received ethics committee approval from the London School of Hygiene & Tropical Medicine, UK and the ethics committees of all participating hospitals. The Northern French Alps Trauma Registry was approved by the ethics committee of the university hospital of Clermont-Ferrand, France.
Provenance and peer review: Not commissioned; externally peer reviewed.
Data sharing statement: Data from the CRASH-2 trial is available via freeBIRD (free bank of injury and emergency research data), hosted by the Clinical Trial Unit of the London School of Hygiene and Tropical Medicine. More information at.
Patient consent for publication: Not required.
References
- 1. GBD 2013 Mortality and Causes of Death Collaborators. Global, regional, and national age–sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. The Lancet 2013;2015:117–71. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Esposito TJ, Sanddal TL, Reynolds SA, et al. Effect of a voluntary trauma system on preventable death and inappropriate care in a rural state. J Trauma 2003;54:663–70. discussion 669-670 10.1097/01.TA.0000058124.78958.6B [DOI] [PubMed] [Google Scholar]
- 3. Kleber C, Giesecke MT, Tsokos M, et al. Trauma-related preventable deaths in Berlin 2010: need to change prehospital management strategies and trauma management education. World J Surg 2013;37:1154–61. 10.1007/s00268-013-1964-2 [DOI] [PubMed] [Google Scholar]
- 4. Roberts I, Shakur H, Afolabi A, et al. CRASH-2 collaborators. The importance of early treatment with tranexamic acid in bleeding trauma patients: an exploratory analysis of the CRASH-2 randomised controlled trial. Lancet 2011;377:1096–101. 1101.e1-2 10.1016/S0140-6736(11)60278-X [DOI] [PubMed] [Google Scholar]
- 5. Rossaint R, Bouillon B, Cerny V, et al. The European guideline on management of major bleeding and coagulopathy following trauma: fourth edition. Crit Care 2016;20:100 10.1186/s13054-016-1265-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Ciesla DJ, Pracht EE, Tepas JJ, et al. Measuring trauma system performance: Right patient, right place-Mission accomplished? J Trauma Acute Care Surg 2015;79:263–8. 10.1097/TA.0000000000000660 [DOI] [PubMed] [Google Scholar]
- 7. Coats TJ, Fragoso-Iñiguez M, Roberts I. Implementation of tranexamic acid for bleeding trauma patients: a longitudinal and cross-sectional study. Emerg Med J 2019;36:78–81. 10.1136/emermed-2018-207693 [DOI] [PubMed] [Google Scholar]
- 8. Metcalfe D, Perry DC, Bouamra O, et al. Regionalisation of trauma care in England. Bone Joint J 2016;98-B:1253–61. 10.1302/0301-620X.98B9.37525 [DOI] [PubMed] [Google Scholar]
- 9. Angus DC, Black N. Improving care of the critically ill: institutional and health-care system approaches. Lancet 2004;363:1314–20. 10.1016/S0140-6736(04)16007-8 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Jamtvedt G, Young JM, Kristoffersen DT, et al. Does telling people what they have been doing change what they do? A systematic review of the effects of audit and feedback. Qual Saf Health Care 2006;15:433–6. 10.1136/qshc.2006.018549 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Ivers N, Jamtvedt G, Flottorp S, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev 2012:CD000259 10.1002/14651858.CD000259.pub3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Ludman PF, de Belder MA, McLenachan JM, et al. The importance of audit to monitor applications of procedures and improve primary angioplasty results. EuroIntervention 2012;8(Suppl P):P62–P70. 10.4244/EIJV8SPA11 [DOI] [PubMed] [Google Scholar]
- 13. Strbian D, Michel P, Ringleb P, et al. Relationship Between Onset-to-Door Time and Door-to-Thrombolysis Time. Stroke 2013;44:2808–13. 10.1161/STROKEAHA.113.000995 [DOI] [PubMed] [Google Scholar]
- 14. McDermott FT. Trauma audit and quality improvement. Aust N Z J Surg 1994;64:147–54. 10.1111/j.1445-2197.1994.tb02168.x [DOI] [PubMed] [Google Scholar]
- 15. Patel HC, Bouamra O, Woodford M, et al. Trends in head injury outcome from 1989 to 2003 and the effect of neurosurgical care: an observational study. The Lancet 2005;366:1538–44. 10.1016/S0140-6736(05)67626-X [DOI] [PubMed] [Google Scholar]
- 16. Shafi S, Barnes SA, Rayan N, et al. Compliance with recommended care at trauma centers: association with patient outcomes. J Am Coll Surg 2014;219:189–98. 10.1016/j.jamcollsurg.2014.04.005 [DOI] [PubMed] [Google Scholar]
- 17. Shakur H, Roberts I, Bautista R, et al. Effects of tranexamic acid on death, vascular occlusive events, and blood transfusion in trauma patients with significant haemorrhage (CRASH-2): a randomised, placebo-controlled trial. Lancet Lond Engl 2010;376:23–32. [DOI] [PubMed] [Google Scholar]
- 18. Bouzat P, Ageron FX, Brun J, et al. A regional trauma system to optimize the pre-hospital triage of trauma patients. Crit Care 2015;19:111 10.1186/s13054-015-0835-7 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Sasser SM, Hunt RC, Faul M, et al. Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011. MMWR Recomm Rep Morb Mortal Wkly Rep Recomm Rep Cent Dis Control 2012;61(RR-1):1–20. [PubMed] [Google Scholar]
- 20. Tibshirani R. Regression Shrinkage and Selection Via the Lasso. J R Stat Soc Ser B Methodol 1996;58:267–88. 10.1111/j.2517-6161.1996.tb02080.x [DOI] [Google Scholar]
- 21. Royston P, Altman DG. Visualizing and assessing discrimination in the logistic regression model. Stat Med 2010;29:2508–20. 10.1002/sim.3994 [DOI] [PubMed] [Google Scholar]
- 22. Steyerberg EW, Vickers AJ, Cook NR, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128–38. 10.1097/EDE.0b013e3181c30fb2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Riley RD, Ensor J, Snell KI, et al. External validation of clinical prediction models using big datasets from e-health records or IPD meta-analysis: opportunities and challenges. BMJ 2016;353:i3140 10.1136/bmj.i3140 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Steyerberg EW, Harrell FE. Prediction models need appropriate internal, internal-external, and external validation. J Clin Epidemiol 2016;69:245–7. 10.1016/j.jclinepi.2015.04.005 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Royston P, Parmar MK, Sylvester R. Construction and validation of a prognostic model across several studies, with an application in superficial bladder cancer. Stat Med 2004;23:907–26. 10.1002/sim.1691 [DOI] [PubMed] [Google Scholar]
- 26. Austin PC, van Klaveren D, Vergouwe Y, et al. Validation of prediction models: examining temporal and geographic stability of baseline risk and estimated covariate effects. Diagn Progn Res 2017;1:12 10.1186/s41512-017-0012-3 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Steyerberg EW, Mushkudiani N, Perel P, et al. Predicting outcome after traumatic brain injury: development and international validation of prognostic scores based on admission characteristics. PLoS Med 2008;5:e165 10.1371/journal.pmed.0050165 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med 2011;30:377–99. 10.1002/sim.4067 [DOI] [PubMed] [Google Scholar]
- 29. Rehn M, Perel P, Blackhall K, et al. Prognostic models for the early care of trauma patients: a systematic review. Scand J Trauma Resusc Emerg Med 2011;19:17 10.1186/1757-7241-19-17 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Austin PC, Steyerberg EW, variable Eper. Events per variable (EPV) and the relative performance of different strategies for estimating the out-of-sample validity of logistic regression models. Stat Methods Med Res 2017;26:796–808. 10.1177/0962280214558972 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Siontis GC, Tzoulaki I, Castaldi PJ, et al. External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. J Clin Epidemiol 2015;68:25–34. 10.1016/j.jclinepi.2014.09.007 [DOI] [PubMed] [Google Scholar]
- 32. MacMahon S, Peto R, Collins R, et al. Blood pressure, stroke, and coronary heart disease. The Lancet 1990;335:765–74. [DOI] [PubMed] [Google Scholar]
- 33. Hutcheon JA, Chiolero A, Hanley JA. Random measurement error and regression dilution bias. BMJ 2010;340:c2289 10.1136/bmj.c2289 [DOI] [PubMed] [Google Scholar]
- 34. Sartorius D, Le Manach Y, David JS, et al. Mechanism, glasgow coma scale, age, and arterial pressure (MGAP): a new simple prehospital triage score to predict mortality in trauma patients. Crit Care Med 2010;38:831–7. 10.1097/CCM.0b013e3181cc4a67 [DOI] [PubMed] [Google Scholar]
- 35. Yücel N, Lefering R, Maegele M, et al. Trauma Associated Severe Hemorrhage (TASH)-Score: probability of mass transfusion as surrogate for life threatening hemorrhage after multiple trauma. J Trauma 2006;60:1228–37. discussion 1236-1237 10.1097/01.ta.0000220386.84012.bf [DOI] [PubMed] [Google Scholar]
- 36. Perel P, Prieto-Merino D, Shakur H, et al. Predicting early death in patients with traumatic bleeding: development and validation of prognostic model. BMJ 2012;345:e5166 10.1136/bmj.e5166 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
bmjopen-2018-026823supp001.pdf (49.4KB, pdf)