Development and Validation of a Prediction Model for Prehospital Triage of Trauma Patients

Key Points

Question

Can a new prehospital prediction model for trauma triage lower the undertriage rate to approximately 10%, with a maximum overtriage rate of 50%?

Findings

In this multicenter cohort study that included 4950 patients with trauma, 8 highly significant predictors associated with injury severity were selected for the prediction model. The new prehospital trauma triage prediction model was externally validated and may lower the undertriage rate to 11.2% with an overtriage rate of 50.0%.

Meaning

This prediction model can be integrated in a mobile app for emergency medical services professionals to improve the prehospital trauma triage.

This multicenter cohort study develops and validates a new prehospital trauma triage protocol based on prehospital predictors associated with severe injury to improve current triage rates.

Abstract

Importance

Prehospital trauma triage protocols are used worldwide to get the right patient to the right hospital and thereby improve the chance of survival and avert lifelong disabilities. The American College of Surgeons Committee on Trauma set target levels for undertriage rates of less than 5%. None of the existing triage protocols has been able to achieve this target in isolation.

Objective

To develop and validate a new prehospital trauma triage protocol to improve current triage rates.

Design, Setting, and Participants

In this multicenter cohort study, all patients with trauma who were 16 years and older and transported to a trauma center in 2 different regions of the Netherlands were included in the analysis. Data were collected from January 1, 2012, through June 30, 2014, in the Central Netherlands region for the design data cohort and from January 1 through December 31, 2015, in the Brabant region for the validation cohort. Data were analyzed from May 3, 2017, through July 19, 2018.

Main Outcomes and Measures

A new prediction model was developed in the Central Netherlands region based on prehospital predictors associated with severe injury. Severe injury was defined as an Injury Severity Score greater than 15. A full-model strategy with penalized maximum likelihood estimation was used to construct a model with 8 predictors that were chosen based on clinical reasoning. Accuracy of the developed prediction model was assessed in terms of discrimination and calibration. The model was externally validated in the Brabant region.

Results

Using data from 4950 patients with trauma from the Central Netherlands region for the design data set (58.3% male; mean [SD] age, 47 [21] years) and 6859 patients for the validation Brabant region (52.2% male; mean [SD] age, 51 [22] years), the following 8 significant predictors were selected for the prediction model: age; systolic blood pressure; Glasgow Coma Scale score; mechanism criteria; penetrating injury to the head, thorax, or abdomen; signs and/or symptoms of head or neck injury; expected injury in the Abbreviated Injury Scale thorax region; and expected injury in 2 or more Abbreviated Injury Scale regions. The prediction model showed a C statistic of 0.823 (95% CI, 0.813-0.832) and good calibration. The cutoff point with a minimum specificity of 50.0% (95% CI, 49.3%-50.7%) led to a sensitivity of 88.8% (95% CI, 87.5%-90.0%). External validation showed a C statistic of 0.831 (95% CI, 0.814-0.848) and adequate calibration.

Conclusions and Relevance

The new prehospital trauma triage prediction model may lower undertriage rates to approximately 10% with an overtriage rate of 50%. The next step should be to implement this prediction model with the use of a mobile app for emergency medical services professionals.

Introduction

In first-world countries, systems of trauma care substantially reduce mortality associated with injury.^1,2,3 Multiple studies^2,3 focused on optimizing such trauma systems by balancing timely access to expert care, the ability of health care professionals and teams to attain and sustain the necessary expertise, and the cost-effectiveness of the overall trauma system. Fundamental to the trauma system is prehospital trauma triage, the goal of which is to identify at-risk patients and provide early and resuscitative care while transporting the patient to the highest appropriate level of care.⁴ Identification of severely injured patients is challenging. Only 0.5% of those injured are severely injured.⁴ Other challenges include assessment of the incident scene and the patient’s physiological state and risk of deterioration, identification of obvious injuries, and consideration of adjuvant factors, such as age.

Emergency medical services (EMS) professionals must differentiate between patients on the scene, often in adverse situations, without use of advanced diagnostic tools. Therefore, the EMS professionals are often forced to perform the prehospital trauma triage based on incomplete data. The importance of prehospital trauma triage cannot be understated; a structured and reliable process is crucial.

Worldwide, protocols are used to help identify severely injured patients. However, none of the existing protocols can achieve the recommended triage rates.^5,6,7 All are simplistic and static tools, whereas patients are dynamic, and more advanced methods are available to use on the scene, such as a prediction model in a mobile app.^8,9

Undertriage occurs when severely injured patients are not transported to a higher-level trauma center and results in delayed care and increased mortality and morbidity.^1,2 Overtriage occurs when patients without severe injuries are taken to a higher-level trauma center, often incurring preventable cost and resource consumption.^3,10 In the Netherlands, level I trauma centers are considered higher-level trauma centers and levels II and III are considered lower-level trauma centers. The American College of Surgeons Committee on Trauma set target levels of undertriage at less than 5% and overtriage as great as 25% to 35%.¹¹ The National Health Institute of the Netherlands recommends an undertriage rate of less than 10%.¹² A target for the overtriage rate was not set; the overtriage rate, however, should depend on the regional circumstances and could be as great as 50%.

In the Netherlands, a protocol based on the Field Triage Decision Scheme is used nationwide (eMethods in the Supplement).¹³ The mean undertriage rate was 33% in 2015.¹⁴ A recent study¹⁵ showed an undertriage rate of 22% and an overtriage rate of 31% in a single inclusive trauma system. At present, existing protocols achieve undertriage rates ranging from 22% to 44%, with overtriage rates ranging from 11% to 22% in a general trauma population.^6,16,17 The aim of the present study was therefore to develop and validate a new prehospital trauma triage prediction model that attempts to lower the undertriage rate to approximately 10%, with a maximum overtriage rate of 50%.

Methods

Study Design and Setting

In the Netherlands, each ambulance service serves a region. In this prospective, multicenter cohort study, all adult patients with trauma and all trauma centers in 2 different regions were included. Data from the Central Netherlands region were used to develop a diagnostic prediction model that was externally validated using the data of the Brabant region. This study was judged by the Medical Ethical Committee of University Medical Center Utrecht, Utrecht, the Netherlands, as not subject to the Medical Research Involving Human Subjects Act and therefore exempt from the need for informed consent.

The Central Netherlands region has 1 level I trauma center, the University Medical Center Utrecht, and 7 level II or III trauma centers. The region covers 535 square miles and has 1.2 million residents. Brabant has 1 level I trauma center, Elisabeth-TweeSteden Hospital, Tilburg, and 10 level II or III trauma centers. The region covers 1343 square miles and has 1.7 million residents. In both regions, the Dutch National Ambulance Care Protocol is used (eMethods in the Supplement).¹³ Patients transported across the borders of these regions were excluded because data were unavailable.

Patients

All patients with trauma who were 16 years or older, determined to be highest priority (with flashing lights and siren) by the dispatch center, and transported to a trauma center in 1 of the 2 regions underwent evaluation. In the Central Netherlands, patients were included from January 1, 2012, through June 30, 2014.¹⁵ In Brabant, patients were included from January 1 through December 31, 2015.

Outcomes and Definitions

Independent predictors associated with severe injury were identified to create a prediction model consisting of a limited number of variables. A severely injured patient was defined as a patient with an Injury Severity Score (ISS) greater than 15 (range, 0-75).

Undertriage was defined as the proportion of severely injured patients transported to a level II or III trauma center. Overtriage was defined as the proportion of patients without severe injuries transported to a level I trauma center. The protocol allows for these patients to be transported to a level I trauma center if it happens to be the nearest hospital.

Data Sources

Prehospital reports from EMS professionals were prospectively collected and included patient demographics, vital signs, description of the trauma mechanism, on-scene physical examination data (including suspected injured body region), and the receiving hospital. The body region suspected of injury by EMS professionals was divided into the head and neck, face, thorax, abdomen, extremities, or skin and others. These regions were chosen based on the categorization of the Abbreviated Injury Scale (AIS) regions that make up the ISS.

The Dutch National Trauma Database registered injuries for all patients admitted to a hospital. The Central Netherlands patient data were also extracted from the electronic patient documentation for patients discharged directly from the emergency department (ED). The injuries were recorded after discharge or 30 days after admission and coded by trained data managers in both regions (including E.A.J.v.R. and F.J.V.). Before 2015, all injuries were coded using the AIS 1990 Update 98 (AIS98); from 2015 and on, the AIS 2005 Update 2008 (AIS08) was used. Therefore, the injuries were coded according to the AIS98 for the Central Netherlands database and according to the AIS08 for the Brabant database. The ISS was calculated based on AIS scores to determine injury severity.

Missing Data

Missing data were analyzed and considered to be missing at random. Multiple imputation by chained equations was used for both regions separately to account for missing prehospital variables rather than deleting patients who had the most data available.¹⁸ Missing values were imputed based on all other predictors as well as the ISS. For both regions, respiratory rate, systolic blood pressure, oxygen saturation level, and Glasgow Coma Scale (GCS) score were imputed. Pulse was imputed for the Central Netherlands region only because this variable was missing in the Brabant data. For the Brabant region, ISS was only available for admitted patients. An ISS less than 15 was assumed for patients discharged from the ED because a previous study¹⁵ demonstrated that all severely injured patients (ISS >15) were admitted or died in the ED.

Statistical Analysis

Data were analyzed from May 3, 2017, through July 19, 2018. Frequencies with percentages were used to describe nominal and ordinal variables, whereas means and SDs were used to describe continuous variables. Bivariable binary logistic regression was used to explore potential predictors associated with severe injury (ISS >15). Analyses were performed on 5 imputed data sets independently and pooled using Rubin’s rules, if applicable.¹⁹

To ensure practical applicability, the maximum number of predictors was limited to 8. A full-model strategy with clinically relevant variables was used to develop the prediction model. To improve the accuracy for future patients and other regions, penalized maximum likelihood estimation was used.²⁰ Penalized maximum likelihood estimation is a rigorous estimation method that potentially results in better generalizability, model reduction, and differential shrinkage of coefficients.²¹ The functional forms of all continuous predictors were defined before modeling. Restricted cubic splines were used to model nonlinear predictors.

The performance of the final prediction model was expressed in terms of discrimination and calibration. Discriminative value was quantified by the C statistic. The receiver operating characteristic curve was plotted and a predefined value for specificity—an overtriage rate of 50%—was used to determine a cutoff point. Calibration was graphically assessed using a calibration plot.

The final model was externally validated with the Brabant data. Owing to heterogeneity between trauma regions (eg, prevalence of severe injury), the model needed to be recalibrated by updating the intercept for the Brabant region.^22,23 Calibration and discrimination of the prediction model were assessed in the validation process. Bootstrapped 95% CIs based on percentiles and 1000 resamples were calculated for C statistics and accuracy metrics. Two-sided P values were calculated using univariable logistic regression, and P < .05 indicated significance. All statistical analyses were performed using R software (version 3.2.4; R Foundation for Statistical Computing).²⁴

Results

Design Data Set

A total of 4950 adult patients with trauma were included for the Central Netherlands region constituting the design data set. To account for missing data, multiple imputation was used for respiratory rate in 324 (6.5%), systolic blood pressure in 345 (7.0%), oxygen saturation level in 662 (13.4%), and GCS score in 230 (4.6%). Mean patient age was 47 (21) years; 2887 (58.3%) were male and 2063 (41.7%) were female; and 435 (8.8%) had an ISS greater than 15 (Table 1). In this cohort, the undertriage rate was 21.6%; the overtriage rate, 30.6%.

Table 1. Baseline Characteristics of the Central Netherlands Region and the Brabant Region.

Characteristic	Region
	Central Netherlands (n = 4950)a	Brabant (n = 6859)b
Demographics
Age, mean (SD), y	47 (21)	51 (22)
Male, No. (%)	2887 (58.3)	3583 (52.2)
Pregnancy, No. (%)	32 (0.6)	25 (0.4)
Use of oral anticoagulants, No. (%)	131 (2.6)	234 (3.4)
Alcohol use, No. (%)	531 (10.7)	746 (10.9)
Other drug use, No. (%)	43 (0.9)	39 (0.6)
Physiologic characteristics, mean (SD)
Systolic blood pressure, mm Hg	139 (24)	140 (24)
Respiratory rate, breaths/min	16 (4)	16 (5)
Oxygen saturation level, %	96 (4)	97 (3)
GCS scorec	14 (2)	15 (2)
Revised Trauma Scored	12 (1)	NA
ABC unstablee	117 (3)	129 (2)
Mechanism of injury, No. (%)
Mechanism criteriaf	819 (16.5)	475 (6.9)
Fall 2-5 m	314 (6.3)	197 (2.9)
Fall >5 m or >3 × body length	77 (1.6)	24 (0.3)
Fall from stairs, 1-10 steps	388 (7.8)	288 (4.2)
Fall from stairs, >10 steps	86 (1.7)	87 (1.3)
Vehicle rollover	96 (1.9)	129 (1.9)
Injury characteristics, No. (%)
Penetrating injury to head, thorax, or abdomen	90 (1.8)	30 (0.4)
Expected (unstable) pelvic fracture	26 (0.5)	11 (0.2)
Neurologic deficit (>1 extremity)	75 (1.5)	60 (0.9)
Symptoms of cerebral contusion or concussion	348 (7.0)	516 (7.5)
Agitation	172 (3.5)	66 (1.0)
Expected injury in AIS region
Head and neck or trauma to head	2635 (53.2)	2393 (34.9)
Face	955 (19.3)	977 (14.2)
Thorax	719 (14.5)	329 (4.8)
Abdomen	332 (6.7)	74 (1.1)
Extremities	2013 (40.7)	1501 (21.9)
Skin and others	85 (1.7)	85 (1.2)
Expected injury in ≥2 AIS regions	1230 (24.8)	309 (4.5)
Burning wound with or without inhalation trauma	77 (1.6)	80 (1.2)
Inhalation trauma	29 (0.6)	38 (0.6)
Clinical characteristics
ISS, mean (SD)g	5 (7)	NA
ISS >15, No. (%)	435 (8.8)	165 (2.4)
Destination, No. (%)
Level I trauma center	1724 (34.8)	1882 (27.4)
Level II trauma center	1326 (26.8)	4208 (61.4)
Level III trauma center	1900 (38.4)	769 (11.2)
Admission to hospital, No. (%)	2047 (41.4)	1842 (26.9)
In-hospital death, No. (%)	61 (1.2)	57 (0.8)

Abbreviations: AIS, Abbreviated Injury score; GCS, Glasgow Coma Scale; ISS, Injury Severity score; NA, not applicable.

^aRespiratory rate was missed in 324 participants (6.5%), systolic blood pressure in 345 (7.0%), oxygen saturation level in 662 (13.4%), and GCS score in 230 (4.6%).

^bSex was missed in 857 participants (12.5%), respiratory rate in 1973 (28.8%), systolic blood pressure in 1145 (16.7%), oxygen saturation in 1431 (20.9%), GCS score in 288 (4.2%), and ISS in 5017 (73.1%) (among the patients discharged from the emergency department).

^cScores range from 3 to 15, with higher scores indicating higher level of consciousness.

^dScores range from 0 to 12, with higher scores indicating better vital signs.

^eDefined as systolic blood pressure less than 90 mm Hg and/or respiratory frequency greater than 29 breaths/min.

^fMechanism criteria include fall of greater than 2 m, motor vehicle crash at greater than 32 km/h, or any type of entrapment.

^gScores range from 0 to 75, with higher scores indicating more severe injuries.

Validation Data Set

In the Brabant region, a total of 6859 adult trauma patients were included in the validation data set. To account for missing data, multiple imputation was used for respiratory rate in 1973 patients (28.8%), systolic blood pressure in 1145 (16.7%), oxygen saturation in 1431 (20.9%), and GCS score in 288 (4.2%). Mean patient age was 51 (22) years; 3583 (52.2%) were male and 3276 (47.8%) were female; and 165 (2.4%) had an ISS greater than 15 (Table 1). The ISS was only available for the admitted patients. In this cohort, the undertriage rate was 27.3%; the overtriage rate, 26.3%.

Model Development and Specification

To develop the prediction model, 43 potential prehospital predictors from the Central Netherlands database were explored using bivariable analysis (Table 2). The following 8 predictors were chosen for the final model, based on clinical reasoning: age; systolic blood pressure; GCS score; mechanism criteria; penetrating injury to the head, thorax, or abdomen; signs and/or symptoms of head or neck injury; expected injury in the AIS thorax region; and expected injury in 2 or more AIS regions. The optimal cutoff point with a minimum specificity of 50.0% (95% CI, 49.3%-50.7%) led to a sensitivity of 88.8% (95% CI, 87.5%-90.0%).

Table 2. Bivariable Logistic Regression Analysis on the Central Netherlands Region^a.

Variable	β Coefficient (SD)	P Value	OR (95% CI)
Patient characteristics
Age, y	0.010 (0.002)	<.001	1.010 (0.005 to 0.014)
Female	−0.447 (0.107)	<.001	0.640 (−0.656 to −0.237)
Alcohol use	0.205 (0.152)	.18	1.228 (−0.093 to 0.504)
Use of oral anticoagulants	0.047 (0.307)	.88	1.048 (−0.555 to 0.649)
Physiologic characteristics
Systolic blood pressure	0.005 (0.002)	.02	1.005 (1.001 to 1.010)
Systolic blood pressure <90 mm Hg	0.818 (0.320)	.01	2.265 (1.209 to 4.244)
Pulse	0.009 (0.003)	.001	1.009 (1.004 to 1.014)
Respiratory rate	0.042 (0.011)	.001	1.043 (0.020 to 0.066)
Respiratory rate <10 or >29 breaths/min	1.477 (0.238)	<.001	4.381 (2.749 to 6.981)
Oxygen saturation level	−0.096 (0.009)	<.001	0.908 (0.893 to 0.924)
GCS scoreb	−0.357 (0.019)	<.001	0.700 (0.674 to 0.727)
Revised Trauma Scorec	−0.846 (0.059)	<.001	0.429 (0.383 to 0.481)
ABC unstabled	2.209 (0.300)	<.001	9.110 (1.620 to 2.798)
Mechanism of injury
Mechanism criteriae	1.272 (0.108)	<.001	3.566 (1.061 to 1.482)
Fall 2-5 m	0.628 (0.168)	.002	1.874 (0.298 to 0.958)
Fall >5 m or >3 times body length	1.777 (0.244)	<.001	5.910 (1.298 to 2.256)
Motor vehicle crash >65 km/h	−0.243 (0.200)	.22	0.784 (−0.634 to −0.148)
Motorcycle crash >32 km/h	1.011 (0.150)	<.001	2.749 (0.716 to 1.306)
Vehicle deformity >50 cm	0.622 (0.488)	.20	1.863 (−0.335 to 1.579)
Vehicle intrusion passenger compartment >30 cm	1.997 (0.495)	<.001	7.368 (1.026 to 2.968)
Vehicle rollover	0.073 (0.354)	.84	1.075 (−0.621 to 0.766)
Motor vehicle vs pedestrian impact >10 km/h	0.599 (0.294)	.04	1.820 (0.023 to 1.175)
Motor vehicle vs bicycle impact >10 km/h	0.382 (0.185)	.04	1.465 (0.018 to 0.745)
No helmet on motorcycle or horse	1.340 (0.243)	<.001	3.819 (0.864 to 1.816)
No seatbelt in vehicle in high-energy trauma	−0.247 (0.521)	.64	0.781 (−1.268 to 0.775)
Deployed airbag in motor vehicle crash	−0.559 (0.314)	.08	0.572 (−1.175 to 0.056)
Entrapment in vehicle	1.328 (0.272)	<.001	3.773 (0.795 to 1.860)
Entrapment elsewhere	1.292 (0.370)	<.001	3.640 (0.566 to 2.018)
Trauma to the head	1.206 (0.109)	<.001	3.340 (0.993 to 1.419)
Suicide attempt	0.890 (0.324)	.006	2.435 (0.254 to 1.526)
Injury characteristics
Penetrating injury to head, thorax, or abdomen	1.248 (0.251)	<.001	3.484 (0.757 to 1.739)
Expected (unstable) pelvic fracture	2.683 (0.400)	<.001	14.623 (1.898 to 3.467)
Neurologic deficit (>1 extremity)	0.590 (0.330)	.07	1.804 (−0.057 to 1.238)
Pupil difference	2.569 (0.300)	<.001	13.049 (1.980 to 3.158)
Symptoms of cerebral contusion or concussion	0.897 (0.150)	<.001	2.451 (1.826 to 3.289)
Agitation	1.782 (0.170)	<.001	5.939 (1.448 to 2.115)
Vomiting	0.920 (0.274)	.001	2.510 (0.382 to 1.458)
Signs and/or symptoms of head or neck injury	1.157 (0.117)	<.001	3.182 (0.927 to 1.388)
Expected injury in AIS region
Face	0.412 (0.116)	<.001	1.510 (0.185 to 0.640)
Thorax	0.445 (0.127)	<.001	1.561 (0.197 to 0.694)
Abdomen	0.252 (0.184)	.17	1.286 (−0.109 to 0.613)
Extremities	−0.190 (0.104)	.07	0.827 (−0.394 to 0.014)
Expected injury in ≥2 AIS regions	1.110 (0.102)	<.001	3.035 (0.909 to 1.311)
Expected injury to spine	−0.344 (0.131)	.009	0.709 (−0.601 to −0.087)

Abbreviations: AIS, Abbreviated Injury scale; GCS, Glasgow Coma Scale; OR, odds ratio.

^aIncludes 4950 patients in the design data cohort. Multiple imputation was used to account for the missing prehospital variables, including respiratory rate in 324 participants (6.5%), systolic blood pressure in 345 (7.0%), oxygen saturation level in 662 (13.4%), and GCS score in 230 (4.6%).

^bScores range from 3 to 15, with higher scores indicating higher level of consciousness.

^cScores range from 0 to 12, with higher scores indicating better vital signs.

^dDefined as systolic blood pressure less than 90 mm Hg and/or respiratory frequency greater than 29 breaths/min.

^eInclude fall of greater than 2 m, motor vehicle crash at a rate of greater than 32 km/h, or any type of entrapment.

Model Performance

This prediction model resulted in an undertriage rate of 11.2% (Δ difference, 10.4%) and an overtriage rate of 50.0% (Δ difference, 19.4%) for the Central Netherlands region. Robust estimation using penalized maximum likelihood showed that all variables in the model were significant independent predictors (Table 3). The model had a good discrimination with a C statistic of 0.823 (95% CI, 0.813-0.832). The recalibration method led to an intercept of 0.894 for the Brabant region. External validation using the Brabant region database showed that the model with the new intercept was well calibrated (Figure 1) and had a good discrimination with a C statistic of 0.831 (95% CI, 0.814-0.848).

Table 3. Multivariate Analysis of the Predictors for ISS>15 in the Central Netherlands Region^a.

Variables	β Coefficient (SD)	P Value	OR (95% CI)
Patient characteristics
Age, y
Spline basis function 1	0.011 (0.004)	.001	3.24 (−0.003 to 0.026)
Spline basis function 2	0.001 (0.005)	.86	0.18 (−0.018 to 0.018)
Physiologic characteristics
Systolic blood pressure
Spline basis function 1	−0.011 (0.002)	<.001	−4.97 (−0.018 to 0.003)
Spline basis function 2	0.020 (0.003)	<.001	7.14 (0.007 to 0.031)
GCS score	−0.337 (0.001)	<.001	−36.69 (−0.391 to −0.291)
Mechanism of injury
Mechanism criteriab	1.314 (0.056)	<.001	23.40 (1.060 to 1.547)
Injury characteristics
Penetrating injury to head, thorax, or abdomen	1.196 (0.131)	<.001	9.13 (0.555 to 1.755)
Signs and/or symptoms of head or neck injury	0.571 (0.056)	<.001	10.23 (1.318 to 0.829)
Expected injury in AIS region of thorax	0.405 (0.071)	<.001	5.72 (0.079 to 0.742)
Expected injury in ≥2 AIS regions	0.713 (0.129)	<.001	12.79 (0.440 to 0.988)
Interceptc	2.069 (0.315)	<.001	6.57 (0.541 to 3.324)

Abbreviations: AIS, Abbreviated Injury Scale; GCS, Glasgow Coma Scale; ISS, Injury Severity Score.

^aIncludes 4950 patients in the design data cohort. Multiple imputation was used to account for the missing prehospital variables, including oxygen saturation level in 662 (13.4%) and GCS in 230 (4.6%).

^bInclude fall of greater than 2 m, motorcycle crash of greater than 32 km/h, or entrapment of a person or body part.

^cIntercept is 0.894 for the Brabant region.

Figure 1. Calibration Plot of External Validation.

The validation data were acquired from the Brabant region of the Netherlands. Error bars indicate 95% CIs.

Discussion

In this prospective, multicenter cohort study, we present a ready-to-use prehospital trauma triage prediction model for the presence of severe injury on the scene in patients with trauma. The model performed well in the derivation set and in external validation. To our knowledge, this is the first externally validated protocol showing acceptable triage rates, with a potential undertriage rate of 11.2% and overtriage rate of 50.0% depending on the chosen threshold.

Worldwide, triage protocols are based on a simple flowchart that includes vital signs, injury type, and mechanism of injury criteria.^{6,25,26,27,28} These triage protocols are simplistic and static: transport to a higher-level trauma center should be considered if just 1 criterion is present. In reality, some factors have a greater association with injury severity than others, and the combination of factors indicates the need for higher-level trauma care. In addition, current protocols often use cutoff points for continuous variables, whereas the prediction model uses coefficients for each predictor to represent each variable’s distinct association with injury severity to increase predictive ability.

The prediction model was based on the following 3 key elements: (1) inclusion of all adult patients with trauma transported by an ambulance to (2) all trauma centers of an entire geographic region, with (3) prehospital variables measured on the scene by EMS professionals. Previous studies have attempted to develop protocols but have not included these 3 key elements. For example, Dihn and colleagues²⁷ developed a triage protocol based on patients taken to a higher-level trauma center only, thereby excluding the undertriaged patients. Others included admitted patients only,^29,30 thus excluding patients discharged from the ED or the potentially overtriaged patients. These models would not be reliable in a general trauma population because they fail to include the patient populations in which improvement is of utmost importance: the undertriaged and overtriaged patients.

Eight predictors were included in the model based on clinical reasoning to achieve the best accuracy while keeping it user friendly without too many factors. Age was included because previous studies showed a higher undertriage rate in elderly patients.^6,31,32,33 Two continuous predictors of the condition of a patient are systolic blood pressure and GCS score.^34,35,36 Penetrating injury is an obvious predictor associated with potential severe trauma. The brain and thorax are 2 of the most commonly injured body regions, with both associated with a high prevalence of severe injury.^37,38,39 Also, multiregional injury was previously found as a strong predictor associated with severe injury.²⁷ Therefore, these 8 predictors were included in the current model. The prediction model resulted in a undertriage rate of 11.2% and an overtriage rate of 50.0% in the Central Netherlands region.

After penalized estimation, the updated diagnostic prediction model was externally validated with data from the Brabant region. The prediction model requires an update primarily owing to the difference in prevalence of severe injury, resulting in a difference in baseline risk.⁴⁰ To account for this difference, the constant value (intercept) in the equation was altered. The constant value can be altered for other regions before applying the prediction model based on prehospital and hospital data of the specific region. External validation—with the altered intercept—showed good discrimination and calibration, indicating that the prediction model would likely be accurate in a region that is heterogeneous with respect to population, prevalence of severe injury, and mechanism of injury. In addition, the injuries were coded differently in both regions: AIS98 was used in the Central Netherlands and AIS08 in the Brabant regions. When using the AIS08, the overall ISS is lower compared with injuries scored with the AIS98.⁴¹ External validation using the AIS08 showed that the prediction model functions well using the most recent AIS.

The prediction model did not achieve the goal of an undertriage rate of less than 5% as targeted by the American College of Surgeons Committee on Trauma.⁴² However, the model is a significant improvement compared with the existing protocols.^5,6,16,43 Whether further improvement of undertriage is achievable by solely improving the protocol is unclear. Previous studies have shown that addition of the judgment of EMS professionals can be useful in the identification of severely injured patients.^44,45,46,47 In this study, EMS professional judgment could not be quantified because it was not recorded. Including EMS professional judgment may improve the undertriage rate even more as well as increase adherence to the protocol. Other factors could have influenced transport decisions, such as geographical distance to a higher-level trauma center. Although distances are relatively small in the Netherlands compared with other countries, distance could have influenced the destination decision, especially in Brabant, because distances are larger in this region. Unfortunately, the effect of distance on the triage quality could not be evaluated in this study, and the distance at which deviation to the nearest hospital is best remains unclear.⁴⁸

In practice, it is not feasible to calculate the risk of severe injury based on an equation that must be memorized and applied on the scene. This problem could be solved by implementing the prediction model in a mobile app; such triage tools are increasingly being developed and used in the prehospital process.^8,9 This mobile app includes every variable, calculates the chance of severe injury, and gives advice on where to transport the patient, which is much more practical for EMS professionals compared with an equation. With a mobile device available on every ambulance, the EMS professionals can calculate the risk of severe injury using the prediction model in the app to decide quickly and more accurately where to transport the patient. The EMS professional judgment could be included in the app. A mobile app with the described prediction model is currently being implemented in different regions in the Netherlands. The implementation aims at reducing undertriage specifically.

Limitations

This study has several limitations. In the final prediction model, missing data were present for the systolic blood pressure and GCS variables. The data were considered to be missing at random, and multiple imputation was used to minimize selection bias. Second, for Brabant, the ISS was only available for the patients who were admitted or who died in the ED. In the Central Netherlands region—where the ISS was available for all patients—all severely injured patients (ISS >15) were admitted or died in the ED. Accordingly, an ISS less than 15 was assumed for patients discharged from the ED. Last, debate remains on the most accurate definition of a severely injured patient. Legitimate classification is difficult and dependent on multiple factors, such as regional circumstances and trauma center level. An ISS greater than 15 might not represent all patients in need of higher-level trauma center resources. However, an ISS greater than 15 is the most used surrogate marker for a severely injured patient when evaluating prehospital trauma triage; therefore, we chose this variable to define a severely injured patient.⁵

Future research should focus on the validation of the prediction model in other regions. Differences in incidence of severely injured patients, and consequently baseline risk, can be large, and therefore different baseline risks should be determined in other populations. A possible solution is to validate the prediction model in other regions using our methods.

The mobile app has been developed and is currently being implemented in the Netherlands (Figure 2). In this mobile app, the equation is integrated in addition to EMS professional judgment. This combination could be optimal to improve triage rates. In addition, the combination could give insight in the value of EMS professional judgment.

Figure 2. Screen Shots of the Mobile App.

EMS indicates emergency medical services; TT app, Trauma Triage App (Synappz).

Conclusions

To our knowledge, this study is the first to develop and validate a prehospital trauma triage protocol based on all adult patients with trauma to be transported to a trauma center within a region and that can lower the undertriage rate to 11.2%, with an overtriage rate of 50.0%, from 21.6% and 30.6%, respectively. This protocol, based on an equation in which each predictor has its own coefficient, can be implemented with a mobile app for EMS professionals and could be of great help to lower undertriage rates.

Supplement.

eMethods. The National Field Triage Protocol of the Netherlands