Major Trauma and the Injury Severity Score - Where Should We Set the Bar?

Abstract

Major trauma is commonly defined using an Injury Severity Score (ISS) threshold of 15. Since this threshold was formulated, there have been significant developments in both the Abbreviated Injury Scale underlying the ISS, and trauma management techniques, both in the preventive and acute-care phases of trauma management. This study assesses whether this ISS threshold is appropriate when evaluating both mortality, and hospital-based indicators of morbidity, in a paediatric population using a large hospital trauma registry. Other registries and datasets using ISS >15 as an inclusion criterion may exclude a substantial body of data relating to significantly morbid trauma patients.

Determining the extent of injury to a patient, either subjectively or objectively, is central to healthcare funding, patient triage (at pre-hospital, intra-hospital and post-discharge phases), and injury epidemiology and research. As trauma is a disease with a wide range of presentation (both in injury types, and the location and effect of individual wounds), useful injury severity measures must be able to meaningfully compare disparate types of trauma. The AAAM’s Abbreviated Injury Scale (AIS) [CMAAS, 1971; Gennarelli & Wodzin, 2005] has for many years provided such a measure, as well as serving as an anatomic injury platform for the development of other severity measures.

One such measure, the Injury Severity Score (ISS) [Baker, O’Neill, Haddon et al, 1974] has been arguably the most used injury severity measure since its initial development. Although the ISS has a number of recognised mathematical, administrative and clinical limitations [Aharonson-Daniel, Giveon, Stein et al, 2006; Copes, Champion, Sacco et al, 1988; Gabbe, Cameron, Wolfe et al, 2005; Kilgo, Meredith, Hensberry et al, 2004; Lavoie, Moore, LeSage et al, 2005, MacKenzie, Shapiro & Eastham, 1985; Russell, Halcomb, Caldwell et al, 2004; Rutledge, 1996; Streat & Civil, 1990], it’s prominence in trauma monitoring and evaluation has resulted in the ISS being regarded as the ‘gold standard’ in trauma severity grading [Lavoie et al, 2005; Rutledge, Hoyt, Eastman et al, 1997].

Use of the ISS to define ‘major’ or ‘severe’ trauma using an ISS threshold has occurred for decades. A substantial majority of registries and datasets [Cottington, Young, Shufflebarger et al, 1988; Dick & Baskett, 1999; Eichelberger, Gotschall, Sacco et al, 1989; Lossius, Langhelle, Søreide et al, 2001; MacLeod, Kobusingye, Frost et al, 2003; Zoltie & deDombal, 1993] adhere to a threshold for major trauma of ISS greater than 15. This threshold was first described by Boyd et al in 1987 as being predictive of 10% mortality, which (purportedly) "most physicians would agree... should be treated at a level one trauma center". However, this definition is both arbitrary (as it addresses only one dimension of injury severity, and that in a non-evidential manner) and changeable (as healthcare system improvements in industrialised nations have over time resulted in decreasing mortality rates from trauma). A significant number of datasets have used one of a wide range of ISS levels as analysis thresholds or inclusion criteria for severe injury, including ISS greater than 8 [Hannan, Waller, Farrell et al, 2004], 10 [Petri, Dyer & Lumpkin, 1995], 11 [Kaida, Petruk, Sevcik et al, 2004; Osmond, Brennan-Barnes & Shephard, 2002], 12 [CIHI, 2006], 16 [Tsai, Chan, Chang et al, 1993] and 20 [Cottington et al, 1988; Eichelberger et al, 1989; Potoka, Schall & Ford, 2001]. However, the choice of a specific threshold in most instances appears to be arbitrary, or at best unstated.

The discriminative ability of an ISS threshold in identifying severe injury, however, relies on a meaningful and reproducible definition of ‘severe’ - and this often remains elusive or unstated. Rating the severity of an injury (ie, the gravity of the isolated acute impact of the disease, discounting the effects of comorbidity, complications and management) depends on the exact dimension of the illness being considered, and may refer to a host of parameters including:

1. likelihood of death;

2. likely hospital and other resource requirements;

3. the cost and complexity of the resources and treatments required;

4. length of recovery;

5. likelihood and extent of permanent impairment;

6. energy dissipation or absorption;

7. incidence of a particular trauma type; and

8. impact on quality of life.

As the ISS is based solely on the AIS, the dimensions which are considered when determining individual injury severity scores for AIS codes will be reflected in the ISS. After being developed by evaluating mortality likelihood after injury [Baker et al, 1974], it was recognised from the earliest days of the ISS that it also correlated well with length of hospital stay, the need for major surgery, and the extent of permanent residual disability [Bull, 1975; Semmlow & Cone, 1976]. It has been noted by several investigators since that parameters such as the length of time spent in ICU or hospital, trauma type incidence, duration of disability, discharge destination and costs of hospital stay may also be considered direct or surrogate measures of injury severity [Bull, 1978; Dick et al, 1999; Gabbe et al, 2005].

However, establishing which dimensions are considered in individual codes, or the extent to which they have influenced the severity assigned, is not possible, as stated in the introduction to the current version of the AIS (2005): "The precise dimensions of severity have not been explicitly determined because these components change with time". While an attempt to keep an instrument contemporary is laudable, it nevertheless remains that the uses to which this instrument and its derivatives are put require ongoing review in order to ensure consistency of trauma evaluation over time. Changes in the AIS may have altered the predictive value of the ISS with regard to specific dimensions over time, and must be re-evaluated. For example, changes in standards of practice and care have altered some aspects of injury severity such as likelihood of survival for a given injury [Champion, 1991]; consequently, ISS thresholds used in their evaluation should also be periodically reviewed.

Establishing an ISS threshold for a registry or study dataset (often as an inclusion criterion for registry or study entry), or a public health monitoring tool, however, depends on the outcome(s) which will be of interest. As the vast majority of incident trauma does not result in death, thresholds identifying a specific level of mortality tend to be relatively high. Studies using receiver operating characteristic (ROC) curves to enable ISS-mediated prediction of mortality in children have arrived at threshold ISS scores of 22 [Bulut, Koksal, Korkmaz et al, 2006], 25 [Orliaguet, Meyer, Blanot et al, 1998] and 28 [Ducrocq, Meyer, Orliaguet et al, 2006]. However, Semmlow [1976] noted that "though differentiation between ISS values less than 20 does not lead to improved mortality prediction, it does provide information regarding amounts of service required by patients". ISS thresholds which are capable of measuring or predicting aspects of morbidity are likely to be significantly lower than those for mortality. This has become more important over time due to declining death rates from some types of trauma, with a resultant increase in demands on hospital and long-term rehabilitation services by these patients, and a shift in trauma service focus towards targeting reductions in morbidity as well as mortality.

In spite of the potential for ROC data to be used in the empirical determination of ISS thresholds, however, there has been little use of them in the literature. Most of the published literature examining the predictive value of ISS in mortality, hospital length of stay (LOS), hospital stay cost, and need for ICU admission has compared ISS performance against alternate predictors of injury severity, whether AIS-based [Balogh, Varga, Tomka et al, 2003; Brenneman, Boulanger, McLellan et al, 1998; Grisoni, Stallion, Nance et al, 2001; Honarmand & Safavi, 2006; Lavoie et al, 2005; Tay, Sloan, Zun et al, 2004], ICD-based [Rutledge, Osler, Emery et al, 1998], ICU-based [Castello, Cassano, Gregory et al, 1999; Vassar, Lewis, Chambers et al, 1999], or for pre-hospital triage [Lett, Hanley & Smith, 1995; Tsai et al, 1993], rather than establishing appropriate threshold level(s) for the dimensions of interest.

One study which has used ROC curves to evaluate the ability of the ISS to discriminate between patients (in likelihood of death, or the necessity of a significant operation or fluid resuscitation to maintain blood pressure) per se was that by Baxt and Upenieks (1990). However, there were a number of limitations with their study design. Firstly, they compromised the precision, and hence usefulness, of their ROC curves by using steps of ISS 2, rather than the complete range of ISS values; secondly, they arbitrarily chose ISS threshold values to evaluate against predictive ability based on their ISS ‘bins’, rather than relying on statistical evidence provided by the ROC curves they had generated; finally, they featured a pre-hoc division of patients into ‘major’ and ‘minor’ without specifying which criteria had been used to differentiate them.

The result of all of this previous work is that, in spite of a plethora of studies employing ROC data, and frequent use of ISS thresholds in injury data collection and analysis, there is still a conspicuous lack of empirical verification of these thresholds. The aim of the present study, therefore, was to determine the optimum ISS threshold (or thresholds) for the potential definition of severe injury in a trauma population, particularly where death may not be the only outcome of interest. Any determined threshold would need to use explicit reproducible methods in its derivation from the data provided by the ROC curve.

METHOD

The Royal Children’s Hospital in Melbourne, Australia is a 250-bed specialist paediatric hospital treating approximately 32,000 inpatients annually. The hospital covers a population of between 4.5 and 5 million people across Victoria and Tasmania, approximately 1 million of which are aged less than 15 years. Since 1999, the hospital has had a trauma registry containing records of all trauma admissions to the hospital, irrespective of severity or outcome. Trauma patients are identified concurrent with their admission, and are identified, entered and coded by a single full-time data manager. Data is held on many aspects of pre-hospital, emergency department and subsequent hospital management, including the calculation of ISS based on the AAAM’s 1998 revision of the AIS.

Patients aged less than 18 years at the time of injury, and with a meaningful ISS (ie, not solely containing AIS codes with injury severity 9) admitted to the RCH between 1 July 2000 and 30 June 2005 were extracted from the trauma registry and reviewed. ROC curves were generated against ISS for death, the need for ICU admission, and LOS greater than 7 days. In order to explicitly define thresholds using ROC curve data in the absence of a definitive guide to the interpretation of such data, two potential methods were identified and used:

1. The simultaneous maximum of sensitivity and specificity, functionally given by the threshold closest to the top left-hand corner of the ROC curve graph; and

2. The maximum possible value for the probability of correct prediction using the threshold, while keeping the sensitivity at or above 0.7, and the specificity larger than 0.8 [Tsai et al, 1993].

It was also felt that specific ISS groups may be of interest. Devised in part after the reasoning of Copes et al (1988), groups were set at ISS ranges of 12–14 (multiply-injured patients with at least two moderate or serious injuries; also patients narrowly excluded by the most common current ISS threshold of 15), 16–24 (patients with at least two serious or one severe injury; also patients narrowly included by the most common threshold), and greater than 24 (critically injured patients). As well as the three outcomes evaluated by ROC curves, the mechanism of injury (specifically, transport-related trauma), gender proportions and injury type proportions were also evaluated in these groups using chi-square testing.

One-way ANOVA testing with post-hoc Bonferroni multiple comparisons testing for pair-wise comparisons was also used to compare LOS and patient age in the three patient groups. Logarithmic transformation of LOS data was required for this test, due to a high degree of positive skew.

All data evaluations were performed using Microsoft Excel 2003 (Microsoft Corporation, Redmond, 2003) and Stata version 8.2 (StataCorp, College Station, 2004).

RESULTS

8,177 trauma patients with a valid ISS were returned from the registry. There were an additional 119 patients without useable injury coding. 566 patients had an ISS greater than 11 (ISS 12–14 - group 1, 105 patients; 16–24 - group 2, 275 patients and >24 - group 3, 186 patients). Together with the entire population sampled, patient group descriptions are given in Table 1.

Table 1.

Descriptive data for all patients, and ISS subgroups. Statistical tests performed across groups 1–3 are given for each variable with post-hoc significances (p<0.05) where relevant.

	1-ISS 12-14	2-ISS 16-24	3-ISS >24	All patients
Patients	105	275	186	8177
Age (yrs)
Average	10.47	8.93	8.81	7.64
Std deviation	4.23	5.02	4.86	4.70
Comparison (ANOVA)	p<0.0099; 1 vs 2 & 1 vs 3
Gender
Male	66	180	123	5202
Female	39	95	63	2975
M:F ratio	1.69:1	1.89:1	1.95:1	1.75:1
Comparison (Chi-square)	p<0.8468
Trauma type
Blunt	99	253	165	7069
Penetrating	4	5	10	567
Burn	2	17	11	541
Comparison (Chi-square)	p<0.1191
LOS (excl death)
Median	5.71	6.00	13.31	1.00
Lower quartile	3.84	3.47	5.91	0.63
Upper quartile	9.80	11.85	30.33	2.00
Comparison (ANOVA; lnLOS)	p<0.0001; 1 vs 3 & 2 vs 3
ICU admits (%)	29 (27.6)	100 (36.4)	127 (68.3)	393 (4.8)
Comparison (Chi-square)	p<0.0001; 1 vs 3 & 2 vs 3
Deaths (%)	0 (0.0)	4 (1.5)	28 (15.1)	32 (0.4)
Comparison (Chi-square)	p<0.0001; 1 vs 3 & 2 vs 3
Transport-related mechanism (%)	62 (59.0)	130 (47.3)	97 (52.2)	1866 (22.8)
Comparison (Chi-square)	p<0.1137

Patients in group 1 (ISS 12–14) were on average significantly older than groups 2 (ISS 16–24) and 3 (ISS >24). This is in keeping with group 1 having the highest proportion of transport-related trauma (as this tends to increase with age), although comparison of mechanism did not reach significance per se. The association between age and transport is further seen by the lower average age, and markedly lower transport-related incidence of the study patients in total, and is illustrated further in Figure 1, which provides more detailed breakdowns of injury mechanism.

Figure 1.

Breakdown of injury mechanism for groups 1–3. Transport-related mechanisms are shaded with lines; non-transport mechanisms with dots.

Group 3 differed significantly from groups 1 and 2 in the proportions of death and ICU admission within each group, and the average length of hospital stay for patients in the group. Perhaps more noteworthy, though, is the fact that there were no differences between groups 1 and 2 for these (or the remaining) variables.

ROC curve graphs, as generated by Stata, are shown for two of the outcomes evaluated in Figure 2. Optimum sensitivity and specificity values for the two threshold determination methods described - simultaneous maximum (ie, of sensitivity and specificity), and maximum proportion correct given good sensitivity and specificity - as well as the ISS thresholds returned, are shown in Table 2. All areas lying under the curves were high, particularly death, although this is in part due to the large number of surviving patients with low ISS. To identify patients with an elevated risk of dying, ISS thresholds needed to be considerably higher than was shown to discriminate between patients who are likely to require long periods in hospital or ICU admission - for death, >20 or >25 (depending on the method used), and for LOS and ICU requirement, between >7 and >9.

Figure 2.

ROC curve graphs showing sensitivity and specificity for differing ISS thresholds in evaluating a) death, and b) (surviving) patients with LOS >7 days. Calculated optimum thresholds using the two methods employed, and the common ISS >15 threshold are shown.

Table 2.

Sensitivity, specificity and proportion of patients correctly classified at the optimum ISS thresholds given by the two methods used in evaluating ROC curves, and areas under each curve (with 95% confidence intervals).

	Sensit’y	Specif’y	% correct	ISS threshold
Death
Simultaneous maximum	96.88%	97.54%	97.54%	>20
% correct maximum	75.00%	98.64%	98.54%	>25
Area under curve	0.9914
95% CI	0.9880 – 0.9949
Need for ICU
Simultaneous maximum	76.08%	91.52%	90.78%	>9
% correct maximum	76.08%	91.52%	90.78%	>9
Area under curve	0.9007
95% CI	0.8831 – 0.9183
LOS >7 days (excl death)
Simultaneous maximum	79.45%	73.67%	74.11%	>7
% correct maximum	79.12%	73.91%	74.30%	>8
Area under curve	0.8332
95% CI	0.8144 – 0.8519

DISCUSSION

Given the lack of an authoritative definition of the ‘best’ method for generating thresholds using ROC data, the results of the current study indicate that patients with an ISS of 8 or higher could be considered for inclusion as ‘severely injured’ patients if morbidity as well as mortality is to be assessed in the trauma patient group. In practical terms, this is a relatively simple threshold to implement, as it includes all patients with any injury of AIS severity greater than 2, or any patient with injuries of AIS severity 2 in more than one body region. It is acknowledged, though, that a number of unrelated clinical, administrative and logistical factors may affect proxy variables for morbidity. For example, ICU admission likelihood may vary due to bed demand, or differences in ICU admission criteria between hospitals. Factors influencing length of stay include the employment of resources such as clinical pathways, and the demographics and severity of patients treated. In this context, the evaluation and implementation of a more standardised measure such as the anticipated AIS Predicted Functional Capacity Index [Gennarelli & Wodzin, 2005] may improve the generalisability of an ISS threshold for morbidity.

The indicated thresholds for determining increased threat-to-life are comparable to those noted in the previous studies examining mortality in paediatric trauma [Bulut et al, 2006; Ducrocq et al, 2006; Orliaguet et al, 1998]. Based on this sample, a more conservative ISS threshold of 21 or higher could be employed. Although all of these studies have evaluated children only, it does not seem unreasonable to suggest that a mortality-based threshold in adults may well be higher than the current ‘standard’ threshold of ISS >15. Recent data from the National Trauma Data Bank in the United States showed a mortality rate of 5.8% for all trauma with ISS between 16 and 24 [Clark & Fantus, 2006], compared with the major trauma threshold definition of 10% asserted two decades ago by Boyd et al [1987]. Assuming such an indicator is still felt to be valid, the obvious implication is that trauma data should be reviewed over a broad population which includes all age ranges, and a diverse spread of hospital types and management strategies. While it is not unlikely that the specific results obtained in the paediatric data presented here would differ to some extent from the general trauma population, it must be stressed that any potential differences are currently untested.

The principal differences seen in the current study between patients with ISS 12–14 (currently excluded from the ‘standard’ ISS threshold) and 16–24 (currently included) appear to relate to epidemiology, rather than trauma management or outcome. From visual inspection of the charts in Figure 1, it seems evident that the injury pattern present in group 1 has more in common with group 3 than group 2. This is not surprising when the incidence of multitrauma in each patient group is considered. Disregarding injuries with an AIS severity of 1, it can be seen mathematically that 100% of patients with an AIS of 12–14 will be multiply-injured; in the current samples; only 31% of patients with an ISS of between 16 and 24 had injuries to multiple regions (as most had a single AIS level 4 injury and an ISS of 16, 17 or 18), compared with 49% of patients with an ISS >24. These differences are associated with the proportion of patients involved in transport-related trauma, as this mechanism (particularly relating to motorised transport) is more likely than other mechanisms such as falls, burns, or non-transport-related collisions (such as sports-related injuries) to cause injury to multiple body systems. The existence of this relationship, at the same time as a stagnation of other examined injury severity dimensions between ISS of (approximately) 10 and 20 is to some extent indicative of the limitations of the ISS in examining specific injury severity dimensions, but also of the ability of the ISS to reflect dimensions other than simply mortality or even morbidity. It also implies that variations in patient demographics and mechanism may hold implications for morbidity and mortality rates within a patient population, which cannot be controlled for in the current (single-hospital, single-country, paediatric) sample.

The power of the ROC curves generated (particularly for death) is likely to have been affected by the large number of patients in the cohort with low ISS creating a ‘skew’ in the dataset, as comparatively very few of this large number met the criteria being evaluated [MacLeod et al, 2003]. However, it should be remembered that unlike previous studies using ROC curves, no attempt was made to discriminate between the ISS and other instruments - only to determine optimum thresholds for one instrument. This should not have been affected by the cohort composition, and indeed could arguably have been biased by the exclusion of part of the injury severity continuum, as it was not known pre-hoc where the thresholds would lie.

Although the other instruments against which the ISS has been evaluated using ROC curves have often outperformed it in the prediction of mortality in particular, it should be remembered that many of these measures were devised with a specific view to maximising mortality prediction, rather than attempting to classify global ‘severity’, as with the AIS (and hence ISS). Moreover, as the ISS is currently used far more than any other severity score (and after 30 years, this is unlikely to change in the foreseeable future), it seems more prudent to maximise the potential of an instrument which is performs at least moderately well - and is actually being used - than to urge the widespread employment of a new instrument offering marginal improvement in performance.

Although the AIS has always been intended to reflect factors other than simply threat-to-life, the AIS, and related scores such as the ISS, are frequently only used to assess mortality likelihood as an outcome, despite a large morbidity burden imposed by trauma in the population. Also, despite considerable change in the AIS system underlying the ISS since it was first developed, there has been little to no research in the past 20–30 years examining appropriate ISS thresholds for major trauma assignation, despite changes in the management of trauma, and resultant changes in associated morbidity, mortality and cost.

There are a number of limitations inherent in the design of the current study. Firstly, while the proxy variables used to evaluate morbidity (ICU requirement and LOS) are relevant to acute-care hospitals, assessment of ongoing impairment, total length of recovery or quality of life would have been useful adjuncts in non-fatal trauma patients. Also, although the number of patients examined was comparatively large for a single-hospital study, the results will be more reflective of institutions which have similar patient demographic profiles, resource demands and patient flow within and through the hospital to the hospital where the study was performed. The paediatric data used in the current study is perhaps the most obvious limitation, but it is also implied, for example, that differences may exist in other countries where injury cause patterns (such as the incidence of penetrating trauma, which is rare in Australia) are dissimilar.

In conclusion, it is felt that registries, studies and public health or government agencies using ISS >15 as a threshold criterion may exclude a substantial body of data (and hence ‘usefulness’ in analysis or decision-making) relating to significantly morbid patients. In order to meaningfully evaluate morbidity, the present study indicates that many patients with an ISS below 15 have similar outcomes to at least those with an ISS between 16 and 24, although presenting with a different epidemiological injury profile. Conversely, if mortality remains the only outcome of interest, a higher threshold may (and probably should) be adopted. It is hoped that the current study will prompt the re-assessment of currently used ISS thresholds, both now and periodically in the future, to confirm that they are returning appropriately severely injured patients for the outcomes being assessed. Although this should ideally be undertaken in the largest and broadest datasets possible (to ensure comparability of major trauma data between different populations and sub-groups), re-evaluation could also occur more locally within existing trauma registries, whether paediatric or population-based.