Severity of injury measures and descriptive epidemiology

Short abstract

The ICISS method for deriving severity of injury is available to all and, until shown otherwise, it is the threat‐to‐life severity measure of choice for the ICD‐10 era

In those countries with national hospital inpatient data systems, we want to use these admissions/discharges/separations (henceforth, referred to as admissions) data for a variety of purposes. At the population level, these include for describing the epidemiology of injury, developing injury indicators, and injury surveillance. In this editorial, I will concern myself solely with the use of these data for descriptive epidemiological purposes.

We know that health service use, following injury, including admission to hospital, is influenced by many factors that are independent of the severity of the injury, including bed availability, access (for example, distance from home to hospital/rurality), concern about intentionality (for example child abuse), and professional variations in practice.¹ So, in a descriptive epidemiological analysis, using admission to hospital as the definition of a case of injury is likely to give a biased picture of the variations in injury incidence by person, place, or time, as well as by external cause.

Defining cases of injury according to whether the injury exceeds a given severity threshold, with the threshold chosen to ensure nearly complete ascertainment from the data source, is one way to overcome these problems.² This begs the question: how should severity be measured when using admissions data?

How should severity be measured?

Severity can be measured on a number of dimensions. Historically the focus in the literature has been on measures of severity in terms of damage to the body that have been validated against mortality outcomes. These are referred to as threat‐to‐life severity measures. Little attention has been given to another equally important dimension, namely threat‐of‐disability. This dimension is also important, and threat‐to‐life measures are poor predictors of disability. Nevertheless, I focus here on threat‐to‐life scales, since they are used more commonly for the analysis of admissions data.

When dealing with national data, direct coding of the severity of injury for each admission, using any of the severity scoring methods, would be very time consuming, require highly skilled staff, and would be very expensive. So much so that it would be out of reach for (almost) all countries. This problem has been bypassed by using indirect methods of deriving severity of injury from diagnosis codes captured in admissions data. These methods are described below.

Severity in the ICD‐9 era

The convention for many national health administrative databases has been for diagnosis of injury to be classified using the WHO ICD‐9, or its modifications ICD‐9‐CM and ICD‐9‐CM‐A.³,⁴,⁵ Software was developed, ICDMAP,⁶,⁷ to derive the Abbreviated Injury Scale (AIS) severity score for the majority of diagnoses within ICD‐9‐CM. AIS, the Injury Severity Score (ISS) and its variations (for example, NISS) are the most widely used scales for measuring severity when using these national data sources.⁸ ISS/NISS have been validated and been shown to be reasonable predictors of the probability of death.

It should be noted that ICDMAP is not without its problems. For example, for intracranial injury, the specificity of ICD‐9‐CM(‐A) is less than AIS and so, particular ICD‐9‐CM(‐A) diagnosis codes each map to several AIS codes, and hence to several severity scores. The software permits the use of translation rules to choose just one severity of injury score for each injury. However, this inevitably lead(s) to some misallocation of severity scores, which is more problematic for some diagnoses and body regions than others. Despite this problem, ICDMAP has been chosen by many to assign severity of injury to large administrative (for example, admissions) data sets.

Severity in the ICD‐10 era

For those countries that have replaced ICD‐9 for coding admissions data with ICD‐10,⁹ or its modifications ICD‐10‐CM and ICD‐10‐AM,¹⁰ the opportunity to derive AIS scores using ICDMAP no longer exists. (I believe that, at this time, the original developers of ICDMAP have no intention of developing an equivalent mapping programme for ICD‐10.) All is not lost, however. In 1996, Osler and colleagues introduced us to the ICD‐9 based Injury Severity Score (ICISS),¹¹ which provides a direct measure of threat‐to‐life.

When using data for which diagnosis is coded to ICD‐9‐CM(‐A), ICISS scores are derived as follows. Using a training set of data (for example, data relating to 1995/96, say), Survival Risk Ratios (SRRs) are derived for each of the injury diagnosis codes—that is, diagnosis specific estimated survival rates for each injury diagnosis. These SRRs can then be applied to any set of data of interest (for example, data relating to 1995–99). For people with single injuries, the ICISS score is identical to the SRR for that injury. Where a person experiences multiple injuries, the SRRs for each of the injuries are multiplied together to give an ICISS score, assuming independent effects of each injury on the likelihood of death.

ICISS is empirically derived, unlike AIS. It has been shown to have good properties relative to AIS/ISS.¹¹,¹²,¹³,¹⁴,¹⁵ Since the introduction of ICD‐10 for coding diagnosis of admissions, the ICISS methods have been developed, and severity scores derived, for ICD‐10, ICD‐10‐CM, and ICD‐10‐AM coded data.¹⁶,¹⁷ So, methods are still available for inferring severity of injury from morbidity data in the ICD‐10 era.

It should be noted, however, that the ICISS approach stands or falls on the accuracy and precision of the estimates of the SRRs. In this issue Clarke and Ahmad state: “A survival probability calculated from an individual diagnosis code may … give a false sense of accuracy if it is based on a small number of cases…” [see page XXX].¹⁸ So, even when large national administrative data sets of admissions are used as the training set from which SRRs are derived, the precision of the SRRs will be a problem for less frequently occurring diagnoses. This weakens the ICISS approach. Additionally, within the New Zealand context, we have identified a number of other difficulties with the method, including: (a) the problems of estimating SRRs using all deaths (not just the minority that occur in hospital); and (b) the method requires the attribution of the cause of death to each injury (not only to serious injury, but also to accompanying minor and superficial injury) that a person sustains, when they experience multiple injuries and subsequently die. Because of these problems, it is worth looking at other approaches to solving this problem—for example, the Barell matrix approach, discussed below.

The Barell matrix and injury severity

The paper by Clark and Ahmad,¹⁸ in this issue, relates to the ICD‐9 era. They derive severity scores based on (average) AIS and SRRs for each cell of the Barell matrix—a matrix used for summarising injury diagnosis data. The Barell matrix has two dimensions: types of injury (n = 12), and body site (n = 35). Each ICD‐9‐CM code has been allocated to a single cell of the matrix in the work of Barell and her colleagues.¹⁹

The ICDMAP, ICISS, and the Barell matrix approaches all seek to allocate severity to a diagnosis code or category. The Barell matrix approach differs from the other two in that the number of diagnosis categories to which a severity score is assigned is far fewer in number—the Barell matrix includes 420 cells, many of which are empty, far fewer than the number of ICD‐9‐CM codes. This permits more precise cell based estimates of AIS (referred to as bAIS in their paper) and SRR (bPS_cell in their paper). On the other hand, the diagnostic categories included in the Barell matrix are far more heterogeneous than those of ICD‐9‐CM. For example, cell A31 of the Barell matrix is leg/ankle fracture. This fracture could be of the tibia, fibula, or femur, it could be simple or compound, open or closed—each of these combination do not have identical severity of injury. Intuitively, the more homogeneous is the diagnosis category, the more likely it is that the severity score associated with it will accurately reflect the severity of all injured persons coded to that diagnosis. The example above suggests that there is more chance of misclassification of severity using the Barell matrix approach than the other approaches.

The advantage of the Clark and Ahmad approach is, however, that it generates AIS scores and SRRs for a relatively small number of diagnostic categories—that is, for each non‐redundant cell of the Barell matrix. This makes it easier to understand, and easier to work with. If the above theoretical concern turns out not to compromise the validity of the measure—for example, when tested for its concordance and calibration—then this is the “acid test” of whether it is any good. The results of the initial validation look reasonably promising.¹⁸

Many countries have moved, or are moving, to the use of ICD‐10 for coding diagnosis in their admissions data. A further challenge in regard to the Barell matrix based approach is to update this work for ICD‐10 coded data. As I have described, the advent of the ICD‐10 era has made mapping of ICD into AIS using ICDMAP unavailable. ICISS is an available solution in the ICD‐10 era. Perhaps the Barell matrix approach to inferring injury severity, once developed for use with ICD‐10 coded data, will provide the option to use AIS based measures as well.

The future

• The ICISS method is available to all. Until shown otherwise, it is the threat‐to‐life severity measure of choice for the ICD‐10 era. There are problems with this approach, some of which have been alluded to above. Another major shortcoming of this measure is the lack of invariance of the SRRs across country and over time. For example, I can guarantee that the SRRs derived in the USA, for example, will be different from those in Australia. Furthermore, work suggests that case fatality rates have been improving over time.²⁰ Consequently, in general, SRRs calculated using a training set of ICD‐9‐CM coded data from 1990/91 will be less than those from 2000/01. In order to use ICISS to derive severity of injury for people admitted to hospital, we have to work around these problems. Further research and development will provide the opportunity to improve the ICISS methods. Other approaches should also be considered, however.

• The Barell matrix based severity measure shows some promise. It may make severity scoring more accessible to a greater number of people due to the simplified methods used compared with ICISS. However, work is necessary to update the methods for the ICD‐10 era, and once updated to test the accuracy of the severity measures generated using an updated Barell matrix approach compared with ICD‐10‐based ICISS. Only if the Barell matrix approach performs as well (or almost so) as the ICISS approach can we be reassured that that approach is acceptable. I eagerly await this further work.

• Although there is a need to improve our threat‐to‐life severity measures for our inpatient data, there is a greater need to develop our ability to assign threat‐of‐disability severity scores. This is of even greater priority than refining our current imperfect threat‐to‐life severity measures.