Development and internal validation of China mortality prediction model in trauma based on ICD-10-CM lexicon: CMPMIT-ICD10

Abstract

Background:

Models to predict mortality in trauma play an important role in outcome prediction and severity adjustment, which informs trauma quality assessment and research. Hospitals in China typically use the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) to describe injury. However, there is no suitable prediction model for China. This study attempts to develop a new mortality prediction model based on the ICD-10-CM lexicon and a Chinese database.

Methods:

This retrospective study extracted the data of all trauma patients admitted to the Beijing Red Cross Emergency Center, from January 2012 to July 2018 (n = 40,205). We used relevant predictive variables to establish a prediction model following logistic regression analysis. The performance of the model was assessed based on discrimination and calibration. The bootstrapping method was used for internal validation and adjustment of model performance.

Results:

Sex, age, new region-severity codes, comorbidities, traumatic shock, and coma were finally included in the new model as key predictors of mortality. Among them, coma and traumatic shock had the highest scores in the model. The discrimination and calibration of this model were significant, and the internal validation performance was good. The values of the area under the curve and Brier score for the new model were 0.9640 and 0.0177, respectively; after adjustment of the bootstrapping method, they were 0.9630 and 0.0178, respectively.

Conclusions:

The new model (China Mortality Prediction Model in Trauma based on the ICD-10-CM lexicon) showed great discrimination and calibration, and performed well in internal validation; it should be further verified externally.

Introduction

Mortality prediction is one of the key targets of trauma-related research. An excellent prediction model not only accurately assesses injury severity in individual patients, but also assesses the quality of medical institutions. Mortality depends on multiple factors, including the patient’ s condition (age, sex, existing diseases), injuries (severity, number, and pattern), and the patients’ physiological reactions to these injuries (shock, unconsciousness, coagulopathy, etc.). Currently, there are mainly two types of trauma models to evaluate the severity of injuries. One is the Injury Severity Score (ISS) based on the Abbreviated Injury Scale (AIS), which is an anatomical scoring system based on expert consensus to classify and quantify injuries.^[1–3] The other is based on International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) codes, such as the Trauma Mortality prediction Model-ICD10 (TMPM-ICD10) and the ICD-based Injury Severity Score.^[4–6]

Although the ISS is widely used, the use of a complex AIS lexicon requires special training for the coders, which requires a large amount of time and resources.^[7] Most Chinese databases have no special AIS codes, and ICD-10 codes are widely used in Chinese hospitals. However, the TMPM-ICD10 is relatively complex and does not take into account the patient's existing diseases and acute physiological reactions. In addition, the ICD-10-CM codes used in China are different from the international standard ICD-10-CM codes. Therefore, we established a new prediction model based on the ICD-10 code: China Mortality Prediction Model in Trauma based on the ICD-10-CM lexicon (CMPMIT-ICD10).

Methods

Study database

The data of this retrospective study came from the discharge database of the Beijing Red Cross Emergency Center, which is the largest trauma emergency center in Beijing and is responsible for the treatment of emergency patients throughout Beijing. The data included the demographic details of patients, ICD-10-CM codes, state of consciousness, mechanism of injury, treatments, cost, and outcome at discharge.

Data collection and processing

From January 1, 2012 to July 1, 2018, all patients admitted to the hospital for traumatic events were included in this study. The traumatic events were those listed in Chapter XX of the ICD-10-CM, excluding cases of hanging, suffocation, drowning or near drowning, poisoning, burning, and electrocution. In addition, patients without baseline information and outcomes were excluded. The study was retrospective, non-interventional, and based on anonymous registration data only.

Development of the new model

Definition of independent variables

ICD-10 codes S00–S99 were selected as descriptors of injuries. The total number of related codes in this database was 1396. We first divided ICD-10-CM codes into 20 new codes (A1, A2……G2, and G3) according to the severity and injury region because many ICD-10-CM codes rarely occurred. The specific classification and coding methods are shown in Supplementary Table 1.

In addition, we extracted the comorbidity information from the ICD-10-CM codes, including myocardial infarction, congestive heart failure, rheumatologic disease, chronic pulmonary disease, cerebrovascular disease, peptic ulcer disease, liver disease, diabetes, hypertension, chronic renal failure, and malignancy.^[8] The corresponding ICD-10 codes are shown in Supplementary Table S1.

We extracted the diagnosis of “traumatic shock” (T79.401) from the ICD-10 codes to assess the physiological reactions of patients because too much blood pressure and heart rate data had been lost. Consciousness was reduced to a binary variable: coma or not. Coma was defined as the inability to respond to commands with movement or open the eyes in response to any stimulus, and a loss of verbal ability (e.g., inability to moan or hum),^[9] in other words, Glasgow coma score ≤8. The mechanism of injury was divided into traffic, fall, blunt injury, penetrating injury, and other.

We divided age into multiple categorical variables divided into 10-year blocks because continuous variables often show a nonlinear relationship with outcomes. If during model building a category showed no or only minor effects, this category was merged with the reference category.^[10]

Establishment of model

We then constructed a logistic regression model. This model examined the joint relationships between prognostic variables of interest and mortality.

The model can be expressed as:

$$P(\text{death})=1/(1+{\text{e}}^{-\text{b}})$$

where P (death) is the possibility of death and e is 2.718282.

The value of b was calculated as follows:

$$\text{b}=\text{b}0+{\sum }_{i=1}^{20}{b}_i{X}_i+{\sum }_{i=1}^{10}{a}_i{C}_i+{\alpha }_1\text{age}+{\alpha }_2\text{sex}+{\alpha }_3\text{coma}+{\alpha }_4\text{traumatic}\text{shock}+{\alpha }_5\text{injury}\text{mechanism}$$

where Xi, I = 1…20 is a binary indicator variable for each of the 20 region-severity codes and Ci, i = 1…10 is a binary indicator variable for each complication. Sex, coma, and traumatic shock were binary variables, with male sex set as the control condition, and age and injury mechanisms as multiple categorical variables.

We used a forward stepwise procedure with a test for backward elimination of covariates to obtain the main effects model. The significance levels were set at 0.05 for entry into and 0.10 for elimination from the model. Variables that failed to achieve sufficient power were eliminated from the model. Each variable was assigned a score based on their coefficients in the model.

Assessment of model performance

We evaluated model performance based on discrimination and calibration.^[11] Discrimination measures the ability of a score to separate survivors from non-survivors.^[12] This is best summarized by calculating the sensitivity and specificity for all potential cutoff points of the score. These values are summarized in a receiver operating characteristic (ROC) curve. The area under the ROC curve (AUC) varied between 0.5 (discrimination by chance) and 1.0 (perfect separation of survivors and non-survivors).^[13,14] Discrimination is the most important value when measuring the performance of a model.

Calibration reflects the consistency between the predicted mortality and observed mortality. In this study, the Brier score was used as the calibration index; the closer the score was to 0, the higher the calibration. Calibration was also graphically checked by assessing the deviation from the 45° line of identity from the graph representing predicted probabilities against observed probabilities of survival.^[15] The probability of survival was categorized into ten bands from 0 to 1 in steps of 0.1, and the proportion of survivors within each band would represent the observed probabilities. The predicted probabilities were the average probabilities within each band.

Internal validation

A special statistical technique, bootstrapping, was used to perform an internal validation of the model's performance. A large number of random samples (this study used 1000 bootstrap iterations) was drawn with replacement from the original sample. Subsequently, for each iteration, a logistic model was fitted within the bootstrap sample and tested on the original sample. The difference in performance indicated the expected optimism, which was subtracted from the apparent performance estimates of the original model.

Statistical analysis

Descriptive statistics are provided as counts and percentages for categorical variables, and mean and standard deviation (SD) for continuous variables. All analyses were performed using SPSS statistical software version 24 (IBM Corp, Armonk, NY, USA) and R software version 4.0.2.

Results

Characteristics of study subjects

During the 7-year study period, a total of 40,205 patients meeting the inclusion criteria were included in the study; 39,009 patients survived and 1196 died. The mortality rate was 3%. The characteristics of patients in the database are shown in Table 1, from which it showed that older, male trauma patients as well as patients with traumatic shock or coma at admission have a high risk of death. Table 2 shows the distribution of new injury codes and complications in the database. The new injury codes are showed to be significantly correlated with mortality, except for C2 and D2, whereas complications are significantly correlated with mortality, except for liver disease, malignant tumor, and chronic lung disease.

Table 1.

Characteristics of patients with traumatic events in the database (2012–2018).

Characteristics	Lived (N = 39,009, 97.0%)	Died (N = 1196, 3.0%)	P value
Age, years, mean (SD)	43.8 (17.2)	53.3 (18.6)	<0.001
Male, n (%)	26,838 (68.8)	935 (78.2)	<0.001
Length of stay, days, mean (SD)	14.5 (19.9)	9.1 (22.4)	<0.001
ICU admission, n (%)	6110 (15.7)	1126 (94.1)	<0.001
Mechanical ventilator, n (%)	5354 (13.7)	1157 (96.7)	<0.001
Mechanism of trauma, n (%)			<0.001
Traffic	20,312 (52.1)	772 (64.5)
Fall	8686 (22.3)	321 (26.8)
Blunt injury	5500 (14.1)	38 (3.2)
Penetrating	2874 (7.4)	13 (1.1)
Other	1637 (4.2)	52 (4.4)
Surgical procedure, n (%)	16,168 (41.4)	922 (77.1)	<0.001
Traumatic shock, n (%)	1000 (2.6)	192 (16.1)	<0.001
Coma, n (%)	1545 (4.0)	947 (73.1)	<0.001

SD: Standard deviation; ICU: Intensive Care Unit.

Table 2.

Distribution of injury and comorbidity in the database.

Injuries and Complications	Lived (N = 39,009, 97.0%)	Died (N = 1196, 3.0%)	P value
Head injury, n (%)
A1	7885 (20.2)	315 (26.3)	<0.001
A2	3694 (9.5)	522 (43.6)	<0.001
A3	1698 (4.4)	339 (28.3)	<0.001
A4	3177 (8.1)	794 (66.4)	<0.001
A5	392 (1.0)	463 (38.7)	<0.001
Face injury, n (%)
B1	8758 (22.5)	205 (17.1)	<0.001
B2	2808 (7.2)	143 (12.0)	<0.001
Neck injury, n (%)
C1	1178 (3.0)	8 (0.7)	<0.001
C2	110 (0.2)	1 (0.1)	0.410
Chest injury, n (%)
D1	3327 (8.3)	69 (5.8)	0.002
D2	7924 (20.3)	242 (20.2)	0.940
D3	5495 (14.1)	329 (27.5)	<0.001
Abdomen injury, n (%)
E1	2923 (7.5)	39 (3.3)	<0.001
E2	1357 (3.5)	126 (10.5)	<0.001
E3	417 (1.1)	82 (6.9)	<0.001
Spine injury, n (%)
F1	3978 (10.2)	58 (4.8)	<0.001
F2	1052 (2.7)	21 (1.8)	0.047
Injury of extremities and pelvic ring, n (%)
G1	16,129 (41.3)	83 (6.9)	<0.001
G2	13,426 (34.4)	199 (16.6)	<0.001
G3	3494 (9.0)	92 (7.7)	0.013
Comorbidity, n (%)
Myocardial infarction	1170 (3.0)	58 (3.8)	<0.001
Congestive heart failure	76 (0.2)	15 (1.3)	<0.001
Chronic renal failure	112 (0.3)	27 (2.3)	<0.001
Cerebrovascular disease	997 (2.6)	71 (5.9)	<0.001
Hypertension	3762 (9.6)	62 (5.2)	<0.001
Diabetes	1956 (5.0)	23 (1.9)	<0.001
Liver disease	542 (1.4)	14 (1.2)	0.520
Malignancy	39 (0.1)	2 (0.2)	0.350
Chronic lower respiratory diseases	461 (1.2)	22 (1.8)	0.400
Peptic ulcer disease	66 (0.2)	16 (1.3)	<0.001

A1, A2…G2, G3 are new region-severity codes.

Presentation of the new model

The model was established using logistic regression analysis. The final model is shown in Table 3. Sex, age, partial new injury codes, partial comorbidities, traumatic shock, and state of consciousness were included in the new model as key predictors of mortality. Each predictor was assigned a score based on its influence on mortality (coefficient). The risk of death was graded according to the total score. The maximum score of this model was 232; we classified 0–47 as extremely low risk (risk of mortality is <10%), 48–60 as low risk (risk of mortality is 11%–30%), 61–73 as medium risk (risk of mortality is 31%–60%), 74–90 as high risk (risk of mortality is 61%–90%), and >90 as extremely high risk (risk of mortality is >90%). Traumatic shock, coma, and advanced age (>80 years) had the greatest influence on risk of mortality. For new injury codes, head injury (A3 = 16; A4 = 17), abdominal injury (E2 = 10; E3 = 22), and spinal cord injury (F2 = 11) had the greatest influence on risk of mortality. Among comorbidities, congestive heart failure and chronic renal failure had the greatest influence on mortality, with a score of 16. The following variables were tested for inclusion, but did not reach sufficient power to be included in the model: mechanism of trauma, A1, B1, B2, C1, C2, D1, D2, E1, F1, G1, G2, hypertension, diabetes, liver disease, malignancy, and chronic lower respiratory diseases.

Table 3.

China Mortality Prediction Model in Trauma-ICD10.

Variable	Category	Coefficient	Score	P value	OR	95% CI of OR
Sex	Male	0.295	3	0.002	1.34	1.12–1.62
Age	≤40	0	0	-	-	-
	41–50	0.415	4	<0.001	1.51	1.21–1.90
	51–60	0.981	10	<0.001	2.64	2.12–3.29
	61–70	1.111	11	<0.001	3.07	2.37–3.97
	71–80	1.817	18	<0.001	6.22	4.59–8.41
	81–90	2.305	23	<0.001	10.03	6.99–14.39
	≥91	2.719	27	<0.001	15.48	7.57–31.66
A2	Y	0.491	5	<0.001	1.64	1.36–1.97
A3	Y	0.686	7	<0.001	1.97	1.61–2.40
A4	Y	1.614	16	<0.001	5.07	4.14–6.22
A5	Y	1.687	17	<0.001	5.35	4.37–6.55
D3	Y	0.227	2	0.010	1.27	1.06–1.52
E2	Y	0.991	10	<0.001	2.69	1.98–3.66
E3	Y	2.205	22	<0.001	9.07	6.28–13.11
F2	Y	1.131	11	<0.001	3.10	1.86–5.16
G3	Y	0.401	4	0.007	1.49	1.12–2.00
Traumatic shock	Y	2.808	28	<0.001	16.58	12.74–21.59
Myocardial infarction	Y	0.619	6	0.002	1.86	1.26–2.73
Congestive heart failure	Y	1.579	16	<0.001	4.73	2.35–9.55
Chronic renal failure	Y	1.576	16	<0.001	4.79	2.71–8.47
Cerebrovascular disease	Y	0.301	3	0.044	1.35	0.98–1.78
Peptic ulcer disease	Y	1.013	10	0.010	2.75	1.27–5.96
Coma	Y	2.910	29	<0.001	18.38	14.88–22.71
Constant		−6.916

A1, A2…G2, G3 are new region-severity codes.

Model performance

Figure 1 shows the ROC curve of the new model. The AUC is 0.964, indicating the model has excellent discrimination. The AUC after internal validation is 0.963. The Brier score is 0.0177, which is very close to 0, indicating that the calibration of the model is acceptable. The Brier score after internal validation is 0.0178, and the model calibration curve and adjusted calibration curve are shown in Figure 2. Mortality is underestimated before 0.5 and overestimated after 0.5; however, the overall difference from the ideal curve was small.

Figure 1.

Receiver operating characteristic curves for China Mortality Prediction Model in Trauma-ICD10.

Figure 2.

The calibration plot that shows apparent calibration curve and bias-corrected calibration curve after bootstrapping.

Figure 3 demonstrates a sample application of the new score. A 72-year-old man with traumatic epidural hemorrhage (S06.4, A4), subarachnoid hemorrhage (S06.7, A3), and coma would have a total score, according to the new model of 73 points, corresponding to a medium risk of mortality.

Figure 3.

Example for the application of the new CMPMIT-ICD10 score. CMPMIT: China Mortality Prediction Model in Trauma.

Discussion

Mortality within 1 h of severe trauma caused by high-energy injury is approximately 45%.^[16] Treatment of these patients requires extreme timeliness. Accurate early prediction of the risk of death may have the potential to inform triage decisions and treatment, or stratify patients for further care. In addition, the establishment of trauma centers in China is still in its infancy; however, there is no proper means to evaluate the medical quality of trauma centers. Based on the above two points, the development of a new trauma mortality prediction model applicable to China based on a Chinese database is of great importance to improve trauma treatment capacity in China.

In contrast to other clinical prediction models, it is more difficult to establish a trauma mortality prediction model because there may be thousands of combinations of injuries, and it is challenging to reflect the influence of different injuries on mortality in one model. Therefore, a method to quantify the severity of different traumas is the key to establishing a trauma prediction model. As the first scoring system to quantify the severity of injury, the ISS has become the common language of trauma surgeons and related research. It is widely used worldwide. The basis of the ISS is the AIS, which is based on expert consensus. The AIS assigns different injuries scores ranging from 1 to 6 points according to their severity, and the ISS consists of the sum of the squares of the three highest AIS scores for the three most injured ISS regions. Subsequent models, such as the Trauma and Injury Severity Score and the Revised Injury Severity Classification, are based on the ISS and incorporate the patient's basic condition, acute physiological response, and biochemical indicators into the model.^[10,17] The predictive ability of these models has greatly improved, and the models are widely used in Europe and America.

Prediction models based on AIS and ISS have many drawbacks; however, the biggest obstacle is the complicated coding system of the AIS, which requires considerable time and effort to train coders, and it is difficult to obtain accurate AIS codes in databases in developing countries, such as China. Moreover, although AIS coding has an accurate description of trauma, it is often not available in the early stages of trauma, which limits the clinical application of this type of predictive model. Therefore, it is more often used for retrospective quality assessment.

The other type of prediction model is based on the ICD-10-CM lexicon, which mainly includes the TMPM-ICD10. In contrast to the AIS codes, the ICD codes are routinely assigned to all injuries that any hospitalized trauma patients sustain. As a result, injury severity models based upon ICD codes allow mortality prediction without recourse to recoding. The ICD-10-CM codes do not actually include the severity of injury; therefore, the TMPM-ICD10 first established two different probit regression mortality models.^[18] The resulting model-averaged regression coefficient values provide an empiric measure of individual injury severities that can be used to compare the severities of individual injury and incorporate individual injury severities into the TMPM-ICD10 mortality model. Nevertheless, the method of calculating the TMPM is too complicated and does not consider the influence of patients’ baseline conditions and acute physiological reactions on mortality.

Our current study simplifies the above process. We first classified the ICD10 codes of injury into 20 categories, and then fitted the regression coefficient of each type of injury, which represents the severity of this type of injury, through regression analysis. Finally, certain injury categories of lower severity did not achieve sufficient power to be included in the model. In other words, the model only considered the effect of severe trauma on mortality. This result has clinical face validity. Trauma surgeons usually describe a patient's clinical condition using the patient's one or two the worst injuries, as opposed to listing all of a patient's injury because they believe that patient outcome is a function of the worst injury.

In addition, in contrast to the TMPM-ICD10, the influence of comorbidity and post-traumatic physiological responses on mortality was further considered in our new model. Instead of calculating the Charlson comorbidity index, which may not be applicable to trauma patients, each comorbidity was included in the model separately.^[19]

An innovation of this study is that traumatic shock was incorporated into the model as a binary variable, instead of establishing a model with blood pressure and heart rate as predictive variables because too much data on blood pressure and heart rate was missing in the database, and the blood pressure and heart rate recorded in the database may have been affected by early care (such as fluid rehydration and blood transfusion). In addition, we simplified the state of consciousness to coma or not, further simplifying the model, which may be more convenient for clinical application. Coma and traumatic shock had the highest scores in the model, indicating that central nervous system injury and exsanguination are the main causes of death, which is consistent with the results of previous epidemiological studies on trauma death.^[20]

The final performance of the model was excellent, with an AUC of 0.964, which is almost the highest level of discrimination among recent studies of trauma prediction models. Although there is a certain deviation between the calibration curve of the model and the ideal 45° line, the Brier score of the model was only 0.0177, indicating that the calibration of the model was good. We used the bootstrap method for internal validation to maximize the use of existing data. The result of internal validation was very similar to the model performance for the original data, which suggests that there was no over-fitting, causing overestimation of model performance.

Another significant advantage of this model is that the prediction of mortality only requires baseline information, such as age, sex, ICD-10-CM codes, and consciousness status. When used for retrospective quality assessment in medical institutions, these variables do not require excessive energy and money to collect. There is less likely to be missing data in the database.

This study has two main limitations. First, although the performance of the model was satisfactory, many ICD-10 codes were combined with a common coefficient in the model after classification, which may not be accurate enough to quantify the severity of injury. Second, no comparison was made between our model and TMPM-ICD10 because the ICD-10-CM coding method of the trauma database in China is different from that in Europe and America. Next, we will conduct a multi-center study to externally validate this new model.

In this study, the CMPMIT-ICD10 was established on the basis of a trauma database in China. The new model showed great discrimination and calibration, performed well in internal validation, and can be further verified externally.

Conflicts of interest

None.