Machine Learning Models for Prediction of Reinjury After Penetrating Trauma

Abstract

This study compares several machine learning models for prediction of reinjury after penetrating trauma.

Hospitalization for violent trauma costs as much as $8.5 billion per year in the United States, and reinjury rates are as high as 40%. Most trauma reinjury studies include data from a single center. However, nearly 60% of reinjured patients present to a different hospital, making prediction of reinjury difficult. Machine learning allows computers to learn from iterations without programming. Machine learning is highly accurate for predicting surgical outcomes and suicide risk but, to our knowledge, has never been used in trauma outcomes. The purpose of this study was to compare several machine learning models for prediction of reinjury after penetrating trauma.

Methods

The Nationwide Readmission Database is the only national database that tracks patients across different hospitals within a state. We queried the period from January 1, 2013, through December 31, 2014, for all survivors of nonelective admission with a penetrating trauma E-code. Reinjury was defined as nonelective readmission with a penetrating trauma E-code. Readmissions were excluded if they could be related to the index admission based on previously described methods. Prediction models were created using 4 machine learning classifiers independently with cross validation performed by dividing the data set into 10 subsets and using 1 for testing and the remaining 9 for training. This process was repeated using all subsets, and means were calculated for receiver operating characteristic curves. Modeling was performed with RapidMiner Studio software (version 7.5; RapidMiner, Inc). Because this research used a deidentified government database, this study was deemed exempt from approval by the institutional review board of the University of Miami. Furthermore, because this did not constitute human subjects research, consent was not required.

Results

Of 63 678 patients admitted for penetrating trauma, 1229 (1.9%) experienced reinjury. The most common E-code was for suicide and self-inflicted injury by cutting and piercing instrument at admission (n = 22 889 [34.2%]) and readmission (n = 798 [63.3%]). The most common diagnosis group for admission was major depressive disorders and other/unspecified psychoses (n = 6913 [10.3%]); for readmission, bipolar disorders (n = 1329 [12.0%]). The Table and Figure show that the gradient boosted trees classifier had the highest accuracy (96.8%) and area under the curve (AUC) (0.74). This classifier reported high importance percentages for E-codes (11.4%-21.4%) and self-injury (5.2%). The logistic regression classifier also performed well (accuracy, 94.7%; AUC, 0.73) (Table and Figure), and self-injury was a significant attribute (z score, 3.38; P < .01).

Table. Results of Cross-Validation for Each Classifier.

Classifier	AUC	Accuracy, %	Specificity, %	NPV, %
Naive Bayes	0.72	81.3	82.0	98.8
Logistic regression	0.73	94.7	96.2	98.6
Gradient-boosted trees	0.74	96.8	98.5	98.2
Deep learning	0.72	95.6	97.2	98.3

Abbreviations: AUC, area under the curve; NPV, negative predictive value.

Figure. Receiver Operating Characteristic Curves for Each Classifier for Predicting Reinjury After Penetrating Trauma.

Data are expressed as mean results of cross-validation.

Discussion

This study is, to our knowledge, the first nationwide evaluation of reinjury after penetrating trauma. We found a reinjury rate of 1.9%, similar to rates of prior multi-institutional studies (0.8%-2.0%). This study found that the most common E-codes from penetrating injury were related to self-injury, and the most common diagnosis groups were related to psychiatric disorders. As many as half of all patients who commit suicide have been reported to have prior admissions for trauma, and nonsuicidal self-injury has been reported to recur in 75% of self-injurers.

Predicting violent behavior has proven to be difficult. Teo et al found that risk assessments by attending psychiatrists were moderately accurate (AUC, 0.70), whereas risk assessments by residents were no better than chance (AUC, 0.52). A growing number of studies have demonstrated the usefulness of machine learning for medical decision making. Outcomes after cerebral arteriovenous malformation radiosurgery have been predicted with 74% accuracy and an AUC of 0.71. Suicide attempts have been predicted using machine learning techniques with an AUC of 0.84. In this study, machine learning was used for the first time to predict reinjury due to penetrating trauma. We found favorable AUCs with high accuracy (Table and Figure), and self-injury proved to be an important component in making this prediction.

This study is limited by use of an administrative hospital database, which may include coding errors or institutional biases. Predictive models using this type of database are inherently limited by these biases. Regardless, this study demonstrated that machine learning may be useful for developing highly accurate predictive models for reinjury after penetrating trauma using a large database of readily available data. Future studies should include validation of the predictive model with outside data and incorporation into existing electronic medical record systems to provide real-time clinical decision support.