Evaluating the Discriminatory Power of a Computer-based System for Assessing Penetrating Trauma on Retrospective Multi-Center Data

Abstract

Objective

To evaluate the discriminatory power of TraumaSCAN-Web, a system for assessing penetrating trauma, using retrospective multi-center case data for gunshot and stab wounds to the thorax and abdomen.

Methods

80 gunshot and 114 stab cases were evaluated using TraumaSCAN-Web. Areas under the Receiver Operator Characteristic Curves (AUC) were calculated for each condition modeled in TraumaSCAN-Web.

Results

Of the 23 conditions modeled by TraumaSCAN-Web, 19 were present in either the gunshot or stab case data. The gunshot AUCs ranged from 0.519 (pericardial tamponade) to 0.975 (right renal injury). The stab AUCs ranged from 0.701 (intestinal injury) to 1.000 (tracheal injury).

Background

In a prior study, a system for assessing thoraco-abdominal penetrating trauma (TraumaSCAN) was developed. This system was evaluated by a retrospective assessment of 26 gunshot wound cases utilizing external wound information, bullet locations, and other patient findings. The results for the system showed good sensitivity and specificity, with areas under the ROC curve ranging from 0.835 to 0.992 for 11 conditions present in the cases assessed.

TraumaSCAN-Web¹ is a platform-independent system for assessing thoraco-abdominal penetrating trauma. It was developed as a successor to TraumaSCAN (TS) ^2–5 and incorporates new methods for geometric reasoning about anatomical structure injury due to gunshot and stab wounds. These methods are based, in part, on a large body of research relating to wound modeling and estimation. ^6–12 It also utilizes an updated Bayesian network (BN) ^13–16 for reasoning about the manifestations of injury to different structures.

The objective of this study is to examine the diagnostic accuracy of TraumaSCAN-Web on a retrospective multi-center data set.

Methods

TraumaSCAN-Web (TSW) was developed at the Decision Systems Group, Brigham and Women’s Hospital. It updates the previous system (TS), which was developed for the Silicon Graphics platform, and took advantage of specialized graphics libraries and dedicated graphics hardware for that platform. This limited possibilities for use of TS on other, more common computing platforms. The introduction of Java, as well as advances in graphics hardware for personal computers, have made it possible to develop a platform-independent web-accessible version. TSW makes use of Java3D, a scenegraph applications programmer interface (API), for rendering complex 3D scenes. The probabilistic reasoning module utilizes UCLA’s SamIAm Bayesian network API (AR Group UCLA, Los Angeles, CA).

Computation time for gunshot wound injury estimation increases exponentially with the number of wounds and bullets because there is a combinatorial explosion in the number of hypotheses to be explored. The space of hypotheses represents all possible pairings of external wounds and bullets. For this reason, both TS and TSW utilize approximate methods when the number of wounds and lodged bullets exceeds four (a maximum of 24 hypotheses). TS’s approach was to estimate the region of damage for such gunshot wound cases by using the convex hull of points corresponding to wound and bullet locations. TSW randomly selects a hypothesis from the space of possible hypotheses and fully explores it. This new approach exploits the fact that different injury hypotheses for the same set of wounds and bullets have overlapping entry-exit wound pairings.

The BN was updated to correct errors discovered in the course of evaluating TraumaSCAN, to expand on the penetrating trauma conditions that can be evaluated, and to improve upon the discriminatory power of the system.

Updated network nodes are listed in Table 1, with the new ones in italics. Efforts were made to optimize evaluation in mediastinal injuries, and to further improve evaluation of abdominal injuries by separating more solid organs from the general category of non-specific intra-abdominal injuries (NSIA).

Table 1.

Summary of patient findings included in the revised probabilistic reasoning module. All additions are listed in italics. TGRI = tenderness, guarding, rebound, ileus. XR = X-ray. IVP = Intravenous Pyelogram. (L) = left. (R) = right. NG = nasogastric.

(L) chest wound present	distended neck veins
(R) chest wound present	stridor
(L) decreased breath sounds	(L) tracheal deviation
(R) decreased breath sounds	(R) tracheal deviation
Hemoptysis	odynophagia
Shock	+ lavage results for blood
muffled heart sounds	blood in NG tube
weak arm pulses	TGRI
weak leg pulses	distended abdomen
+ IVP for left renal injury	free air in abdomen
+ IVP for right renal injury	+ XR: lacerated diaphragm
hematuria

Link changes for previously existing nodes include adding causal relationships for tension pneumothorax and hemothorax with respect to chest wounds, and updating the links for the new nodes. Both arm and leg pulses were linked to descending thoracic aortic injury, correcting an error that only linked weak arm pulses in the original. ¹⁷ Summary figures are available for both networks, and are shown in Figure 1 (original) and Figure 2 (updated).

Figure 1.

Original Bayesian network for TraumaSCAN.

Figure 2.

Updated Bayesian network for TraumaSCAN-Web.

Modification of the BN required updating some of the conditional probability tables. This new data was obtained from an expert clinician (John R. Clarke).

Thoraco-abdominal penetrating trauma case data was extracted from two sources. The Partners Healthcare Research Patient Data Registry (RPDR) ¹⁸ was queried for cases occurring between 1994 and 2002 using relevant ICD9-CM diagnosis codes including supplemental External Cause of Injury codes (E-Codes). Table 2 summarizes the codes used for this purpose. Codes from the upper box were ORed together and then ANDed with codes from the lower box. (i.e., patients selected had one or more wounds as well as a gunshot or stab injury).

Table 2.

All ICD9-CM codes and E-Codes used for the identification of cases. XX used in a diagnosis code indicates all sub-codes are included. SI = self-inflicted. LI = legal intervention.

Open wound of back (876.XX)

Open wound of buttock (877.XX)

Open wound of chest (wall) (875.XX)

Internal injury of thorax, abdomen and pelvis (860–869)

Assault by handgun (E965.0)

Assault by hunting rifle (E965.2)

Assault by other and unspecified firearm (E965.4)

Assault by cutting and piercing instrument (E966)

Assault by military firearms (E965.3)

Injury by handgun (E985.0)

Injury by hunting rifle (E985.2)

Injury by other and unspecified firearm (E985.4)

Injury by cutting and piercing instruments (E986)

Injury by military firearms (E985.3)

Injury by airgun (E985.6)

SI injury by handgun (E955.0)

SI injury by hunting rifle (E955.2)

SI injury by unspecified firearm, explosives (E955.9)

SI injury by cutting and piercing instruments (E956)

SI injury by military firearms (E955.3)

Injury due to LI by firearms (E970)

Injury due to LI by cutting and piercing instrument (E974)

The RPDR is a central data warehouse for both Brigham & Women’s Hospital and Massachusetts General Hospital. Additional cases from MCP Hospital were identified using a similar process with that institution’s trauma registry from 1988 to 2001.

A total of 471 cases were ultimately obtained, and the patient charts corresponding to these cases were abstracted. Cases were categorized by mechanism of injury (gunshot and stab), then filtered to exclude a number of conditions.

Any case lacking either a diagram or detailed description of the wound locations was excluded because TSW requires estimates of surface wound and bullet locations to provide reliable assessment results. The geometric modeling methods in TS require gunshot cases to have an even number of external wounds and lodged bullets in total. This is because the injury calculations are generated on the basis of a path between two points (a wound pair). For similar reasons, extensive bullet fragmenting and shotgun cases were excluded. Shotgun cases were never intended to be evaluated by TS/TSW.

A fifty percent random subset of cases was selected to evaluate TSW. This sample was chosen to reserve the remaining data for future use in training the BN on case data and reevaluating its discriminatory power. The evaluation data set included 80 gunshot cases and 114 stab cases. Discharge summaries, radiological reports, operative notes, ICD-9 codes, and autopsy reports were used for determining actual injuries.

In the patient chart, wound location information is usually documented by drawing on an anterior/posterior anatomy figure with an accompanying textual description. The diagram is used as the primary source of this information unless found to be consistently conflicting with multiple other descriptions.

The wound information and clinical findings from each case were transcribed to TraumaSCAN-Web, and the conditional probabilities outputted. The AUC ¹⁹ for each diagnosis was calculated using the nonparametric method. ²⁰

Results

Sensitivities and specificities obtained for the 80 cases of gunshot injuries, and 114 cases of stab injuries were used to calculate AUCs for 16 conditions (stab) and 18 conditions (gunshot) of 23 possible conditions.

There were some diagnoses that had no occurrences (tension pneumothorax and esophageal injuries in both data sets, tracheal injuries in gunshot wounds, and descending aorta and left renal injuries in stab wounds), and we were unable to evaluate the system’s ability to discriminate in those instances.

For each condition, the AUC was reported with the 95 percent confidence intervals (95% CI). The gunshot results are given in Table 3, and the stab results are given in Table 4. There are 26 pieces of evidence that can be input into TraumaSCAN-Web to aid diagnostic reasoning about injury consequences in the BN. On average, 17 out of the 26 findings were known for each gunshot case, and 16 out of the 26 were known for each stab case. Discrimination of injuries and diagnoses relating to the lung in gunshot cases ranged from an AUC of 0.845 to 0.915, and 0.810 to 0.934 in stab cases. Heart related AUC’s ranged from 0.519 to 0.720 for the gunshot cases, and 0.975 to 0.978 for the stab cases. These conditions were relatively rare in the case data (Tables 3 and 4). Non-specific intra-abdominal injury (includes intra-abdominal organs and vascular structures but excludes abdominal wall injuries) estimates were 0.875 for gunshot cases and 0.755 for stab cases. Renal injuries were 0.947 to 0.957 for gunshot wounds and 0.81 for stab wounds. Other intra-abdominal solid organ injury estimates varied from 0.701 to 0.898 depending on the mechanism of injury.

Table 3.

Summary of AUC’s for each condition for gunshot wound cases. Desc = Descending. Pneumo = Pneumothorax. TB = Tracheo-bronchial. NS IA = Non-specific intra-abdominal

Diagnosis	Events	AUC (95% CI)
Right Lung Injury	11	0.910 (0.819–1.000)
Left Lung Injury	19	0.915 (0.849–0.982)
Heart Injury	4	0.720 (0.459–0.982)
Esophagus Injury	0	-
Tracheal Injury	0	-
Desc. Aorta Injury	2	0.814 (0.645–0.983)
Diaphragm Injury	12	0.857 (0.726–0.988)
Liver Injury	10	0.767 (0.601–0.933)
Intestinal Injury	18	0.811 (0.704–0.918)
Stomach Injury	13	0.898 (0.821–0.975)
Left Kidney Injury	4	0.944 (0.887–1.000)
Right Kidney Injury	1	0.975 (0.940–1.000)
TB Tree Injury	0	-
Right Hemothorax	8	0.870 (0.790–0.950)
Left Hemothorax	13	0.893 (0.791–0.996)
Right Simple Pneumo	6	0.881 (0.793–0.968)
Left Simple Pneumo	13	0.845 (0.737–0.953)
Right Tension Pneumo	0	-
Left Tension Pneumo	0	-
Pericardial Tamponade	2	0.519 (0.111–0.927)
NS IA Injury	34	0.875 (0.793–0.957)
Right Renal Injury	1	0.975 (0.940–1.000)
Left Renal Injury	4	0.947 (0.891–1.000)

Table 4.

Summary of AUC’s for each condition for stab wound cases. Pneumo = Pneumothorax. TB = Tracheo-bronchial. NS IA = Non-specific intra-abdominal

Diagnosis	Events	AUC (95% CI)
Right Lung Injury	21	0.887 (0.823–0.950)
Left Lung Injury	18	0.910 (0.857–0.962)
Heart Injury	8	0.975 (0.937–1.000)
Esophagus Injury	0	-
Tracheal Injury	1	1.000 (1.000 – 1.000)
Desc. Aorta Injury	0	-
Diaphragm Injury	9	0.830 (0.721–0.938)
Liver Injury	12	0.833 (0.721–0.946)
Intestinal Injury	11	0.701 (0.541–0.860)
Stomach Injury	4	0.814 (0.724–0.904)
Left Kidney Injury	0	-
Right Kidney Injury	4	0.805 (0.482–1.000)
TB Tree Injury	0	-
Right Hemothorax	12	0.810 (0.720–0.900)
Left Hemothorax	5	0.899 (0.827–0.972)
Right Simple Pneumo	16	0.884 (0.811–0.957)
Left Simple Pneumo	18	0.934 (0.889–0.979)
Right Tension Pneumo	0	-
Left Tension Pneumo	0	-
Pericardial Tamponade	2	0.978 (0.946–1.000)
NS IA Injury	36	0.755 (0.658–0.852)
Right Renal Injury	4	0.809 (0.489–1.000)
Left Renal Injury	0	-

Discussion

The results obtained from measuring AUC’s for each condition indicated that TraumaSCAN-Web performed well in general for discriminating occurrences of injuries. Lung injuries were well estimated in both mechanisms, likely due to the overall surface area of these structures, and the relative frequency of injury. Cardiac estimations were poor for gunshot wounds, but quite good for stab wounds. Solid organ intra-abdominal injury estimations performed much better on gunshot wounds, which is to be expected since both the trajectory and depth of stab wounds is less well characterized. Non-specific intra-abdominal injuries were moderately well estimated by both methods.

Limitations of this study include the size of the study population. Although much larger than the original pilot study, this sample size is still relatively small, and manual data extraction is the primary limitation for increasing the study sample size. Transcription error may have occurred twice in this evaluation: when the original medical team documented the injuries, and when the user inputs wound locations to TSW.

Overall, this study shows promise for the system as a triage tool for correctly estimating injuries based on initial clinical findings and wound locations. This system is currently being evaluated for its ability to help general surgery and emergency department residents correctly diagnose injury patterns. In addition, future work for the system will include training the system on actual case data to refine the diagnostic reasoning module beyond the expert clinician estimates currently being used.