Validation of thoracic injury rule out criteria as a decision instrument for screening of chest radiography in blunt thoracic trauma

Abstract

Background

Thoracic injury rule out criteria (TIRC) were first introduced as a decision instrument for selective chest radiography in blunt thoracic trauma in 2014. However, the validity of this model has not been assessed in other studies. In this regard, the present survey evaluates the validity of TIRC model in a multi-center setting.

Methods

In this cross-sectional study, clinical presentations and chest radiograms of multiple trauma patients referring to 6 educational hospitals in Iran were evaluated. Data were gathered prospectively during 2015. In each center, data collection and interpretation of radiograms were conducted by two different emergency medicine specialists. Measures were then taken for assessment of discriminatory power and calibration of the model.

Results

Data from 2905 patients were gathered (73.17% were male; the mean age was 33.53 ± 15.42 years). Area under the receiver operating characteristics curve of the TIRC model for detection of thoracic traumatic injuries was 0.93 (95%CI: 0.93–0.94). Sensitivity and specificity of the model were 100 (98.91–100) and 67.65 (65.76–69.45), respectively. The intercept of TIRC calibration plot was 0.08 (95%CI: 0.07–0.09), and its slope was 1.19 (95%CI: 1.15–1.24), which are indicative of the model being perfect in detecting presence or absence of lesions in chest radiograms.

Conclusion

The findings are corroborative of external validation, good discrimination, and proper calibration of TIRC model in screening of multiple trauma patients for obtaining chest radiograms.

1. Background

Thoracic trauma as one of the most important causes of morbidity and mortality in the first four decades of life^{1, 2} is the cause for 25% of trauma-related deaths.³ Early diagnosis of these injuries can significantly decrease their burden. Radiology plays an important role in the assessment of trauma patients. In the past, the Advanced Trauma Life Support recommended that radiograms should be obtained from the chest, pelvis, and neck of all multiple trauma patients. However, recently it has limited the neck and pelvis radiograms to hemodynamically unstable patients or subjects with positive findings in their physical examinations.⁴ These changes are made based on the findings that suggest plain radiograms present little information useful for making decisions about the hemodynamically stable patients or subjects with negative cervical and pelvic physical examinations.^{5, 6, 7, 8, 9}

Recently similar attempts have been made to decrease the burden of obtaining unnecessary radiograms in thoracic trauma. In this regard two different scoring systems of NEXUS chest and thoracic injury rule out criteria (TIRC) have been proposed by two separate research teams.^{10, 11} NEXUS chest, developed based on the data from 2628 patients referring to three trauma centers, is a 7-item scoring system that includes age of more than 60 years old, rapid deceleration mechanism (falling from a height of more than 20 ft or a motor vehicle accident with a speed of more than 40 mph), chest pain, intoxication, altered mental status, distracting painful injury and tenderness to chest wall palpitation as factors predictive of thoracic trauma. The validation study of this scoring system was conducted on 9905 patients.¹¹ However, information about the rapid deceleration criterion is rarely available to the physicians. This is mainly due to inaccurate recording of information about the accidents by emergency medical services, particularly in developing countries, which can limit the application of this scoring system.

In 2014 Forouzanfar et al. develop the TIRC model based on the data gathered from 2608 patients referring to one trauma center. According to this model any of these factors can predict a thoracic lesion: age over 60, hemodynamic instability, Glasgow Coma Scale (GCS) lower than 15, crepitation, decrease in pulmonary sounds, pain and tenderness of the chest wall, thoracic skin abrasion, and dyspnea.¹⁰ As can be seen, this model does not have the limitations of NEXUS chest scoring system. However, the validity of the TIRC model has not been evaluated in another survey. Accordingly, in the present study, we aimed to assess the validity of TIRC model in a multi-center survey.

2. Methods

2.1. Study setting

In this cross-sectional survey conducted during 2015, clinical presentations and chest radiograms of multiple trauma patients referring to 6 educational hospitals in Iran were evaluated. The study protocol was approved by the ethics committee of Tehran University of Medical Sciences. All the researchers were committed to the principles of Helsinki's Declaration and informed written consent was obtained from all the included patients or their legal guardians.

2.2. Participants

Hemodynamically stable multiple trauma patients aged over 15 years were included through a consecutive sampling method. Exclusion criteria included being the subject of penetrating traumas and not willing to participate. These criteria were based on the study carried out by Forouzanfar et al.¹⁰

2.3. Data collection

Prospectively gathered data included information about demographic characteristics of the patients (age, gender, trauma mechanism) and findings of their physical examinations. Data were collected by the emergency medicine specialists. Subsequently chest radiograms were obtained from the subjects in two views of anteroposterior and lateral.

Physical examination of the patients further focused on presence of painful distracting injuries, altered mental status (GCS < 15), tachypnea, pain or tenderness in thoracic region, dyspnea, abrasion or wound caused by the trauma in chest region, deformity of the thorax, tenderness in upper abdominal region, crepitation in palpation, diminished pulmonary sounds, and subcutaneous emphysema.

Final diagnosis of thoracic injury was made based on chest radiography. Computed tomography scan (CT scan) was performed in 3.2% of patients, which were demanded by other specialists such as surgeons or internal medicine specialists. In these patients, final diagnosis was based on CT scan findings. Chest radiograms were interpreted by another emergency medicine specialist, blinded to the survey's objectives. Verification of these interpretations was done for 3% of the radiograms by a radiologist. The inter-tater agreement was calculated to be 98%. Abnormal findings of the chest-X rays (CXR) included pneumothorax, hemothorax, pulmonary contusion, pneumomediastinum, mediastinal widening, subcutaneous emphysema, and fractures of the ribs, sternum, clavicles and scapulas.

2.4. Statistical analysis

Sample size calculation was based on a 6.5% prevalence of thoracic lesions in multiple trauma patients¹² considering a confidence interval of 95% (α = 0.05), a power of 90% (β = 0.1), and a maximum error of 1.5% (d = 0.015). The minimum number of patients required as the sample population was estimated to be 1038 subjects.

STATA software version 11.0 was used for data analysis. Qualitative variables are presented as frequencies and percentages and quantitative factors are presented as means and standard deviations. Validation of the model was performed using the methods proposed by recent review articles.^{13, 14} These methods assess the discriminatory power, calibration of a predictive model, or a combination of these two criteria. The discriminatory power was evaluated through calculating area under the curve of the receiver operating characteristic curve (ROC) and presenting the result as sensitivity, specificity and likelihood ratios. Based on TIRC cut off (presence of at least one criteria),¹⁰ data were presented as true positive (true prediction of presence of thoracic injury in CXR), true negative (true prediction of absence of thoracic injury in CXR), false positive (false prediction of presence of thoracic injury in CXR), and false negative (false prediction of absence of thoracic injury in CXR).

General calibration of the model was also evaluated through drawing a calibration plot, in which the numbers of observed and predicted CXR findings per decile of the linear predictor of the TIRC model were compared. In this plot, the perfect calibration is the reference line with an intercept of zero and a slope of 1. The overall performance was eventually assessed via the brier score in order to evaluate predictive accuracy and predictive reliability of the model.

3. Results

3.1. Baseline characteristics

Data were collected from a total of 2905 patients (73.17% were male; the mean age was 33.53 ± 15.42 years). The most prevalent trauma mechanisms were pedestrian (37.81%), car accidents (31.23%), motorcycle accidents (12.34%) and falling (14.03%). 312 patients (10.74%) presented with some levels of altered mental status, 732 subjects (25.20%) had thoracic pain, chest tenderness was found in 581 patients (20.0%) and 451 cases had thoracic skin abrasions (Table 1).

Table 1.

Baseline characteristics of studied patients.

Variable	N (%)
Age (years)
<60	2650 (91.22)
≥60	255 (8.78)
Gender
Male	2125 (73.17)
Female	779 (26.83)
Mechanism of traumaa
Motorcycle accident	358 (12.34)
Pedestrian	1097 (37.81)
Car accident	906 (31.23)
Falls	407 (14.03)
Other	133 (4.58)
Glasgow Coma Scale
15	2593 (89.26)
Less than 15	312 (10.74)
Dyspnea
Yes	458 (15.77)
No	2447 (84.23)
Distracting pain
Yes	1162 (40.0)
No	1743 (60.0)
Thoracic skin abrasion
Yes	451 (15.52)
No	2454 (84.48)
Chest deformity
Yes	50 (1.72)
No	2855 (98.28)
Chest wall tenderness
Yes	581 (20.0)
No	2324 (80.0)
Crepitation
Yes	122 (4.20)
No	2783 (95.8)
Abdominal tenderness
Yes	444 (15.28)
No	2461 (84.72)
Decrease in pulmonary sounds
Yes	172 (5.92)
No	2733 (94.08)
Chest wall pain
Yes	732 (25.20)
No	2173 (74.80)

^aMotorcycle accident: including motorcycle–car accident and motorcycle–motorcycle accident. Pedestrian: including accident of pedestrian with motorcycle of car. Car accident: including car–car accident and car rollover.

3.2. Discrimination

Area under the ROC curve of the TIRC model in predicting thoracic injuries was 0.93 (95%CI: 0.93–0.94) (Fig. 1). Table 2 presents the sensitivity and specificity of this rule out criteria in different cut-off points. Considering the presence of at least one of the criteria proposed by TIRC model resulted in 435 true positive, 1671 true negative and 799 false positive cases. No cases were false negative. Accordingly, the sensitivity and specificity were calculated to be 100 (98.91–100) and 67.65 (65.76–69.45), respectively. Positive and negative likelihood ratios for the TIRC model were also found to be 3.09 and 0.

Fig. 1.

Area under the curve of the TIRC model.

Table 2.

Sensitivity, specificity, positive likelihood ratio (LR+) and negative likelihood ratio (LR−) of the thoracic injury rule out criteria (TIRC) at various cut-off points.

Cut point	Sensitivity	Specificity	LR+	LR−
≥0	100.0	0.0	1.0	–
≥0.5	100.0	67.7	3.1	0.0
≥1	100.0	71.0	3.4	0.0
≥2	99.1	77.3	4.4	0.0
≥3	77.7	85.7	5.4	0.3
≥4	73.1	90.9	8.0	0.3
≥5	61.4	94.0	10.2	0.4
≥6	55.2	96.6	16.0	0.5
≥7	54.3	97.4	20.9	0.5
≥8	49.7	97.7	21.1	0.5
≥9	43.9	99.6	120.5	0.6
≥10	39.5	99.7	122.1	0.6
≥11	31.3	99.7	96.5	0.7
≥12	17.5	99.7	61.6	0.8
≥13	9.9	100.0	244.2	0.9
≥15	6.0	100.0	–	0.9
≥16	0.5	100.0	–	1.0

3.3. Calibration

Fig. 2 presents the calibration plot for the TIRC model and shows the observed and predicted CXR positive findings per decile of the linear predictor of the model. The perfect calibration is the reference line with an intercept of zero and a slope of 1, which is indicated by a continuous black line in Fig. 2. The TIRC calibration plot (scatter plot) had an intercept equal to 0.08 (95%CI: 0.07–0.09) and a slope of 1.19 (95%CI: 1.15–1.24). As can be seen the TIRC model showed a close adherence to the reference line, which indicates that the model is perfect in predicting the presence or absence of a lesion in CXRs.

Fig. 2.

The calibration plot for the TIRC model.

3.4. Overall performance

Brier score for the TIRC model was 0.07 and its scaled reliability was calculated to be 0.002. These figures are indicative of the high predictive accuracy and reliability of the model.

4. Discussion

The findings of this study confirm the external validation, good discrimination, and good calibration of the TIRC model in screening the multiple trauma patients for obtaining chest radiograms. The present survey was the first to evaluate the validity of the TIRC model in a multi-center setting, while its derivation study was conducted on 2600 subjects referring to a single center.¹⁰ Therefore, generalization of the model was considered as one of the limitations in their study¹⁰ till the present survey found a similar validity for the model through a multi-center approach.

In comparison, both the NEXUS chest and the TIRC model have acceptable sensitivities in detection of thoracic traumatic injuries. In the NEXUS chest validation study, Rodriguez et al. reported its sensitivity to be 98.8%.¹¹ The present survey found a sensitivity of 100% for the TIRC model. The present survey found a sensitivity of 100% for the TIRC model. Therefore, it seems that obtaining CXR is not necessary in stable multiple blunt trauma patients, who are conscious, under 60 years, or have no decrease in pulmonary sounds, dyspnea, thoracic skin abrasions, and crepitation. Although the sample size of this survey is far less than the sample population included the study of Rodriguez et al.; however, the very fitting calibration of the TIRC model presented in this study indicates that the model can perfectly detect the presence or absence of injuries.

Few studies have aimed to develop scoring systems for detection of blunt thoracic injuries. In the most important one, Bokhari et al. stated that patients with blunt traumas to the thoracic region, who are hemodynamically stable and their physical examinations are normal there is no need for chest radiography. In the rest of their study these researchers show that abnormal breath sounds in pulmonary auscultation, pain or tenderness of the chest wall and tachypnea have sensitivities of 100%, 57.1% and 42.8% in detection of hemopneumothorax, respectively. Although the negative predictive values of the two latter factors are more than 99%.¹⁵ Wisbach et al. also showed that in hemodynamically stable patients with normal physical examination of the thorax, chest radiography is unnecessary. This study declared that chest radiography should only be performed, if an evident clinical indication is present, similar to the way that cervical and pelvic radiographies are approached.¹² On the other hand, Dubinsky and Low believed that clinical presentation of the patient is not a suitable prognostic factor for detection of thoracic injuries in blunt traumas. However, most of the injuries in their study were rib fractures, and none of the patients had hemopneumothorax; therefore, their results might be subject to selection bias.¹⁶ Based on the results of the present study and other surveys on this matter, it can be concluded that in most cases scoring systems can be utilized in order to avoid unnecessary radiographies. In this regard, TIRC as an accurate scoring system is recommended to be used to order CXR for multiple trauma patients.

It should also be kept in mind that overall the diagnostic yield of chest radiography is not very high in detection of thoracic traumatic injuries. In this regard, the 5 meta-analyses performed by the authors of this survey showed that ultrasonography has higher sensitivity and specificity in detection of pneumothorax, hemothorax, pulmonary contusion, fracture, and pleural effusion compared to chest radiography.^{17, 18, 19, 20} Accordingly, the application of chest radiography in detection of thoracic injuries might be better re-evaluated and replaced with ultrasonography. However, studies with larger sample populations are needed to make decisions regarding this issue.

The large sample population and the multi-center setting can be pointed out as strengths of the present study. The number of included patients was approximately three times the minimum number required for the survey, which warrants the power of the study. Moreover, the included subjects are from four different cities that confirms that the results could be generalized to the total population. However, presence of selection bias is not fully avoidable. Furthermore, the event of radiograms having been interpreted by multiple physicians rather than a single one was another limitation of this study. Nevertheless, in order to verify these interpretations, 5% of the CXRs were evaluated and interpreted by a blinded radiologist for the second time. The inter-rater reliability was calculated to be 98%, which was congruent with the results of other similar surveys confirming the validity of CXR interpretations made by emergency medicine specialists.²¹ Finally, the presence of occult pneumothorax in our study was unavoidable, because not all of our patients received CT scan.

5. Conclusion

The findings confirmed the external validation, good discrimination, and good calibration of the TIRC model in screening of multiple trauma patents for chest radiography. The present survey was the first to validate the TIRC model in a multi-center setting that warrants its applicability in all the patients with thoracic traumatic injuries.

Author contributions

Study design and concerting: S.S., M.Y., M.H., V.R.-M.; data gathering: S.A., N.M., A.M.J., A.T.; data analysis: M.H., M.B., P.Gh., S.S.; drafting: M.Y., M.B., P.Gh.; other authors critically revised the manuscript.

Funding

This research has been supported by Tehran University of Medical Sciences & Health Services grant number: 93-02-38-25619.

Conflicts of interest

The authors have none to declare.