Trauma Indices for Prediction of Acute Respiratory Distress Syndrome

Abstract

Background

A myriad of trauma indices have been validated to predict probability of trauma survival. We aimed to compare the performance of commonly used indices for the development of the Acute Respiratory Distress Syndrome (ARDS).

Materials and Methods

Historic, observational cohort study of 27,385 consecutive patients admitted to a statewide referral trauma center between July 11, 2003 and October 31, 2011. A validated algorithm was adapted to identify patients with ARDS. Each trauma index was evaluated in logistic regression using the area under the receiver operating characteristic curve.

Results

The case-rate for ARDS development was 5.8% (1,594). The receiver operating characteristics for ISS had the best discrimination and had an area under the curve of 0.88 (95% CI 0.87–0.89). Glasgow coma score (0.71, 95% CI 0.70–0.73), A Severity Characterization of Trauma (0.86, 95% CI 0.85–0.87), Revised Trauma Score (0.71, 95% CI 0.70–0.72) and thorax Abbreviated Injury Score (0.73, 95% CI 0.72–0.74) performed worse (p<0.001) and Trauma and Injury Severity Score (0.88, 95% CI 0.87–0.88) performed equivocally (p=0.51) in comparison to ISS. Using a cutoff point ISS greater than or equal to 16, sensitivity and specificity were 84.9% (95% CI 83.0%–86.6%) and 75.6% (95% CI 75.1%–76.2%), respectively.

Conclusions

Among commonly used trauma indices, ISS has superior or equivocal discriminative ability for development of ARDS. A cutoff point of ISS greater than or equal to 16 provided good sensitivity and specificity. The use of ISS greater than or equal to 16 is a simple method to evaluate ARDS in trauma epidemiology and outcomes research.

1.1 INTRODUCTION

Traditional prediction models for death after injury have incorporated injury severity using anatomical scores such as the abbreviated injury score (AIS) and injury severity score (ISS) [1,2]. Criticisms of these trauma scoring systems include the inability to account for multiple injuries and the underscoring of certain body regions [3]. Subsequently, composite indices have been designed, incorporating physiologic parameters to injury severity, such as the Glasgow Coma Score (GCS) [4]. Trauma and injury severity score (TRISS) and A Severity Characterization of Trauma (ASCOT) are among these and are able to produce reproducible results that are superior to scores solely based on injury in predicting probability of survival after trauma [5,6].

Acute respiratory distress syndrome (ARDS) is an organ dysfunction occurring after severe trauma, which is associated with increased morbidity and mortality [7]. Case-rates for ARDS development have ranged between 5–10% [7–10] for all trauma patients, risk increasing with higher ISS score [11]. Prediction models for ARDS have been evaluated in medical patients [12] but few studies stratified by blunt injuries have been investigated in the prediction of ARDS in injured patients [8,11,13]. These trauma prediction models were performed in single center cohorts, incorporating multiple clinical risk factors and anatomical trauma scores. Most require chronic health measurements rarely performed at the bedside, in addition to blood transfusion data [14]. Previously validated prediction models for in-hospital death after trauma such as ISS, Revised Trauma Score (RTS), TRISS and ASCOT have not been evaluated in the context of ARDS development [5].

We aimed to compare the performance of commonly used anatomic, physiologic, and composite indices for the development of ARDS. In a large cohort of patients from a statewide referral trauma center with a detailed registry, we identified ARDS patients in accordance with the Berlin Definition [15]. We hypothesized that trauma indices possess discriminative ability for predicting development of ARDS, similar to their performance in predicting death.

2.1 MATERIALS AND METHODS

2.2 Setting and Design

The R. Adams Cowley Shock Trauma Center (STC) at the University of Maryland Medical is a free-standing, adult trauma center and a major healthcare resource in the State of Maryland for over three decades [16]. The center has over 5,000 trauma encounters annually with coverage of over 6 million people from urban to rural regions. All patients admitted to the trauma center are recorded in the institution’s trauma registry. Trauma indices listed in Table 1 are routinely calculated and recorded into the registry by dedicated and trained injury coders.

Table 1.

Trauma Outcome Scores

NAME OF SCORE	RANGE	TYPE OF SCORING SYSTEM	PARAMETER
Glasgow Coma Score	Ordinal Scale: 3–15	Physiologic	Best verbal, motor, eye response on admission to trauma center
Revised Trauma Score	Ordinal Scale: 0–7.84	Physiologic	First set of respiratory rate, systolic blood pressure, Glasgow Coma Score
Abbreviated Injury Score of Thorax	Ordinal Scale: 1–6	Anatomic	Threat to life associated with chest injury using ICD-9 coding
Injury Severity Score	Ordinal Scale: 0–75	Anatomic	Three most severely injured body regions squared
Trauma Injury Severity Score	Interval probability of survival: 0–100%	Combined (Anatomic + Physiologic)	Revised trauma score, age, age units, injury mechanism
A Severity Characterization of Trauma	Interval probability of survival: 0–100%	Combined (Anatomic + Physiologic)	Abbreviated injury score, revised trauma score, age, age units, injury mechanism

2.3 Population

Chest radiographs were completed in over 92% of the trauma encounters during initial evaluation at STC. Chest radiograph reports were retrieved in 28,682/38,609 (74.2%) patients through an electronic query using medical record numbers (MRN) from the trauma registry matched with the University of Maryland Clinical Data Repository. Patients without chest radiograph reports were excluded because this precluded ARDS evaluation. These unlinked records resulting from inconsistencies between administrative databases and trauma registries have been previously described [17]. Patients missing any of the trauma indices listed in Table 1 accounted for 3.0% (1297) and were also excluded from analysis. The final analysis cohort consisted of 27,385 patients. Characteristics (demographics, injury mechanism, and outcomes) of the excluded cohort are found in Supplemental A.

2.4 Acute Respiratory Distress Syndrome Definition

An internally and externally validated automated electronic screening tool [18,19] was adapted to identify patients with ARDS [15]. In the study of Level I trauma patients, the screening tool demonstrated 87% sensitivity (95% Confidence Interval (CI) 82%–92%) and 89% specificity (95% CI 85%–93%) [18]. In our cohort, ARDS was identified using the Berlin Definition [15] with cases occurring within seven days of admission. The ARDS cases had trauma as their primary risk factor with the assumption that hydrostatic edema was not the primary cause of respiratory failure. Qualifying chest radiograph reports and qualifying partial pressure of oxygen in arterial blood (PaO2)/ fractional inspired oxygen (FIO2) ratios were identified within 24 hours of each other in mechanically ventilated patients receiving a minimum positive end expiratory pressure of 5 mm H₂O. Administrative data were used to identify patients receiving mechanical ventilation. The overall accuracy of using administrative data was 93% from a random sampling of 461 patients (1.7%) which is consistent with prior reports from our institution [20]. The protocol for the automated screening system is displayed in Figure 1.

Figure 1.

Acute Respiratory Distress Syndrome Identification Protocol

Protocol Protocol for automated screening system to identify patients with acute respiratory distress syndrome

Chart review (M.A.) was performed in 3,890 patients with 8,179 chest radiographs that were inconclusive for ARDS. In these patients, an additional 480 (12.3%) met ARDS criteria. Chart review was also performed in 925 patients missing PaO2 or FIO2 data. Of these, an additional 401 patients (43.4%) met ARDS criteria. PaO2 data were not available in 25 patients; for these, a validated measure of the pulse oximetry SpO2/FIO2 ratio was used instead [21].

2.4 Data Analysis

Our primary outcome was prediction/discrimination of ARDS development. We identified ARDS occurring within seven days of trauma admission. Trauma indices evaluated were the following: ISS, TRISS, RTS, ASCOT, admission Glasgow Coma Score (GCS), and Abbreviated Injury Score (AIS) of thorax (Table 1). These indices comprised of anatomic severity scores, physiologic severity scores, and the combination of both. These scoring systems have previously been validated for trauma death [22] and are also commonly recorded and published in North American Trauma Registries [5, 23–26]. The AIS for thorax was evaluated in our analysis because we showed, previously, that approximately two-thirds of trauma-ARDS patients had thorax AIS ≥3 [7], correlating with greater than a 20% case rate of pulmonary contusions, and an association with ARDS development [11].

Continuous variables were evaluated as medians with interquartile ranges, and analyzed using either Wilcoxon-Mann-Whitney or Kruskal Wallis non-parametric tests. Proportions were analyzed using a Chi-Square test. The performance of each trauma index in predicting ARDS development was evaluated in logistic regression using the area under the receiver operating characteristic curve (ROC). We quantified model discrimination using the area under the ROC curve [27]. Logistic regression model performance was measured with likelihood ratio tests. All the models showed adequate performance with the likelihood ratio having a p<0.0001. The nonparameteric approach of DeLong et al. [28] was used to compare the ROC curves against the reference trauma index with the best area under the ROC. Adjusted p values were used for the effect of multiple comparisons using Bonferroni’s method. Finally, we evaluated injury severity score over a range of cutoff points and calculated the corresponding sensitivity, specificity, and positive/negative likelihood ratios.

Analysis was performed using SAS Version 9.1.3 (SAS Institute, Cary, NC). Approval of this study with a waiver of consent and Health Insurance Portability and Accountability Act authorization was provided by the institutional review board at the University of Maryland..

3.1 RESULTS

The analysis cohort comprised of 27,385 patients. Excluded patients were younger and had a greater proportion of non-white race. Comparisons between the excluded cohort (11,224) and final analysis cohort (27,385) are shown in Supplemental A. A large proportion of patients in the excluded cohort encountered penetrating trauma or were discharged home. In the analysis cohort, the proportion of patients who received mechanical ventilation was 22.4% (6,139) and the case-rate for ARDS development was 5.8% (1,594). The mean time to ARDS development was 1.50±1.9 days, and patients with ARDS required five-fold more days of mechanical ventilation than the remainder cohort receiving mechanical ventilation (p<0.001) (Table 2). In consideration for isolated pulmonary contusions on admission chest radiograph biasing towards a shorter mean time, we excluded patients with chest traumas that were serious or worse (Thorax AIS ≥3) and found similar results to our entire cohort in mean time to ARDS development at 1.6±1.8 days. Characteristics associated with ARDS development included older age, male sex, white race, blood alcohol concentration, and blunt trauma. Patients with ARDS had a more severe thorax injury score than patients alive without ARDS and patients dead without ARDS (p<0.001). ARDS patients also had longer hospital length of stay and the majority (1080, 67.8%) required acute care after discharge with few (207, 13.0%) being discharged home. The in-hospital case-fatality rate was 3.8% for all trauma encounters and 18.5% for patients with ARDS.

Table 2.

Patient Characteristics and Outcomes by Acute Respiratory Distress Syndrome group

Variables		Total (27,385)	Alive without ARDS (25,168)	ARDS (1,594)	Dead without ARDS (623)	P-value
Characteristics
Age (years)		37 (24–51)	36 (23–50)	46 (27–61)	49 (26–75)	<0.001
Sex (Male)		19,174 (70.0)	17,489 (69.5)	1,249 (78.4)	436 (70.0)	<0.001
Race (White)		16,770 (61.2)	15,252 (60.6)	1,099 (68.9)	419 (66.6)	<0.001
Blood alcohol content (>0 mg/dL)		7,585 (27.7)	6,965 (27.7)	468 (29.4)	152 (24.4)	0.002
Injury Mechanism
Blunt		23,792 (86.9)	21,911 (87.1)	1,391 (87.3)	490 (78.7)
Penetrating		3,373 (12.3)	3,065 (12.2)	179 (11.2)	129 (20.7)
Crush		197 (0.7)	169 (0.6)	24 (1.5)	4 (0.6)	<0.001
Burn		7 (0.0)	7 (0.0)	0	0
Ingestion		1 (0.0)	1 (0.0)	0	0
Other		15 (0.1)	15 (0.1)	0	0
Injury Scores
ISS		9 (4–17)	6 (4–14)	27 (19–36)	34 (26–45)	<0.001
Thorax AIS		0 (0–1)	0 (0–1)	3 (0–4)	2 (0–4)	<0.001
ASCOT		0.99 (0.97–0.99)	0.99 (0.98–.099)	0.88 (0.53–0.97)	0.20 (0.05–0.53)	<0.001
TRISS		0.99 (0.97–1.00)	0.99 (0.98–1.00)	0.89 (0.66–0.96)	0.35 (0.11–0.71)	<0.001
RTS		7.8 (7.8-7.8)	7.8 (7.8-7.8)	7.6 (5.9–7.8)	4.1 (3–10)	<0.001
Admission GCS		15 (15-15)	15 (15-15)	14 (7–15)	4 (3–10)	<0.001
Disposition
Length of Stay (Days)		0.9 (0.3–4.1)	0.7 (0.2–3.5)	17.3 (10.0–27.5)	0.7 (0.2–3.0)	<0.001
Mechanical Ventilation (Days) N= 6,139		2 (1–9)	2 (1–4)	11 (2–20)	2 (1–4)	<0.001
Home		21,099 (77.1)	20,892 (83.0)	207 (13.0)	0
Acute Care		5,200 (19.0)	4,120 (16.4)	1,080 (67.8)	0
Chronic Carea		89 (0.2)	82 (0.3)	7 (0.4)	0	<0.001
Other		79 (0.3)	74 (0.3)	5 (0.3)	0
Death		918 (3.4)	0	295 (18.5)	623 (100)

^aChronic care constitutes skilled nursing facility and long term acute care.

Continuous values were presented as medians with interquartile ranges, and categorical values were presented as numbers with %.

ISS = injury severity score; AIS = abbreviated injury score; ASCOT = a severity characterization of trauma; TRISS = trauma injury severity score; RTS = revised trauma score; GCS = Glasgow Coma Score

Severe injury with an ISS ≥16 accounted for 28.0% (7,650) of the cohort. Among patients with ARDS development, 85.6% (1389) had severe injury and 60% (955) of these patients had serious chest trauma (AIS thorax ≥3). For ARDS development, the ROC for ISS had the best discrimination and showed an area under the curve of 0.88 (95% CI 0.87–0.89) (Figure 2). Comparisons of injury scores from Table 1 were made in reference to ISS. GCS (0.71, 95% CI 0.70–0.73), ASCOT (0.86, 95% CI 0.85–0.87), RTS (0.71, 95% CI 0.70–0.72) and thorax AIS (0.73, 95% CI 0.72–0.74) performed worse in comparison to ISS (p<0.001) and TRISS (0.88, 95% CI 0.87–0.88) performed equivocally when compared to ISS (p=0.51). ROC contrast estimates against ISS with standard errors are shown in Table 3. The optimal trade off between sensitivity and specific occurred at ISS ≥16 for ARDS development with a positive likelihood ratio of 3.5 (95% CI 3.6-3.6) (Table 4). Using a cutoff point ISS ≥16 for ARDS development, sensitivity and specificity were 84.9% (95% CI 83.0%–86.6%) and 75.6% (95% CI 75.1%–76.2%), respectively.

Figure 2.

Receiver operating characteristic comparing physiologic, anatomic, and composite trauma indices

ROC = receiver operating characteristic; ISS = injury severity score; AIS = abbreviated injury score; ASCOT = a severity characterization of trauma; TRISS = trauma injury severity score; RTS = revised trauma score; GCS = glasgow coma score

Table 3.

Cutoff points for Injury Severity Score in development of Acute Respiratory Distress Syndrome

Injury Severity Score	Sensitivity (%) (95% CI)	Specificity (%) (95% CI)	Likelihood Ratio (+) (95% CI)	Likelihood Ratio (−) (95% CI)
≥ 3	99.4 (98.9–99.8)	20.2 (19.7–20.7)	1.3 (1.2–1.3)	0.0 (0.0-0.0)
≥ 5	98.6 (98.0–99.0)	30.7 (30.2–31.3)	1.4 (1.4–1.4)	0.1 (0.0–0.1)
≥ 10	94.1 (93.0–95.3)	60.6 (60.0–61.2)	2.4 (2.3–2.5)	0.1 (0.1-0.1)
≥ 14	87.6 (86.0–89.3)	71.1 (70.5–71.6)	3.0 (2.9–3.1)	0.2 (0.2-0.2)
≥ 16	84.9 (83.0–86.6)	75.6 (75.1–76.2)	3.5 (3.3–3.6)	0.2 (0.2–0.3)
≥ 18	76.3 (74.2–78.4)	81.8 (81.3–82.2)	4.2 (4.0–4.4)	0.3 (0.3-0.3)
≥ 20	73.6 (71.4–75.8)	84.1 (83.6–84.5)	4.6 (4.4–4.9)	0.3 (0.3-0.3)
≥ 25	61.9 (59.5–64.3)	89.5 (89.2–89.9)	5.9 (5.5–6.4)	0.4 (0.4–0.5)
≥ 33	38.3 (35.9–40.8)	95.3 (95.0–95.5)	8.1 (7.2–9.1)	0.7 (0.6–0.7)

Table 4.

Receiver operating characteristic contrast estimation and testing results against Injury Severity Score

Contrast	Estimate	Standard Error	95% Confidence Interval	P-value
GCS - ISS	−0.16	0.0068	(−0.18, −0.15)	<.0001
Thorax AIS - ISS	−0.15	0.0059	(−0.16, −0.14)	<.0001
ASCOT - ISS	−0.02	0.0035	(−0.02, −0.01)	<.0001
TRISS - ISS	−0.002	0.0032	(−0.01, 0.00)	0.5047
RTS - ISS	−0.17	0.0067	(−0.18, −0.15)	<.0001

ISS = injury severity score; AIS = abbreviated injury score; ASCOT = a severity characterization of trauma; TRISS = trauma injury severity score; RTS = revised trauma score; GCS = Glasgow Coma Score

To further evaluate the discriminative ability of ISS, we stratified the analysis to mechanism of injury. In the evaluation of blunt injuries only, we showed nearly identical results to the whole cohort with area under the ROC for ISS at 0.88 (95% CI 0.88–0.89). This had superior discriminative ability than the other trauma indices except TRISS (p=0.28). For penetrating injuries, the ROC for ISS showed an area under the curve of 0.83 (95% CI 0.80–0.85) and continued to perform superior to GCS, RTS, ASCOT, TRISS, and thorax AIS after adjustment for multiple comparisons.

We identified 47.4% (N=756/1594) of the chest radiographs in ARDS cases occurred in the initial trauma assessment (defined as chest radiographs ordered in the first day of trauma admission). Patients with at least a serious chest injury (Thorax AIS ≥3) comprised 63.5% (N=276) of this subgroup which likely reflects more pulmonary contusions. In sensitivity analysis we evaluated the 756 cases with ARDS in the first day of trauma admission as non-ARDS cases because of possible misclassification due to interobserver variability between pulmonary contusions with or without ARDS on chest radiograph. In this analysis, a similar ROC AUC for all the trauma indices was identified (Supplemental B). ISS AUC was 0.86 (95% CI 0.85–0.87) and was superior to the other indices except TRISS (performed equivalent with p=0.33).

4.1 DISCUSSION

Prior prediction models for discriminating ARDS in trauma patients incorporate composite indices with multiple clinical and laboratory factors [8, 11, 13]. These include the acute physiology and chronic health evaluation score (APACHE) and blood transfusion requirements, which require additional calculations at the bedside and are infrequently used by clinicians. In comparing commonly used trauma indices for triage and mortality prediction, ISS proved to be the best for predicting ARDS development. Serious chest trauma makes a strong contribution to ARDS development [29]. While the majority of ARDS patients had a serious chest injury, it was the more comprehensive system, ISS, which performed better than the directed thorax AIS. Even in sensitivity analysis addressing misclassification bias from another cause of respiratory failure, ISS had a similar ROC AUC and performed superior or equivalent to other trauma indices. This likely represents the complexity and heterogeneity of the syndrome from both direct and indirect pulmonary risk factors [30]. RTS, a physiology-based score, performed the worst. This further supports the superior predictive value of anatomy-based metrics for ARDS development in trauma patients.

Although ISS is the most frequently studied tool amongst the trauma indices [5], it is criticized for missing multiple injuries to a single body region and evaluating each body region equally [2,3]. However, physiology-based scoring can be problematic as well; GCS is less reliable in settings of alcohol and drug use [31–33], and RTS is a tool developed for out-of-hospital triage [35]. We showed both GCS and RTS did not perform as well as ISS for ARDS development. Composite indices incorporating both anatomic and physiologic metrics were developed to have superior discriminative ability than ISS for predicting survival after trauma. Among composite indices, TRISS incorporates physiologic parameters, demographics, and injury mechanism whereas ASCOT is similar but adds AIS. TRISS and ASCOT were previously shown to perform similarly in predicting probability of survival after trauma and both performed superior to ISS [4,5,35–37]. However, for predicting ARDS development, ISS was superior to all the indices, except TRISS in which it was equivalent (blunt injury) or superior (penetrating injury). The composite indices are calculated with a logistic regression model which adds much complexity to the bedside evaluation whereas ISS is a calculation that performs superior or equivocal and can be performed more easily to evaluate ARDS.

Prior studies identified ISS cutoff points between 10 and 25 as a risk factor for ARDS development but did not delineate the best cutoff point [10,11,38,39]. We found that an ISS ≥16 is the best cutoff point to dichotomize risk for ARDS development, with a sensitivity and specificity of 84.9% and 75.6%, respectively. A positive likelihood ratio of 3.5 suggests that patients developing ARDS are almost four-times as likely to have an ISS ≥16 early after traumatic injury. Future studies will need to validate ISS externally for ARDS development at other trauma centers.

Use of a robust and efficient screening tool may improve research into the care of injured patients at risk for the development of ARDS. These patients differ in clinical outcomes and biomarker profiles compared to those with non-traumatic risk factors [40]. Currently, trials assessing methods to prevent ARDS development utilize ARDS prediction models validated from non-trauma cohorts of at risk patients [12, 41, 42]. As more patients survive their injuries, as evidenced by a case-fatality of 3.8% in our cohort, ARDS has become a common contributor to morbidity after survivorship in trauma [23, 43]. Over two-thirds of the ARDS survivors from STC required acute care after discharge, a major contributor to increased healthcare utilization and morbidity [44, 45]. Easily applicable methods for ARDS research are important to identify early determinants of the syndrome to facilitate investigation of prevention or early, targeted therapies. Using an ISS ≥16 based on our results is an easily identifiable criterion for patients at risk for ARDS development so institution of effective measures such as a lung-protective ventilation strategy can occur in a targeted population [46].

4.2 Limitations

The retrospective nature of this study introduces biases which cannot be controlled such as misclassification of ARDS. We showed a case-rate for ARDS development and ARDS fatality similar to previous trauma studies [8,9,11,40]. More patients in the excluded cohort were discharged home or died and had a shorter length of stay (upper quartile at 1.2 days) which likely did not allow time for ARDS development and would not have changed our results. Our mean time of 1.5 days to ARDS development was also reported in a large prospective cohort of trauma patients at another center [47]. Furthermore, we did not distinguish between isolated pulmonary contusions without ARDS and pulmonary contusions with ARDS which may have introduced a misclassification bias. Nevertheless, pulmonary contusions are a well-described etiology for ARDS and contusion sizes of 24% or greater strongly predict ARDS development [48]. We did not evaluate other risk factors for ARDS (i.e., aspiration of gastric contents, plasma-derived ARDS biomarkers) as these were not documented in our registry. Although these factors may improve discriminatory performance, they introduce subjectivity or measures not typically obtained clinically. Furthermore, this is a single center experience which makes it difficult to determine whether the findings are generalizable to other trauma centers. However, the patient cohort at STC is representative of the aggregate of trauma patients treated at other trauma centers in the United States [24]. Future directions, in addition to validating this tool in ARDS trauma cohorts prospectively, include comparing the performance of ISS to other ARDS prediction models [12].

4.3 CONCLUSION

Among trauma indices, ISS better discriminates than other anatomic and physiologic trauma indices for development of ARDS. A cutoff point of ISS ≥16 provided good sensitivity and specificity for ARDS development. Although TRISS was equivalent to ISS in predicting ARDS development, TRISS requires more variables in a multiple regression analysis which makes ISS a more straightforward tool that can be applied with greater ease by investigators and providers. This is the first study demonstrating the performance of ISS against other commonly used trauma indices, and identifying the test characteristics for a severe injury cutoff with ISS≥16 for ARDS development. Our results provide important criteria for future studies targeting the population of trauma patients at risk for ARDS development. External validation is needed before its application in epidemiology and outcomes research.