Predicting mortality in trauma patients - A retrospective comparison of the performance of six scoring systems applied to polytrauma patients from the emergency centre of a South African central hospital

Abstract

Introduction

Over 90% of trauma-related deaths worldwide, ensue in low- and middle-income countries. Multiple useful trauma scoring systems have been devised. Although validated in high-income countries, they cannot always be replicated in resource-limited countries. This study compares six trauma scores to identify the best-suited system to use for polytrauma patients in a hospital in Pretoria, South Africa.

Methods

This is an observational retrospective analysis of polytrauma admissions from 1 July 2016 to 31 December 2016. Data collected from patients' records from the EC of Steve Biko Academic Hospital, was analysed using Stata Release 14. Outcomes were recorded as 30-day survival, ICU– and overall hospital LOS. Scores pertaining to patient mortality, were compared in terms of sensitivity, specificity, and cut-off points based on ROC curve. Finally, for LOS Pearson correlation analysis was used.

Results

At the best calculated mortality prediction cut-points for the scores, the sensitivities and specificities were respectively 87% and 68% for TRISS, 81% and 61% for ISS, RTS yielded 81% and 60%, while for REMS it was 61% and 69%. The SI and RSI (cut-points used in agreement with the literature) produced sensitivities 58% and only 48%, and specificities of 73% and 83%, respectively.

45(41,7%) patients required ICU admission. Though the ICU LOS best correlated with ISS(r = 0.2710), the ICU LOS correlation coefficient was weak for all trauma scores. None of the scores had a significant p value for hospital LOS.

Discussion

Among the scores compared, TRISS had the highest sensitivity and NPV for mortality prediction in this South African polytrauma population. ISS correlated best with ICU LOS. However, compared to developed countries, ROC analyses & predictability of these scores fare relatively worse, and no correlation was found with hospital LOS. Therefore, we conclude that further studies are needed to ascertain a more suitable system for resource-limited settings.

African relevance

• •Globally, over 90% of trauma-related deaths, transpire in low- and middle-income countries
• •Trauma scores are calculated based on standard parameters such as vital data and require no additional equipment. However, the scores can be valuable predictors of severity and can be used to direct scarce resources to appropriate patients
• •These scores have been validated in high income countries, but their performance isn't well researched in low- and middle-income countries

Introduction

Trauma is the leading cause of death in persons between the ages of 15 and 44 years [1]. Over 90% of trauma-related deaths occur in low- and middle-income countries (LMIC) and Africa has the highest trauma associated mortality rate of all continents [2]. In South Africa (SA), injury-related mortality is six times higher than the global average [3].

Evaluation of severely injured patients in emergency centers (ECs) must be efficient. Grading injury severity is vital, making scoring systems invaluable [4], [5].

Although multiple scores have been validated in high-income countries, they have not been well investigated in resource-limited settings [3], [4], [5], 6., [7], 8., [9], [10], [11], 12..

This study was conducted in a central hospital with an established trauma surgical service in Tshwane, Gauteng. The following scores were compared using a clinical record based retrospective methodology: shock index (SI), reversed shock index (RSI), injury severity score (ISS), revised trauma score (RTS), trauma and injury severity score (TRISS) and the rapid emergency medicine score (REMS).

The rationale behind researching these scores specifically, was to ascertain the validity of scoring systems developed specifically with the use of data from high income countries, in LMIC. Many of these scoring systems, especially TRISS, ISS and RTS, are considered to be the gold standard for the prediction of trauma outcomes in developed countries 13., [14], 15..However, these systems cannot always be replicated in resource-limited settings due to different patient characteristics, and also absent predictor variables. Hence, there is no literature consensus regarding interpretation and feasibility of such scores in developing countries [12].

The aims of this study were to identify the trauma-related scoring system that best predicted mortality (primary outcome), Intensive care unit (ICU) stay (secondary outcome) and hospital length of stay(LOS) (secondary outcome) in our setting.

Methods

An observational retrospective data analysis of all polytrauma admissions from 1 July 2016 to 31 December 2016 was performed. Although the data set is older, there is no reason to suspect that the data would not be valid. The patient profile of the study centre has not changed including drainage areas, overall patient numbers and categories of severity. Although trauma care is always evolving, the principles of care are still the same.

Approval of the Faculty of Health Sciences Research Ethics Committee of the University of Pretoria was obtained (Ref: 287/2017). Consent was obtained from the CEO and the heads of EC, critical care and trauma surgery. Participants remained anonymous at all times.

Polytrauma was defined as injuries scoring >2 points in ≥2 different AIS regions, together with ≥1 additional variable from the following physiologic parameters: hypotension (SBP ≤ 90 mmHg), GCS ≤ 8, acidosis (base excess (BE) ≤ −6.0), coagulopathy (international normalized ratio ≥ 1.4/partial thromboplastin time ≥ 40 s), and age (≥70 years) [16]. These were calculated based on EC records. After each analysis, six trauma scores were determined and calculated. A description of the individual scoring systems used in our study, can be found in Appendix 1.

The EC and trauma service of the hospital used for this study, is the referral center for a large area crossing interprovincial borders. It's the only hospital in the area providing specialist surgical services including trauma-, neuro-, and cardiothoracic surgery. This hospital was chosen for its high number of poly-trauma admissions.

Data was collected from patients' records. The hand-written admission register of all EC admissions for the period stated was obtained and all trauma cases were identified. The records of these trauma cases were then acquired from the file room and screened. Records of all trauma patients were obtained from the file room and screened. Records of polytrauma patients aged ≥18 years, presenting to the EC and managed by either the EC or trauma teams, were included. All records of patients who did not meet inclusion criteria, burns cases, drowning cases, and patients who sustained pathological fractures were excluded.

The clinical records were used to obtain data on demographics (age and gender), injury mechanisms, types of injuries, BP, MAP, HR, RR, SPO₂, GCS and BE (if assessed). By using this data, the principal investigator retrospectively determined the scores. The ISS was established through record observation while the other scores were calculated. For missing vital data, imputation was used to avoid information loss. Values that were not reported were assumed to be clinically unimportant and therefore likely to be within normal ranges. Deleting incomplete cases only gives reliable results when the missing values are random [17]. Missing data for BP, HR and RR was replaced by the median, and for SPO₂ and GCS was replaced by the mode of all cases without outstanding data. Median and mode were used as determined by the statistician, since the data found in this polytrauma subset of trauma, was skewed. Missing data points included BP (n = 1;1,2%), HR (n = 0), RR (n = 14; 16,7%), SPO2 (n = 14; 16.7%) and GCS (n = 2; 2.4%). Only one missing value per patient was tolerated.

Patient outcome was recorded as 30-day survival or leaving the hospital alive, ICU LOS and hospital LOS. Patients who refused treatment or were transferred to another hospital within 30 days were excluded from the study as no outcome data could be traced.

To compare the mortality prediction accuracy, data was analysed using Stata Release 14. Continuous variables were presented as means ± standard deviation (SD) and categorical values were analysed using Chi-square test. P < 0.05 indicated statistical significance and the confidence level of the study was kept at 95%. Discrimination was measured by receiver operating characteristic (ROC) curves and logistic regression analysis was performed. Scores were compared in terms of specificity, sensitivity, and cut-off points based on ROC curve, associated with patient mortality. The observed diagnostic cut points on the ROC curve were determined as the point with minimum distance from the left upper corner of the unit square, thereby ensuring the highest true positive and lowest false positive rate. Finally, Pearson correlation was used for analysis of overall hospital LOS and ICU LOS.

Results

A total of 1013 trauma files were recruited. One hundred and twenty seven patient-records met the “polytrauma” criteria. The reasons for additional exclusions are displayed in Fig. 1. One hundred and eight patients were enrolled in the study.

Fig. 1.

Flowchart included patients.

The mean age values of n = 108 patients were 36.5 ± 14.4 years. Ninety-three (86.1%) individuals were male and fifteen (13.9%) female. Blunt force trauma predominated (75.9%), with the most common causes of admission being road traffic injuries (63%) and assaults, totalling 33.3%. The mean trauma scores for all individuals are shown in Table 1. While 45 patients (41.7%) required ICU admission, 77 patients (71.3%) were alive 30 days after hospitalization (Table 1).

Table 1.

Baseline characteristics.

Age, mean (±SD)	36.5 (±14.4)
Gender
Male, n (%)	93 (86.1%)
Female, n (%)	15 (13.9%)
Mechanism of injury
Traffic accident, n (%)	68 (63.0%)
Gunshot wound, n (%)	13 (12.0%)
Stab wound, n (%)	11 (10.2%)
Assault (other), n (%)	12 (11.1%)
Fall, n (%)	4 (3.7%)
Blunt, n (%)	82 (75.9%)
Penetrating, n (%)	26 (24.1%)
Baseline Trauma Scores at the EC
SI, mean (±SD)	0.89 (±0.39)
RSI, mean (±SD)	1.28 (±0.45)
ISS, mean (±SD)	31.07 (±11.52)
RTS, mean (±SD)	6.28 (±1.76)
TRISS, mean (±SD)	76.48 (±26.58)
REMS, mean (±SD)	3.67 (±3.05)
30-day survival
Yes, n (%)	77 (71.3%)
SI, mean (±SD)	0.83 (±0.37)
RSI, mean (±SD)	1.37 (±0.42)
ISS, mean (±SD)	28.01 (± 9.05)
RTS, mean (±SD)	7.05 (±1.12)
TRISS, mean (±SD)	87.82 (±15.81)
REMS, mean (±SD)	3.39 (±2.59)
No, n (%)	31 (28.7%)
SI mean (±SD)	1,06 (±0.43)
RSI mean (±SD)	1.11(±0.46)
ISS mean (±SD)	38.83 (±13.98)
RTS mean (±SD)	5.09 (±1.88)
TRISS mean (±SD)	52.81 (±31.47)
REMS mean (±SD)	5.28 (±3.92)
ICU admission
No, n (%)	63 (58.3%)
Yes, n (%)	45 (41.7%)
Length of stay
ICU, mean days (±SD)	12.1 (±12.8)
ICU, median (IQR)	6.5 (3–16)
Hospital, mean days (±SD)	18.7 (±26.6)
Hospital, median (IQR)	8 (3−22)

SD = standard deviation; IQR = interquartile range; ICU = intensive care unit; SI = shock index; RSI = reverse shock index; ISS = injury severity score; RTS = revised trauma score; TRISS = trauma and injury severity score; REMS = rapid emergency medicine score.

Mortality

In this study 31 (28.7%) patients died within 30 days of hospital admission. Numerical values were used to calculate means and compare the survivor and mortality groups. Mean SI, RSI, ISS, RTS, TRISS and REMS of survivors and non-survivors are presented in Table 1. As seen, mean SI, ISS and REMS were higher for the mortality group, while the mean RSI, RTS and TRISS were higher for survivors. The differences in mean were statistically significant in all scores (p < 0.001).

The SI and RSI were evaluated differently from the other scoring systems and their cut-off points were used in agreement with the literature [7], 8., [9], [10], [11], 18., 19., 20.. At a cut-off of >0.9, SI had a sensitivity of 58% and specificity of 73% whereas RSI cut-off at <1, produced sensitivity of only 48% and specificity of 83%. The positive likelihood ratio (LR) for SI as well as RSI was 2.13 and 2.87, and the negative LR was 0.58 and 0.62 respectively. SI yielded a NPV of 81% with a PPV of 46%, while RSI had a NPV of 80% and a PPV of 53%.

AUROC curve using TRISS, ISS, RTS, and REMS for death prediction was 0.828, 0.755, 0.715 and 0.656 respectively, all statistically significant (Fig. 2). TRISS (82.8%) and REMS (65.6%) had the highest and lowest area under ROC (AUROC) for prediction of mortality respectively.

Fig. 2.

ROC curve and comparison of area under ROC of ISS, RTS, TRISS, and REMS in mortality prediction of adult trauma patients.

ROC = Receiver operating characteristic; ISS = Injury Severity Score; ARTS = Revised Trauma Score; TRISS = Trauma and Injury Severity Score; REMS = Rapid Emergency Medicine Score; CI = confidence interval; *Statistically significant for all scores.

The best cut-off point for predicting mortality in polytrauma patients was <12 for TRISS with sensitivity of 87% and specificity of 68%, while for ISS it was >32 with sensitivity of 81% and specificity of 61%. For RTS the best cut-off point was 8, yielding sensitivity of 81% and specificity of 60%, and for REMS it was found to be >5 with 61% sensitivity and 69% specificity (Table 2).

Table 2.

Comparison of sensitivity, specificity, positive and negative likelihood ratios, positive and negative predictive values and optimum cut-off points based on ROC in TRISS, ISS, RTS and REMS.

	-LR**	+ LR*	Sensitivity (%)	Specificity (%)	NPV	PPV	Cut-off
TRISS	0.1911	2.6826	87.10%	67.53%	92.85%	51.89%	<12
ISS	0.3171	2.0699	80.65%	61.04%	88.70%	45.44%	>32
RTS	0.3240	2.0031	80.65%	59.74%	88.49%	44.64%	<8
REMS	0.5596	2.0031	61.11%	69.49%	81.61%	44.64%	>5

* + LR: Positive likelihood ratio = sensitivity/1-specificity; **- LR: Negative likelihood ratio = 1-sensitivity/specificity. NPV: Negative predictive value; PPV: Positive predictive value; ROC: Receiver operating characteristic. TRISS: Trauma and Injury Severity Score; ISS: Injury Severity Score; RTS: Revised Trauma Score; REMS: Rapid Emergency Medicine Score.

Overall ICU and hospital LOS

The mean number of days spent in ICU were 12, while mean duration of hospitalization was ±19 days (Table 1).

ICU LOS correlated significantly with the ISS (positive), while the TRISS (negative) also approached statistical significance. The correlation coefficient for all the other scores were weak (Table 3).

Table 3.

Correlation between ICU and hospitalization duration and Trauma Scores.

Scores	SI	RSI	ISS	RTS	TRISS	REMS
ICU LOS	r = 0.0868	r = −0.0069	r = 0.2710	r = −0.1001	r = −0.1828	r = 0.2017
	p = 0.38	p = 0.94	p = 0.005*	p = 0.3051	p = 0.05**	p = 0.08
Hospital LOS	r = −0.0624	r = 0.0221	r = 0.1031	r = 0.0477	r = 0.0222	r = 0.0121
	p = 0.53	p = 0.82	p = 0.29	p = 0.63	p = 0.82	p = 0.92

* Statistical significant; ** Aproached statistical significance.

Although ISS had the best correlation with hospital LOS, none of the trauma scores had a significant p value and all the scores demonstrated weak correlation coefficients for this outcome (Table 3).

SI = shock index; RSI = reverse shock index; ISS = injury severity score; RTS = revised trauma score; TRISS = trauma and injury severity score; REMS = rapid emergency medicine score; r = correlation coefficient.

Discussion

In this study, we assessed anatomical (ISS), physiological (RTS, SI, RSI and REMS), and combined scoring systems (TRISS) among adult polytrauma patients to ascertain which score would be most effective in our hospital population.

We found a 29% 30-day mortality outcome for polytrauma patients in this study which is higher than the 20% mortality often reported in level I trauma centres of developed countries [21]. Possible explanations for this finding is that SA trauma mortality is generally higher than the universal rate [3]. The higher mortality rate is found in most LMIC, due to various cultural, socioeconomic, and environmental factors that places the population of these regions at a higher risk than those living in high income countries [22]. Our high mortality rate corresponds to a mortality rate of 26%, that was found in polytrauma patients, in a study done by da Costa et al. REF in Sao Paulo, Brazil [23].

When analysing the means of the SI and RSI with regards to 30-day survival vs. 30-day mortality, we found that the means did not differ that greatly. Overall, our analyses did not establish good sensitivity, specificity and predictive values for predicting mortality using SI/RSI. This is similar to a study by King et al. [24] who found that the SI had only a 37% sensitivity and 83% specificity to predict poor outcome (including mortality) in trauma patients. Moreover, we found no correlation between the SI/RSI, and ICU and/or hospital LOS. Our finding differs from three studies that found a RSI < 1.0 to be associated with increase in mortality, along with a lengthier ICU and hospital stay 18., 19., 20.. Granting these study groups were larger, it should be considered that vital sign based shock scores may be unsuitable in our population, supporting the findings of Barnes et al. [25] This may be due to a different spectrum of injuries, for example a higher incidence of community assault cases compared to high income countries, or pre-hospital delays, possibly attributable to a large patient-load serviced by understaffed emergency medical services (EMS) with limited capabilities, and longer response times due to disproportionate distribution of EMS and poorly resourced rural areas [26], in the SA setting. That being said, there were only 12 poly-trauma assault cases in our study – which may be a reflection of the trauma profile in our unit. All the other reasons would however apply to our patient population.

Evaluation of ROC curves indicated that ISS and REMS of survivors were significantly lower than for non-survivors (p < 0.0001 and p = 0.0198 respectively). Moreover, RTS (p < 0.0001) and TRISS (p < 0.0001) for survivors were significantly higher than non-survivors. These findings are similar to international studies [27], 28., [29], [30], 31..

The 30-day survival (mean = 88%) and mortality (mean = 53%) groups differed significantly by >30 points in the TRISS score. With the calculated cut-off point, TRISS had a favourable p-value, plus the highest NPV (93%) and sensitivity (87%). TRISS appeared to be the most suitable tool to rule out 30-day mortality in our population. This finding was similar to Eryilmaz et al. [29] and Yousefzadeh-Chabok et al. [30] who found TRISS to be a better predictor of mortality than ISS and RTS. TRISS has outperformed APACHE II in the polytrauma population [32], but has not yet been compared to its derivative, REMS. This study suggests that REMS is outdone by TRISS. Conversely, Demeteriades et al. [33] found a good overall prediction of the TRISS, but that it decreased sharply for severely injured patients. Our study did not support that finding. Deshmukh et al. [34] also found that TRISS was a poor predictor of survival in injured patients, contradicting our results.

In our study, the ISS and RTS had high NPVs (± 88%) and moderate sensitivities (± 80%) as predictors of 30-day mortality. At the determined cut-offs (ISS > 32 and RTS < 8), both scores ruled mortality in. Various studies have shown ISS [35], [36] and RTS [37], 38., [39] as accurate predictors of mortality. However, ISS cut-off points were lower and all trauma patients were included – which makes that population different to ours (polytrauma victims). Champion et al. [37] found high specificity and PPVs for these scores as opposed to our findings of high sensitivity and NPVs.

Although a significant difference in REMS means was found between survival and 30-day mortality groups, it was less predictive than the ISS, RTS and TRISS. Imhoff et al. [40] found REMS to be similar to RTS, and superior to ISS in mortality prediction. Lee et al. [41] also demonstrated REMS' superiority to ISS, which is different to our findings.

Only TRISS and ISS scores indicated a significant correlation with ICU LOS. Although the correlation was not powerful, ISS was the best score to predict the ICU LOS. This finding is similar to Li et al. [39] Comparatively, Orhon et al. [5] found no correlation between ISS and ICU LOS. This could be explained by the extent of injuries sustained – our study group had a much higher ISS (Mean ± SD of 38.83 ± 13.98 vs 24.37 ± 12.85) with a much higher ICU admission rate (41% vs 10%). The latter also found a significant correlation between hospital LOS and ISS, RTS and TRISS, whereas none of the trauma scores in our study demonstrated a statistically significant correlation with overall hospital LOS, again possibly attributed to an overall difference in our populations.

Our analysis had several limitations. The study was underpowered due to a low number of polytrauma outcomes found in the records that were enrolled for the time period. The hospital used in this study, still made use of paper-based record-keeping and had no trauma registry available, which would have made data-collection easier.

At the time the study was conducted African trauma scores such as the Kampala trauma score and the Malawi trauma score were not well recognized or widely used in the South African population yet. These scores were subsequently not included in our study but would be of importance in future studies.

Assumptions about absent vital data had to be made due to incomplete documentation. GCS in patients arriving to the EC already intubated or paralyzed may have been inaccurate which may have affected some of the scores. Although some patients received critical care in the EC, the time spent in the EC was not included in the ICU LOS, which may have influenced our findings.

The impact of pre-existing comorbidities on mortality and hospitalization was not included in the analysis and thus remains unclear. However, the studied demographic was predominantly younger adults, which would potentially limit the impact of comorbidities on our findings. This could be an important avenue of future research on this topic.

Conclusion

Our study showed that TRISS performed the best in predicting the primary outcome of mortality in our population of polytrauma patients. It also showed that the ISS correlated well with ICU LOS. These scores may therefore be important tools in our population.

When compared to developed countries, the predictability of these scores fared relatively worse, and no correlation was found with hospital LOS.Larger studies on big trauma registries are needed to confirm our findings on the best predicting scores for mortality and ICU LOS. Such studies may provide further clarity on our findings, which will improve the allocation of resources to the most appropriate patients and contribute to the management of polytrauma patients in similar settings. A prospective cohort study may also be a good option, since it would better counter the potential biases associated with a retrospective methodology.

Dissemination of results

The findings of this study have been shared with staff members at the data collection site through an informal discussion. We also plan to share the link to the article on various Emergency Medicine Pretoria social media platforms.

Authorship contribution statement

Authors contributed as follow to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; and drafting the work or revising it critically for important intellectual content: MM contributed 65%, AE 20% and MG 15%. All authors approved the version to be published and agreed to be accountable for all aspects of the work.

Declaration of competing interest

The authors declared no conflicts of interest.

Appendix A. Supplementary data

Supplementary material