Validation of the Denver Emergency Department Trauma Organ Failure Score to Predict Post-Injury Multiple Organ Failure

Abstract

Background

Early recognition of trauma patients at risk for multiple organ failure (MOF) is important to reduce the morbidity and mortality associated with MOF. The objective of the study was to externally validate the Denver Emergency Department (ED) Trauma Organ Failure (TOF) Score, a six-item instrument that includes age, intubation, hematocrit, systolic blood pressure, blood urea nitrogen, and white blood cell count, which was designed to predict the development of MOF within seven days of hospitalization.

Study Design

Prospective multi-center study of adult trauma patients between November, 2011 and March, 2013. The primary outcome was development of MOF within seven days of hospitalization, assessed using the Sequential Organ Failure Assessment Score. Hierarchical logistic regression analysis was performed to determine associations between Denver ED TOF Score and MOF. Discrimination was assessed and quantified using a receiver operating characteristics (ROC) curve. The predictive accuracy of the Denver ED TOF score was compared to attending emergency physician estimation of the likelihood of MOF.

Results

We included 2,072 patients with a median age of 46 (IQR 30-61) years and 68% male. The median injury severity score was 9 (IQR 5-17) and 88% of patients had blunt mechanisms. Among participants, 1,024 patients (49%) were admitted to the intensive care unit, and 77 (4%) died. MOF occurred in 120 (6%; 95% CI: 5%-7%) patients and of these, 37 (31%; 95% CI: 23%-40%) died. The area under the ROC curve for the Denver ED TOF Score prediction of MOF was 0.89 (95% CI 0.86-0.91) and for physician estimation of the likelihood of MOF was 0.78 (95% CI 0.73-0.83).

Conclusions

The Denver ED TOF Score predicts the development of MOF within seven days of hospitalization. Its predictive accuracy outperformed attending emergency physician estimation of the risk of MOF.

INTRODUCTION

Trauma is the leading cause of death in the United States in people 1 to 44 years of age and accounts for 41 million emergency department (ED) visits per year.^1-3 Post-injury multiple organ failure (MOF) is common among seriously injured trauma patients and organ failure is identified in one or more systems in 29% of all trauma patients.⁴ Despite recent improvements in trauma resuscitation strategies, MOF remains the leading cause of morbidity and mortality among those who survive the immediate post-injury period.^5-19 Given the morbidity, mortality, and healthcare costs associated with MOF after traumatic injury, early recognition of this syndrome rather than delayed treatment is important.^10,12

Limited data are available to predict MOF in trauma in the early post-injury period. Previous models predicting MOF in trauma incorporate variables obtained 24-48 hours after injury when opportunities for early intervention have passed. These predictive models were analyzed by Cryer et al. who noted that MOF was already present in a high proportion of trauma patients when these models were implemented to predict whether it would occur.¹¹ Given this, investigators recommend that risk stratification for the development of MOF in trauma patients begin on the day of injury.^10,11

Because the morbidity and mortality associated with MOF is so high, early identification of MOF is still the best strategy to improve healthcare outcomes.¹² Early identification for patients at risk for MOF is especially important to provide appropriate hemodynamic monitoring and importantly to facilitate triage of these patients to a higher level of trauma care with a cadre of trauma specialists. Since there are limited resources in regionalized trauma care, a clinical prediction tool that successfully identifies patients at risk for MOF in need of specialized healthcare resources and transfer to a higher level of trauma care is especially important. This type of tool may facilitate goal directed resuscitation and timely triage to successfully reduce the morbidity associated with MOF thereby improving trauma outcomes and reducing healthcare costs.

We recently derived and internally validated a clinical prediction instrument at the Denver Health Medical Center, the Denver ED Trauma Organ Failure (TOF) Score,²⁰ which uses clinical and laboratory data within four hours of ED arrival in adult trauma patients to predict the development of MOF within seven days of hospitalization (Table 1). The objective of this study was to externally validate the Denver ED TOF Score²⁰ and assess its performance in a prospective multi-center cohort of trauma patients at three Level 1 trauma centers. To determine the utility of the Denver ED TOF Score in clinical practice, we also compared the predictive accuracy of the Denver ED TOF Score to physician judgment. We hypothesized that the Denver ED TOF Score would: (1) accurately predict the development of MOF within seven days of hospitalization in a heterogeneous trauma population; and (2) be more sensitive and specific for predicting the development of MOF within seven days of hospitalization than the clinical judgment of attending emergency physicians.

Table 1.

The Denver Emergency Department Trauma Organ Failure Score for Prediction of Multiple Organ Failure in Adult Trauma Patients

Predictor	Score*
Age ≥65 y	1
Emergent intubation†	3
Hematocrit <20%	2
Hematocrit ≥20% and <35%	1
Emergency department systolic blood pressure <90 mmHg	1
Blood urea nitrogen ≥30 mg/dL	1
White blood cell count ≥20,000/ μL	1

^*Score was derived from Vogel JA, et al.²⁰

^†Emergent intubation defined as intubation in the prehospital or emergency department settings.

METHODS

Study Design and Setting

This was a prospective multicenter cohort study performed at three urban, level 1 trauma centers: Denver Health Medical Center (DHMC) in Denver, Colorado; Oregon Health and Science University (OHSU) in Portland, Oregon; and University of California Davis Medical Center (UCDMC), in Sacramento, California. The DHMC patients included in this external validation study were a different cohort than patients enrolled for the internal validation of the Denver ED TOF Score at DHMC. The study was approved by the Institutional Review Boards at each site.

Selection of Participants

Adult trauma patients (≥18 years of age) who were admitted to the hospital for a trauma-related cause were included in this study. The enrollment periods at DHMC, OHSU, and UCDMC were from November 1, 2011 to July 3, 2012; February 12, 2012 to November 20, 2012, and January 20, 2012 to March 17, 2013, respectively.

Patients were excluded from this study if they were: (1) <18 years of age; (2) died in the ED; or (3) transferred from another hospital. The DHMC and OHSU enrolled consecutive adult trauma patients in the study, and UCDMC enrolled a convenience sample of adult trauma patients. The convenience sample at UCDMC was enrolled during time periods when research assistants or the co-investigators were available in the ED to identify and enroll patients in the study.

Data Collection

Data were collected prospectively by the treating emergency physician using a structured paper-based data collection instrument. Data collected included: (1) demographics (age, gender, and race/ethnicity); (2) date of visit; (3) initial vital signs; (4) lowest systolic blood pressure and highest heart rate within four hours of arrival or until the patient left the ED, whichever occurred first; (5) intubation in the prehospital (PH) or ED setting; (6) complete blood count; (7) basic metabolic panel; and (8) attending emergency physician estimation of the likelihood of developing MOF using a continuous probability scale. If more than one laboratory value was available within four hours of hospital arrival, the highest value of white blood cell count and blood urea nitrogen were used, and the lowest value of the hematocrit were used to calculate the Denver ED TOF Score. All data collection instruments were reviewed and retrospective chart abstraction was performed to obtain results for those objective data that were incomplete. The attending emergency physician estimation of the likelihood of developing MOF on a continuous scale (0-100) was collected prospectively in the ED without their knowledge of the Denver ED TOF Score or the purpose of the study. These data were considered missing if the emergency physician did not prospectively estimate the likelihood of MOF, and no attempt was made to retrospectively ascertain these estimates. In the majority of cases, the attending emergency physician had access to the clinical and laboratory data that comprise the Denver ED TOF Score when estimating the likelihood of the development of MOF within seven days of hospitalization. However, in unstable trauma patients who were taken emergently to the operating room complete laboratory data would not have been available when the attending emergency physician estimated the likelihood of MOF. Additional data for all patients were obtained from each center's trauma registry and included: trauma mechanism (blunt or penetrating), injury type (motor vehicle accident, auto-pedestrian accident, motorcycle accident, gunshot, stabbing, bicycle accident, assault, fall, other); date and time of presentation to the ED; date and time of admission to the hospital; ED disposition (ward, observation unit, ICU), procedures performed during hospitalization, Abbreviated Injury Scale (AIS) for each body region (head, face, neck, chest, abdomen/pelvis, extremities, skin); Injury Severity Score; ICD-9 external cause of injury code; and final diagnoses (ICD-9 codes); hospitalization characteristics (ICU LOS, hospital LOS, and survival to hospital discharge); and discharge destination.

Outcome Measures

The primary outcome, development of MOF within seven days of hospitalization, was assessed using the Sequential Organ Failure Assessment (SOFA) Score.²¹ The SOFA Score is a valid measure of MOF and has been used to reliably assess the occurrence of organ dysfunction and failure in trauma patients.^22-23 The SOFA Score consists of measures of function across six organ systems.²¹ The lowest values of the PaO₂/FiO₂ ratio, platelet count, GCS score, mean arterial pressure, and urine output, and the highest values of total bilirubin, adrenergic agent use, and creatinine were recorded for each 24-hour period and used in the score calculation. In instances in which an ICU patient did not have a platelet count, total bilirubin, or creatinine obtained in a 24-hour period, the value for these absent labs was assumed to be normal in the calculation of the SOFA score. The occurrence of multiple organ dysfunction (MOD) was defined as a SOFA score ≥1 in two or more systems and MOF was defined as a SOFA Score ≥3 in two or more organ systems on any of the first seven hospital days.²¹ Secondary outcomes collected for this study included MOD, ICU admission, ICU LOS, hospital LOS, and in-hospital mortality.

The medical records of all study patients admitted to the ICU at any time during the seven days following admission were identified and systematically reviewed by trained abstractors to obtain the variables for the SOFA score. SOFA score data were assessed on admission and in 24-hour intervals for the first seven days following hospitalization, and were recorded in an electronic closed-response data collection instrument (Microsoft Access [Microsoft Corporation, Redmond, WA] or REDCap [Research Electronic Data Capture).²⁴ The abstractors were trained to use the data collection instrument to systematically abstract each endpoint using standardized medical record abstraction methodology.²⁵ At DHMC and OHSU, the abstractors for the study were emergency medicine resident physicians. The abstractors at UCDMC were trained research assistants with experience abstracting data for research. The abstractors at OHSU and UCDMC were not blinded to the purpose of the study. The abstractors at DHMC were blinded to the purpose of the study, and 10 percent of the charts were re-abstracted to verify reliability of the abstraction process. Inter-rater reliability was assessed to determine the overall agreement on the individual components of the SOFA score for each of the first seven hospital days; the median raw agreement was 0.75 (interquartile range [IQR] 0.61-0.83), the median intra-class correlation for continuous variables was 0.85 (IQR 0.81-0.90), and the median kappa for categorical variables was 0.78 (IQR 0.36–0.89). It is possible that errors in abstraction may have impacted computation of the SOFA Score.

Data Management and Statistical Analyses

Data were transferred electronically from the trauma registry into separate electronic spreadsheets (Microsoft Excel, Microsoft Corporation, Redmond, WA). SOFA Score data were collected and managed using a Microsoft Access Database or REDCap tools. REDCap is a secure, web-based application designed to support data capture for research studies, providing: (1) an intuitive interface for validated data entry; (2) audit trails for tracking data manipulation and export procedures; (3) automated export procedures for seamless data downloads to common statistical packages; and (4) procedures for importing data from external sources.²⁴ Each electronic file was then transferred into native SAS format and concatenated. All analyses were performed using SAS Version 9.3 (SAS Institute, Inc., Cary, NC) or Stata Version 12.0 (Stata Corporation, College Station, TX).

Continuous data are reported as medians with IQRs and categorical data, including sensitivity and specificity estimates, are reported as percentages with 95% CIs. Hierarchical logistic regression analysis was performed to estimate the association between the Denver ED TOF Score and the primary outcome while accounting for clustering at the level of the institution. Hosmer Lemeshow goodness-of-fit statistic for the model was Chi square 7.38, p> 0.06. Discrimination was assessed using a receiver operating characteristics (ROC) curve and quantified using the area under the ROCs.

Sample Size Estimation

Using data from the study in which the Denver ED TOF Score was derived,²⁰ we estimated 0.9% of all patients with a Denver ED TOF Score of 0-1 (low risk) would develop MOF, 12% of those with a Score of 2-3 (moderate risk) would develop MOF, and 40% of those with a Score of ≥4 (high risk) would develop MOF. We estimated requiring 100 total patients who developed MOF in order to include sufficient numbers in each risk group. We estimated the prevalence of MOF in the overall cohort to be 5%, thus requiring 2,000 total patients to provide sufficient statistical power by achieving statistical separation (defined by 95% confidence intervals) for the prevalence of MOF between each of the three risk groups. The power was at least 80% for the study.

RESULTS

During the study period, 2,072 patients met criteria for inclusion and represent our study sample. The median age was 46 (IQR 30-61) years and 68% were male. The median injury severity score was 9 (IQR 5-17) and 88% of patients had blunt mechanisms. A complete description of the demographics and characteristics of the study sample is presented in Table 2.

Table 2.

Demographics and Characteristics (n=2,072)

Variable	Data	Missing, n (%)
Total, n	2,072
Median age, y, n (IQR)	46 (30 – 61)	1 (0)
Male, n (%)	1,405 (68)	3 (0)
Race, n (%)*		4 (1)
American Indian	20 (1)
Asian	44 (2)
Black	135 (7)
Other	45 (2)
Pacific Islander	4 (0)
Unknown	89 (4)
White	1,721 (84)
Hispanic ethnicity, n (%)	352 (17)
Mechanism, blunt, n (%)	1,824 (88)	4 (0)
Injury type, n (%)		3 (0)
Assault	168 (8)
Auto-pedestrian crash	112 (5)
Bicycle crash	121 (6)
Fall	623 (30)
Gunshot	80 (4)
Motor vehicle crash	537 (26)
Motorcycle crash	137 (7)
Other	185 (9)
Stabbing	109 (5)
Emergent intubation, prehospital or ED, n (%)	281 (14)	2 (0)
Median Injury Severity Score, n (IQR)	9 (5 – 17)	3 (0)

^*In the race category, other represents a race other than those represented by the standard categories of white, black, American Indian, Asian, and Pacific Islander.

ED, emergency department; IQR, interquartile range; MOF, multiple organ failure.

Of the 2,072 patients, 1,024 (49%) were admitted to ICU with a median ICU LOS of 1.7 (IQR 1.0-3.5) days, and of these 743 (73%) had a ICU stay of ≥24 hours, and the median ICU LOS for these patients was 2.6 (1.6-5.0) days. The median hospital LOS was 2.8 (IQR 1.2-6.7) days. One hundred twenty (6%, 95% CI 5%-7%) subjects developed MOF, and of these, 37 (31%, 95% CI 23%-40%) died. A complete description of the outcomes for the study sample is presented in Table 3.

Table 3.

Study Outcomes (n=2,072)

Variable	Data	95% CI
In-hospital mortality, n (%)	77 (4)	3, 5
MOF* within 7 d, n (%)	120 (6)	5, 7
Mortality in patients with MOF, n (%)	37 (31)	23, 40
MOD† within 7 d, n (%)	553 (27)	25, 29
Mortality in patients with MOD, n (%)	43 (8)	6, 10
ICU admission, n (%)	1,024 (49)	47, 52
ICU length of stay; d, median (IQR)	1.7 (1.0 – 3.5)	1.6, 1.9
Hospital length of stay; d, median (IQR)	2.8 (1.2 – 6.7)	2.6, 3.0

^*MOF defined as a score of 3 or higher in 2 or more systems as defined within the SOFA score.

^†MOD defined as a score of 1 or higher in 2 or more systems as defined within the SOFA score.

ICU, intensive care unit; IQR, interquartile range; MOD, multiple organ dysfunction; MOF, multiple organ failure; SOFA score, Sequential Organ Failure Assessment Score.

The accuracy of the Denver ED TOF Score for the prediction of MOF is presented in Table 4. These data suggest that the risk level for MOF using the Denver ED TOF Score could be considered low, moderate, and high with Denver ED TOF Score of 0-1, 2-3, and ≥4, respectively. Application of the Denver ED TOF score to the study sample demonstrated an approximately exponential increase in the prevalence of MOF with increasing risk score values (Figure 1). For each one point increase in the Denver ED TOF Score, the odds of developing MOF increased 2.5 times (95% CI 2.2-2.9).

Table 4.

Accuracy of the Denver Emergency Department Trauma Organ Failure Score for the Prediction of Multiple Organ Failure (n=2,072)

Denver ED TOF Score	MOF		Sensitivity, %	95% CI	Specificity, %	95% CI
	Yes	No
>0	111	838	92.5	86.2, 96.5	57.1	54.8, 59.3
0	9	1114
>1	100	393	83.3	75.4, 89.5	80.9	79.1, 82.6
≤1	20	1579
>2	92	239	76.7	68.1, 83.9	87.8	86.2, 89.2
≤2	28	1713
>3	56	99	46.7	37.5, 56.0	94.9	93.9, 95.9
≤3	64	1853
>4	19	36	15.8	9.8, 23.6	98.2	97.5, 98.7
≤4	101	1916
>5	8	5	6.7	2.9, 12.7	99.7	99.4, 99.9
≤5	112	1947
>6	2	1	1.6	0.2, 5.8	100.0	99.7, 100.0
≤6	118	1951
>7	0	0	0.0	0.0, 3.0	100.0	99.8, 100.0
≤7	120	1952

CI, confidence interval; ED, emergency department; MOF, multiple organ failure; TOF, Trauma Organ Failure.

Figure 1.

Observed probability of multiple organ failure for the Denver Emergency Department Trauma Organ Failure Score. Bars represent 95% confidence intervals.

Of the 2,072 patients, 1,717 (83%) had an attending emergency physician estimation of MOF. Among the patients with a physician estimate of MOF, those considered at high risk for MOF (physician estimate of ≥70%), 8.9% were admitted to the ICU. Among patients with a moderate to high risk of MOF by the Denver ED TOF Score, 36.7% were admitted to the ICU. Patients with physician estimate of MOF were compared to those without an attending physician estimate of MOF and were similar except a slightly higher proportion of in-hospital mortality was noted in patients who had a physician estimate of MOF. The area under the ROC curve for the development of MOF for the Denver ED TOF Score was 0.89 (95% CI 0.87-0.91), whereas the area under the ROC curve for the physician estimation of the likelihood of the development of MOF was 0.78 (95% CI 0.73-0.83) (Figure 2). The median physician estimation of MOF for the cohort using a continuous probability scale ranging from 0-100 was 5 (IQR 1-15). The percent of patients with MOF compared to the attending emergency physician estimation of the likelihood of MOF is depicted in Figure 3.

Figure 2.

Discrimination of the Denver Emergency Department Trauma Organ Failure Score (Area under the curve = 0.89, 95% confidence interval 0.86-0.91), and physician prediction of likelihood of multiple organ failure (Area under the curve = 0.78 (95% confidence interval 0.73-0.83).

Figure 3.

Comparison of patients with multiple organ failure to attending emergency physician estimation of the likelihood of multiple organ failure (N=1,717).

DISCUSSION

The goal of our study was to validate the Denver ED TOF Score,²⁰ which uses data within four hours of ED arrival to predict MOF within seven days of hospitalization. In this study we used a prospective multicenter approach to externally validate the Denver ED TOF Score in a heterogeneous adult trauma population and to determine the utility of the Denver ED TOF Score in clinical practice. We compared the predictive accuracy of the score to physician judgment, and found the Denver ED TOF Score to be a valid predictor of MOF and with a higher sensitivity than physician judgment. This suggests that the Denver ED TOF Score may be another tool to help physicians stratify adult trauma patients at risk for MOF.

Despite significant advances in trauma care and systems since MOF was initially described over 40 years ago,^26-27 MOF remains a leading cause of morbidity and late mortality after trauma in the severely injured trauma patient.^5-19 It is estimated that trauma patients with failure in three organ systems have a mortality of approximately 67% and in patients with failure in four or more organ systems, mortality approaches 100%.⁵ In a recent longitudinal, multicenter study of blunt trauma patients with hemorrhagic shock without head injury, Sauaia et al evaluated trends in MOF. They noted that the high morbidity and mortality prevalences among those who developed MOF persisted despite advances in trauma care.¹² In addition, MOF is associated with significant ICU and healthcare resource use and costs.^{12-13,15,17-19} The cost of caring for trauma patients with MOF is more than double the cost of treating trauma patients who do not have MOF.¹² Sauaia et al recently contended that MOF after traumatic injury “remains a resource-intensive, morbid, and lethal condition”.¹²

MOF after trauma occurs in a bimodal distribution, with onset of MOF “early” and “late” in the post-injury course.^9,12,28-30 The development of late MOF was believed to be associated with a “second-hit” after injury such as infection which resulted in a dysfunctional, systemic amplified inflammatory state and subsequent MOF.^9,12,28-30 Recent investigations demonstrate that MOF remains common after traumatic injury but that the timing of MOF occurrence has significantly changed. Currently, MOF primarily occurs early in the post-injury course, and there has been a significant decrease in late-onset MOF.^12,16 This is likely due, in part, to improved resuscitative strategies such as low tidal volume ventilation, strict glycemic control, limited crystalloid resuscitation, and restrictive autologous red cell transfusions.^12,16 Given the fact that MOF is now demonstrated to most frequently occur early in the post-injury phase, development and validation of a prediction tool that can accurately and promptly identify patients at-risk for MOF early in their clinical course is critically important.

Early identification of those at low- and high-risk for MOF may improve delivery of care, limit healthcare costs, and enhance healthcare resource utilization. The Denver ED TOF score uses clinical data readily available within four hours of ED arrival. The score includes age, need for emergent intubation, initial hematocrit, lowest ED SBP within four hours, blood urea nitrogen, and white blood cell count (Table 1).²⁰ Clinical prediction tools such as the Denver ED TOF Score have been suggested to supplement clinical judgment in the identification of a diagnoses and potential therapeutic course of action, which may result in improved patient care and decreased costs.^31-32 The Denver ED TOF Score may serve as a particularly useful instrument to identify early those patients at risk of developing MOF, thereby facilitating aggressive, goal-directed resuscitation and triage of these critically ill patients. Patients who are identified as high risk of MOF should be admitted to the intensive care unit to facilitate ongoing hemodynamic monitoring and focused resuscitation. By monitoring these patients closely and pursuing aggressive resuscitation, it may be possible to avert or reduce the development of MOF. Additionally, the Denver ED TOF Score would be a useful tool to identify early those patients at risk for MOF to facilitate timely secondary triage to a higher level of trauma care. The transfer of patients at high risk for MOF to a center with a cadre of trauma specialists may prevent or reduce the occurrence of MOF thereby improving trauma outcomes and reducing healthcare costs.

We previously derived and internally validated the Denver ED TOF Score using a retrospective cohort at the DHMC. In the derivation study, discrimination of the Denver ED TOF Score was determined using the area under the ROC curve and was 0.92 (95% CI 0.90-0.94).²⁰ In this prospective, multi-center external validation of the score, the area under the ROC curve for the Denver ED TOF Score was 0.89 (95% CI 0.86-0.91). In general, when compared to internal validation, the accuracy of a prediction tool is commonly less when externally validated; however, the performance of the Denver ED TOF Score in this multicenter external validation that included a heterogeneous adult trauma population, supports its use as a valid measure to identify trauma patients in the ED who are at risk for the development of MOF within seven days of hospitalization.

Based upon our findings, we recommend that the risk level for MOF using the Denver ED TOF Score be considered low, moderate, or high with Denver ED TOF Scores of 0-1, 2-3, or ≥4, respectively. Low-risk patients are not without risk for MOF and ongoing monitoring of their hemodynamic status and potential progression of organ dysfunction after injury may still be indicated. However, patients with a score of 0 are at lowest risk (MOF prevalence = 0.8%) and may not require subsequent monitoring. Patients considered moderate- or high-risk by the Denver ED TOF score would benefit from advanced trauma care and close monitoring at a tertiary care trauma center. Goal-directed resuscitation of these higher-risk patients will be especially important to help reduce the occurrence of MOF and to ultimately improve outcomes for these critically ill patients.

Before integrating a clinical prediction instrument into practice, it must undergo rigorous external assessment to demonstrate its validity and generalizability.^31-32 We anticipate the next steps for the Denver ED TOF score will include external validation of the score in Level 2 trauma centers and non-trauma centers to assess the practical applicability of the score in clinical practice. A clinical trial of use of the score in a regional fashion to identify those patients at high-risk for MOF and who may benefit from transfer to a higher level of care as compared to current regional triage criteria may be helpful. This would include an evaluation of the outcomes for patients who were transferred based upon the score and a comparison of the score to physician gestalt for the need for transfer to a higher level of trauma care. These studies would advance our understanding of how the Denver ED TOF Score may be used in clinical practice and in integrated trauma systems.

The study has certain limitations. Excluding patients who were discharged from the ED may have introduced selection bias. It is possible, albeit highly unlikely, that some of the patients who were discharged from the ED developed MOF.

Physicians may have been biased in their estimation of patient risk for the development of MOF. Some of the emergency physicians may have had knowledge of the Denver ED TOF Score, which may have impacted their prediction estimate. To minimize this possibility, we collected more variables than necessary to calculate the Denver ED TOF Score on the data collection instrument, no information about the score was available in the ED, and the original derivation work had not yet been published at the time the external validation study was conducted²⁰ In addition, we collected attending emergency physician estimate risk of MOF for the study; prediction capabilities of other types of physicians (e.g., surgeons, anesthesiologists, residents, etc.) may be different.

The physician estimation of MOF was collected prospectively, and was considered missing if no estimate was provided by the attending emergency physician. In this study, a significant proportion (83%) of patients had a physician estimation of MOF. If the physicians who did not complete the estimate of MOF were either significantly better or worse at predicting the likelihood of MOF than the remaining cohort of emergency physicians, this could have impacted our findings related to predictive accuracy of the Denver ED TOF Score as compared to the physician estimation of the likelihood of MOF.

Chart abstraction has inherent limitations. To minimize these limitations, we implemented rigorous training and abstraction methods. We used trained abstractors to obtain the variables necessary to derive the SOFA score, and conducted focused training of the abstractors prior to initiation of abstraction. We completed re-abstraction of 10% of the charts at DHMC and compared these results to verify agreement of the abstraction process, which demonstrated good inter-rater reliability between abstractors.

CONCLUSIONS

The Denver ED TOF Score predicts the development of MOF within seven days of hospitalization in a heterogeneous adult trauma population. The predictive accuracy of the Denver ED TOF outperformed attending emergency physician estimation of the risk of MOF. The Denver ED TOF Score is a useful tool to identify patients early in the post-injury phase who may be at risk for the development of MOF. Early identification of these patients may facilitate aggressive, goal-directed resuscitation and enhanced resource allocation to improve outcomes in trauma.

ABBREVIATIONS

ICD

International Classification of Diseases

ICU

Intensive Care Unit

IQR

Interquartile Range

LOS

length of stay

CI

confidence interval