Evaluating trauma center process performance in an integrated trauma system with registry data

Abstract

Background:

The evaluation of trauma center performance implies the use of indicators that evaluate clinical processes. Despite the availability of routinely collected clinical data in most trauma systems, quality improvement efforts are often limited to hospital-based audit of adverse patient outcomes.

Objective:

To identify and evaluate a series of process performance indicators (PPI) that can be calculated using routinely collected trauma registry data.

Materials and Methods:

PPI were identified using a review of published literature, trauma system documentation, and expert consensus. Data from the 59 trauma centers of the Quebec trauma system (1999, 2006; N = 99,444) were used to calculate estimates of conformity to each PPI for each trauma center. Outliers were identified by comparing each center to the global mean. PPI were evaluated in terms of discrimination (between-center variance), construct validity (correlation with designation level and patient volume), and forecasting (correlation over time).

Results:

Fifteen PPI were retained. Global proportions of conformity ranged between 6% for reduction of a major dislocation within 1 h and 97% for therapeutic laparotomy. Between-center variance was statistically significant for 13 PPI. Five PPI were significantly associated with designation level, 7 were associated with volume, and 11 were correlated over time.

Conclusion:

In our trauma system, results suggest that a series of 15 PPI supported by literature review or expert opinion can be calculated using routinely collected trauma registry data. We have provided evidence of their discrimination, construct validity, and forecasting properties. The between-center variance observed in this study highlights the importance of evaluating process performance in integrated trauma systems.

INTRODUCTION

With the exponential rise in health care costs, health care authorities worldwide are expressing the urgent need to obtain information on health care performance.[1] Evidence has suggested that the dissemination of data on performance can lead to improvements in the quality and efficiency of health care services.[2,3] The most widely used conceptual model for health care performance evaluation, proposed by Donabedian, describes performance according to three domains: Structure, process, and outcome.[4] Process indicators refer to clinical processes performed in the health care setting and should be relevant, reliable, accessible, and clear.[2] Accessibility implies that they can be evaluated at low cost with data that are easy to obtain on a routine basis.[2] According to the US Agency for Healthcare Quality, performance indicators should be evaluated using measures of discrimination, validity, and forecasting.[5]

Currently, trauma quality assurance activities vary greatly across trauma systems and are often limited to hospital-based audit of adverse patient outcomes despite the availability of clinical data in trauma registries.[6] In addition, while numerous process indicators have been suggested in the literature, many cannot be applied using registry data, and evidence of their validity is scarce.[7]

The objectives of this study were to (1) identify a series of process performance indicators (PPI) supported by literature review and/or expert consensus that can be calculated using trauma registry data and (2) evaluate selected PPI in terms of discrimination, construct validity, and forecasting.

MATERIALS AND METHODS

Identifying PPI

PPI were selected in three stages. First, a review of the literature was performed to identify which of these PPI were supported by literature published between 1985 and 2008 using PubMed, EmBase, CINAHL, Cochrane, and Proquest. Second, a review of websites of state/provincial authorities, professional associations, and trauma centers from the USA, Canada, and Australia was performed to identify PPI that were used in practice. Third, a multidisciplinary group of clinical experts serving as a steering committee for the Quebec provincial trauma system had already established 14 PPI they intended to follow. PPI were selected if they met any of the following three criteria: Supported by the literature, used by at least two systems without literature support, or suggested by the group of experts. PPI definitions were based on the literature. PPI definitions that varied across systems for the same indicator (e.g., delays to operate) were refined by the steering committee. PPI that could be measured using the Quebec Trauma Registry were then identified (and calculated for each of the 59 designated trauma centers.

Study data

This multicenter retrospective cohort study was based on the inclusive trauma system of the province of Quebec, Canada. The Quebec trauma system was instated in 1993 and involves regionalized care from urban level I trauma centers through to rural community hospitals. Trauma level designations are based on American College of Surgeons criteria and are revised periodically with on-site visits.[8] During the study period, the system included 6 level I (including 2 pediatric), 4 level II, 21 level III, and 28 level IV trauma centers. Standardized pre-hospital protocols ensure that major trauma cases are taken to these hospitals and standing agreements regulate between-center transfers within the system.

Data were drawn from the Quebec Trauma Registry, which is mandatory for all 59 designated trauma centers and includes all deaths following injury, intensive care unit admissions, hospital stays >2 days, and between-center transfers. The Province of Quebec, Canada, has universal health coverage, and linkage with hospital discharge data has shown that the trauma registry captures approximately 75% of all trauma patients who meet these inclusion criteria and more than 85% of the most severe pathologies (i.e., traumatic brain injuries, thoracic and abdominal injuries).[9] The trauma registry is based on the American College of Surgeons model. Registry data are extracted from patient files by medical archivists who use standardized coding protocols. Anatomic injury is coded with the Abbreviated Injury Scale (AIS) according to the guidelines published by the Association for the Advancement of Automotive Medicine.[10] The registry is centralized at the Quebec Ministry of Health and is subject to both systematic and periodic validation to identify and correct aberrant data values in all relevant data fields and to verify date and time chronology. Supervision by a data coordinator, yearly on-going training, an electronic forum of coding queries, and thrice-yearly meetings with key stakeholders (e.g., trauma physicians, researchers, administrators) are used to improve data reliability and validity.

This study was based on data collected during eight administrative years, i.e., between April 1998 and March 2006. Deaths on arrival and patients who arrived with no vital signs and expired within 30 min of arrival were considered inevitable in the context of trauma center evaluation and were thus excluded from the study population. Patients with an isolated hip fracture were also excluded because this pathology is commonly considered to be the consequence of a chronic disease rather than a traumatic event.[11] Hospital readmissions are not included in the trauma registry. If patients are entered twice for the same injury due to transfer, only information from the definitive acute care hospital was retained. Only 2% of patients in the study sample were included more than once in the registry for repeated traumatic events.

Implementing PPI

We first identified the patient population eligible for each PPI. Conformity to the PPI was then established for each patient in the eligible population. The observed proportion of conformity, calculated as the ratio of patients whose care conforms to the PPI divided by all patients eligible for the PPI, is subject to regression to the mean bias whereby it is more extreme than the “true” proportion, particularly for low-volume centers.[12] We therefore used shrinkage techniques that estimate the “true” proportion using information from all centers rather than just the center under evaluation – estimates are shrunk toward the global mean by a factor that is inversely proportional to the sample size.[13] These shrinkage estimates not only account for regression-to-the-mean bias, but also improve the precision of estimates for low-volume centers and account for inflation of the type I error due to multiple comparisons, which otherwise leads to the detection of too many outliers. Efron and Morris[13] argue that estimates derived through shrinkage are more suitable for policy making, for ordering (i.e., ranking), and for group comparisons (i.e., between-center comparisons) than conventional estimates and they have become standard for health care performance evaluation.[13] The “true” proportion of conformity to each PPI along with 95% confidence intervals (CI) for each of the 59 trauma centers was calculated using a random-intercept multilevel model. Conformity to each PPI is presented using a modified caterpillar plot where centers are ordered by designation level and volume to avoid ranking and to conserve the order of centers across PPI. Outliers are defined as trauma centers with 95% CI that exclude the global mean.

Evaluating PPI

PPI were evaluated using measures of discrimination, construct validity, and forecasting, according to US Agency for Healthcare Quality recommendations.[5] Discrimination was defined as the ability of PPI to differentiate performance and was evaluated using between-center variance estimates along with 95% CI. CI that exclude 0 are considered to be consistent with significant deviations across trauma centers.[14] Construct validity was defined as the degree of association between specific PPI and other indicators of quality and was assessed by evaluating the correlation between each indicator and two hospital characteristics that have been shown to be related to performance: (i) designation level and (ii) volume. Forecasting was defined as the ability of PPI to predict future performance and was assessed by evaluating the correlation within indicators over time; indicators derived from data collected in the first 4 years of the study were compared to the same indicators derived from data collected in the last 4 years. Correlation was assessed with Pearson's correlation coefficient and asymptotic 95% CI on “true” proportions (shrinkage estimates). These proportions were first transformed using a square-root arcsine transformation to normalize their distribution[15] and were then weighted with the median number of eligible patients across PPI for each center.

All analyses were performed with the SAS system (SAS Institute Inc., Cary, NC, USA; Version 9.2). Ethical approval was obtained from institutional ethics committees and the “Commision de l’accès à l’information du Québec.” The identity of trauma centers is not revealed to protect institutional confidentiality.

RESULTS

Identification of PPI

We identified 130 articles in peer-reviewed journals as well as 60 documents from websites of trauma centers, health authorities, and professional associations, which led to the identification of 137 PPI. After regrouping similar PPI, the list was reduced to 48. Applying the three selection criteria further reduced the list to 32 indicators (listed in the Appendix). Twenty-three were backed up by literature published in peer-reviewed journals, five were used by at least two trauma systems without literature support, and four were identified by the Quebec steering committee. Of the 32 PPI, 15 were obtainable from Quebec Trauma Registry data listed in Table 1.

Table 1.

Process performance indicators

Study population

The Quebec Trauma Registry contains information on 125,907 patients admitted between April 1998 and March 2006. The exclusion of deaths on arrival and deaths occurring within 30 min of arrival in patients without vital signs (n = 4837) and isolated hip fractures (n = 21,626) resulted in a sample of 99,444 patients from 59 centers available for analyses. Mean age (±standard deviation) was 48.5 years (±25.2), 30.7% of patients were 65 years of age or over, 58.4% were men, 20% had an Injury Severity Score (ISS) >15, and 29% were injured in a motor vehicle collision. Penetrating trauma represented only 4% of the study sample. Trauma center average yearly volume of patients respecting the study inclusion criteria was 218-1313 for level I centers, 341-605 for level II centers, 100-541 for level III centers, and 5-118 for level IV centers.

Overall conformity to PPI related to transfer of patients to level I/II trauma centers (PPI #1-#3) was high, whereas conformity to PPI associated with a delay of 1 h (#4, #8, #10) was low [Table 1]. A wide range of conformity across trauma centers (min-max) was observed for all PPI except #6, #8, and #10, and trauma center outliers were detected for all but two PPI.

Figures 1-3 show between-center performance for three PPI (three examples were selected as space constraints precluded presentation of all 15). Only level III/IV centers were evaluated for PPI #1 “transfer of moderate-severe traumatic brain injury patients to a level I/II center” and overall conformity for this PPI was over 80% [Figure 1]. Seven centers had higher than average conformity, whereas five centers had lower than average conformity. Overall conformity for PPI #5, securing patients’ airway in the Emergency Department (ED), was only 64% [Figure 2]. Eight of the 10 level I/II centers had higher than average conformity, whereas 3 out of 21 level III centers had lower than average conformity. Global conformity to PPI #12, ED stay <4 h, was only 49%. Seven of 10 level I/II centers had higher than average conformity, while 8 out of 21 level III centers had lower than average conformity [Figure 3].

Figure 1.

Percent conformity with PPI #1 “transfer of TBI patients with GCS ≤12 initially received in level III/IV center to a level I/II center”

Figure 3.

Percent conformity with PPI #12 “emergency department stay <4 h in patients with ISS ≥15”

Figure 2.

Percent conformity with PPI #5 “protection of airway in the emergency department for patients with a glasgow coma score <9”

All PPI, apart from stabilize/embolize unstable pelvic fractures and epidural hematoma surgery <1 h (both associated with very small sample sizes), were associated with statistically significant between-center variance, indicating their ability to discriminate between trauma centers [Table 2]. Five of the 10 PPI that applied to all levels of care had a statistically significant association with designation level. In particular, conformity to timely reduction of major dislocations (#4), airway protection (#5), and deaths on the ward rather than in the ED later than 1 h following arrival (#11) was higher in level I/II centers than in level III/IV centers [Table 2]. Similar results were observed for patient volume with the exception that conformity to timely surgery for open long bone fractures (#7) and therapeutic laparotomy (#13) was lower in centers with high annual patient volumes. We observed a significant positive correlation between performance results in the two eras (April 1998 – March2002 and April 2002 – March 2006) for 11 of the 13 PPI with sufficient data [Table 2].

Table 2.

Evaluation of process performance indicators: Discrimination (inter-center variation), construct validity (association with designation level and volume), and forecasting (correlation between 1999-2002 and 2003-2006)

DISCUSSION

In this multicenter retrospective cohort study, we identified a series of PPI based on literature review and expert consensus that can be calculated using trauma registry data. Results provide evidence of their discrimination, construct validity, and forecasting properties. These PPI can be implemented rapidly with routinely collected data and can be used to drive system-wide quality improvement efforts. The between-center variance observed in this study highlights the importance of evaluating process performance in integrated trauma systems.

Significant between-center variance indicates that all but two of the 15 PPI discriminate well across trauma centers. High correlation in performance between the two time periods demonstrates that most indicators have good forecasting properties; the performance of a trauma center in a given period of time is associated with that in the next period. In addition, PPI #4 (reduce dislocation of major articulations in <1 h), #5 (airway secured in ED for patients with Glasgow Coma Score <9), and #11 (deaths >1 h after arrival occur on ward) had statistically significant positive associations with designation level and volume. These results partially support their construct validity; level I/II trauma centers may be expected to show higher conformity to standards of care than level III/IV centers. In addition, higher levels of care and increased patient volume have been reported to be associated with improved patient outcome.[16,17] However, level I/II centers showed lower conformity to PPI #3 (transfer of spinal cord injury) and PPI #15 (no reintubation within 48 h of extubation) than lower level centers. For PPI #3, this situation is well documented in our system where a certain reluctance to transfer spinal cord injuries has been identified.

According to a recent systematic review published by Stelfox and colleagues,[7] only four multicenter studies based on global adult trauma populations have described the implementation of a series of PPI using trauma registry data.[18–21] None were implemented across all levels of trauma care. In addition, all were based on American College of Surgeons Committee on Trauma (ASCOT) audit filters proposed in 1993[8] despite the fact that only 19% of all PPI identified in the systematic review were ASCOT based.[7] Results of these studies suggest that only 9 of the 22 ASCOT filters are applicable with regular trauma registry data.

Evidence of the validity and reliability of trauma care performance indicators is scarce.[7] Evidence for reliability is mostly limited to preventable death classification and no information is available on the reliability of PPI derived from trauma registry data.[7] Work by Stelfox and colleagues suggests that there is no evidence of content validity or construct validity on trauma care process indicators derived from trauma registry data in the literature[7] and we found no study evaluating forecasting properties. Predictive criterion validity of certain PPI has been evaluated by assessing associations with outcomes such as mortality, morbidity, length of stay, costs, and complications.[18–21] However, results are contradictory and suggest that as many PPI are associated with favorable outcomes as with poor outcomes.[22] However, few of these studies used robust risk adjustment techniques and many were based on patient-level rather than hospital-level data which is subject to confounding by indication.[23]

Strengths and limitations

This study was based on a trauma registry with excellent population coverage of moderate to major trauma and rigorous data quality control mechanisms. In addition, the trauma registry used for analyses is based on uniform inclusion criteria with standardized prospective data collection procedures and represents an inclusive trauma system where designation is conducted according to American College of Surgeons Committee on Trauma criteria.[8] PPI were selected using a review of trauma system documents and peer-reviewed articles and the validity of indicators was partly addressed in analyses.

Potential limitations which may affect the interpretation of results include data quality, low volumes, and the generalizability of results. The validity of PPI relies on the reliability, validity, and completeness of trauma registry data, and data quality control measures are an essential part of any trauma system data collection effort. The Quebec Trauma Registry uses standardized coding procedures and rigorous data quality control mechanisms to improve data validity and reliability, and PPI were retained based on data quality issues as well as feasibility. However, as is common in trauma registries,[24] a high proportion of patients had missing data on physiological status. Multiple imputation procedures, widely used to address missing data in group comparisons, are not suitable for flagging individuals.[25] Missing physiological data, specifically the Glasgow Coma Score, was an issue for PPI #1, #5, and #10. This and other data quality issues may generate misleading results if data quality and completeness vary across trauma centers.

Some PPI are based on low numbers of eligible patients, particularly for low-volume centers. For example, “stabilize/embolize unstable pelvic fracture” and “epidural hematoma surgery <1 h” only had 40 and 254 eligible patients over the entire study period, respectively. This problem is partly addressed by multilevel modeling which shrinks estimates based on low sample sizes toward the global mean to improve precision. However, PPI may not be informative due to low sample sizes. Indeed, in the study population, no outliers were detected, indicating that these PPI do not discriminate across centers. Despite this, experts tend to agree that indicators should not be discarded based on low sample sizes alone because they may be flagged for high-volume centers and they can be useful for the calculation of composite performance measures.[26]

The Quebec Trauma Registry is based on American College of Surgeons criteria, but cannot directly replicate all trauma registries. Therefore, the PPI used here will not be integrally applicable across all trauma systems. In addition, PPI that can currently be evaluated using data collected in most trauma systems are unlikely to be the only PPI of interest. Indeed, out of 32 PPI that we identified from literature review and expert opinion, 17 were rejected because they could not be calculated (accurately) using Quebec Trauma Registry data. In addition, certain PPI could not be refined optimally due to lack of information in the trauma registry (e.g., taking account of non-surgical management decisions). In the long term, trauma registries should be adapted to a series of consensus-based PPI that have demonstrated not only discrimination, construct validity, and forecasting, but also reliability and predictive criterion validity on outcomes that may be more meaningful than mortality.[27,28] However, feasibility and the cost of data collection should be part of the selection process. In any event, process indicators are not static and will need to be adapted to local clinical contexts and be revised over time according to evidence-based clinical practice.

CONCLUSION

This study provides evidence of the feasibility and importance of implementing PPI using trauma registry data and gives guidance on methodological issues and presentation of results. We have identified a series of 15 PPI that can be used to drive system-wide trauma care quality improvement efforts using routinely collected trauma registry data. We have partially demonstrated the validity of these indicators. However, future research should evaluate the association between PPI and indicators of structure and outcome performance using hospital-level data as well as the influence of data quality. In addition, efforts should be invested into developing a composite measure of process performance.

References

1. American College of Surgeons Committee on trauma. Interfacility Transfer of Injured Patients: Guidelines for Rural Communities; 2002.

2. Royal College of Surgeons of England. Report of the Working Party on the Management of Patients with Head Injuries; 1999.

3. Hedges JR, Adams AL, Gunnels MD. ATLS practices and survival at rural level III trauma hospitals, 1995-1999. Prehosp Emerg Care 2002;6:299-305.

4. Tator CH, Duncan EG, Edmonds VE, Lapczak LI, Andrews DF. Neurological recovery, mortality and length of stay after acute spinal cord injury associated with changes in management. Paraplegia 1995;33:254-62.

5. Heinemann AW, Yarkony GM, Roth EJ, Lovell L, Hamilton B, Ginsburg K, et al. Functional outcome following spinal cord injury. A comparison of specialized spinal cord injury center vs general hospital short-term care. Arch Neurol 1989;46:1098-102.

6. Scivoletto G, Morganti B, Molinari M. Early versus delayed inpatient spinal cord injury rehabilitation: An Italian study. Arch Phys MedRehabil 2005;86:512-6.

7. Oakes DD, Wilmot CB, Hall KM, Sherck JP. Benefits of early admission to a comprehensive trauma center for patients with spinal cord injury. Arch Phys Med Rehabil 1990;71:637-43.

8. DeVivo MJ, Kartus PL, Stover SL, Fine PR. Benefits of early admission to an organised spinal cord injury care system. Paraplegia 1990;28:545-55.

9. Aung TS, el Masry WS. Audit of a British Centre for spinal injury. Spinal Cord 1997;35:147-50.

10. Smith M. Efficacy of specialist versus non-specialist management of spinal cord injury within the UK. Spinal Cord 2002;40:10-6.

11. Amin A, Bernard J, Nadarajah R, Davies N, Gow F, Tucker S. Spinal injuries admitted to a specialist centre over a 5-year period: A study to evaluate delayed admission. Spinal Cord 2005;43:434-7.

12. Calgary Health Region. Regional Trauma Services: Annual report 2004-2005.

13. South Western Sydney regional trauma registry: Trauma 10-Year Report 1995-2004.

14. Available from: www.oregon.gov/DHS/ph/ems/trauma/docs/9qiic.pdf [Last accessed on 2012 Dec 24].

15. Performance Improvement Guidelines for North Carolina Trauma Centers. April1, 2002 (with revised monitor/indicator filters of 2003).

16. Utah trauma system performance improvement guide. 2003.

17. Institute for emergency medical services systems. Trauma centers. Available at: http://www.miemss.org/home/Hospitals/TraumaCenters/tabid/131/Default.aspx. [Last accessed on 2012 Dec 24].

18. Available from: http://www.health.state.mn.us/traumasystem/hospresources/resourcemanual/index.html [Last accessed on 2012 Dec 24].

19. Available from: http://www.texinfo.library.unt.edu/Texasregister/html/2006/jun-09/tables-and-graphics/200602913-2.pdf [Last accessed on 2012 Apr 21].

20. Eastern Association for the Surgery of Trauma: Clinical guidelines, Available from: www.east.org [Last accessed on 2012 Dec 24].

21. Hicks IR, Hedley RM, Razis P. Audit of transfer of head-injured patients to a stand-alone neurosurgical unit. Injury 1994;25:545-9.

22. Winchell RJ, Hoyt DB. Endotracheal intubation in the field improves survival in patients with severe head injury. Trauma Research and Education Foundation of San Diego. Arch Surg 1997;132:592-7.

23. Cooper KR, Boswell PA. Reduced functional residual capacity and abnormal oxygenation in patients with severe head injury. Chest 1983;84:29-35.

24. Riemer BL, Butterfield SL, Diamond DL, Young JC, Raves JJ, Cottington E, et al. Acute mortality associated with injuries to the pelvic ring: The role of early patient mobilization and external fixation. J Trauma 1993;35:671-5; discussion 676-7.

25. Latenser BA, Gentilello LM, Tarver AA, Thalgott JS, Batdorf JW. Improved outcome with early fixation of skeletally unstable pelvic fractures. J Trauma 1991;31:28-31.

26. Matalon TS, Athanasoulis CA, Margolies MN, Waltman AC, Novelline RA, Greenfield AJ, et al. Hemorrhage with pelvic fractures: Efficacy of transcatheter embolization. AJR Am J Roentgenol 1979;133:859-64.

27. Panetta T, Sclafani SJ, Goldstein AS, Phillips TF, Shaftan GW. Percutaneous transcatheter embolization for massive bleeding from pelvic fractures. J Trauma 1985;25:1021-9.

28. Agolini SF, Shah K, Jaffe J, Newcomb J, Rhodes M, Reed JF 3rd. Arterial embolization is a rapid and effective technique for controlling pelvic fracture hemorrhage. J Trauma 1997;43:395-9.

29. Bassam D, Cephas GA, Ferguson KA, Beard LN, Young JS. A protocol for the initial management of unstable pelvic fractures. Am Surg 1998;64:862-7.

30. Wong YC, Wang LJ, Ng CJ, Tseng IC, See LC. Mortality after successful transcatheter arterial embolization in patients with unstable pelvic fractures: Rate of blood transfusion as a predictive factor. J Trauma 2000;49:71-5.

31. Hamill J, Holden A, Paice R, Civil I. Pelvic fracture pattern predicts pelvic arterial haemorrhage. Aust N Z J Surg 2000;70:338-43.

32. Miller PR, Moore PS, Mansell E, Meredith JW, Chang MC. External fixation or arteriogram in bleeding pelvic fracture: Initial therapy guided by markers of arterial hemorrhage. J Trauma 2003;54:437-43.

33. Shapiro M, McDonald AA, Knight D, Johannigman JA, Cuschieri J. The role of repeat angiography in the management of pelvic fractures. J Trauma 2005;58:227-31.

34. Sarin EL, Moore JB, Moore EE, Shannon MR, Ray CE, Morgan SJ, et al. Pelvic fracture pattern does not always predict the need for urgent embolization. J Trauma 2005;58:973-7.

35. Sadri H, Nguyen-Tang T, Stern R, Hoffmeyer P, Peter R. Control of severe hemorrhage using C-clamp and arterial embolization in hemodynamically unstable patients with pelvic ring disruption. Arch Orthop Trauma Surg 2005;125:443-7.

36. Fangio P, Asehnoune K, Edouard A, Smail N, Benhamou D. Early embolization and vasopressor administration for management of life-threatening hemorrhage from pelvic fracture. J Trauma 2005;58:978-84.

37. Kindsfater K, Jonassen EA. Osteomyelitis in grade II and III open tibia fractures with late debridement. J Orthop Trauma 1995;9:121-7.

38. Kreder HJ, Armstrong P. A review of open tibia fractures in children. J Pediatr Orthop 1995;15:482-8.

39. Poon WS, Li AK. Comparison of management outcome of primary and secondary referred patients with traumatic extradural haematoma in a neurosurgical unit. Injury 1991;22:323-5.

40. Cohen JE, Montero A, Israel ZH. Prognosis and clinical relevance of anisocoria-craniotomy latency for epidural hematoma in comatose patients. J Trauma 1996;41:120-2.

41. Mendelow AD, Karmi MZ, Paul KS, Fuller GA, Gillingham FJ. Extradural haematoma: Effect of delayed treatment. Br Med J 1979;1:1240-2.

42. Haselsberger K, Pucher R, Auer LM. Prognosis after acute subdural or epidural haemorrhage. Acta Neurochir (Wien) 1988;90:111-6.

43. Goris RJ, Gimbrere JS, van Niekerk JL, Schoots FJ, Booy LH. Early osteosynthesis and prophylactic mechanical ventilation in the multitrauma patient. J Trauma 1982;22:895-903.

44. Johnson KD, Cadambi A, Seibert GB. Incidence of adult respiratory distress syndrome in patients with multiple musculoskeletal injuries: Effect of early operative stabilization of fractures. J Trauma 1985;25:375-84.

45. Svenningsen S, Nesse O, Finsen V, Hole A, Benum P. Prevention of fat embolism syndrome in patients with femoral fractures-immediate or delayed operative fixation? Ann Chir Gynaecol 1987;76:163-6.

46. Bone LB, Johnson KD, Weigelt J, Scheinberg R. Early versus delayed stabilization of femoral fractures. A prospective randomized study. J Bone Joint Surg Am 1989;71:336-40.

47. Hofman PA, Goris RJ. Timing of osteosynthesis of major fractures in patients with severe brain injury. J Trauma 1991;31:261-3.

48. Poole GV, Miller JD, Agnew SG, Griswold JA. Lower extremity fracture fixation in head-injured patients. J Trauma 1992;32:654-9.

49. Charash WE, Fabian TC, Croce MA. Delayed surgical fixation of femur fractures is a risk factor for pulmonary failure independent of thoracic trauma. J Trauma 1994;37:667-72.

50. Starr AJ, Hunt JL, Chason DP, Reinert CM, Walker J. Treatment of femur fracture with associated head injury. J Orthop Trauma 1998;12:38-45.

51. Brundage SI, McGhan R, Jurkovich GJ, Mack CD, Maier RV. Timing of femur fracture fixation: Effect on outcome in patients with thoracic and head injuries. J Trauma 2002;52:299-307.

52. Pape HC, Auf'm'Kolk M, Paffrath T, Regel G, Sturm JA, Tscherne H. Primary intramedullary femur fixation in multiple trauma patients with associated lung contusion-a cause of posttraumatic ARDS? J Trauma 1993;34:540-78.

53. Fakhry SM, Rutledge R, Dahners LE, Kessler D. Incidence, management, and outcome of femoral shaft fracture: A statewide population-based analysis of 2805 adult patients in a rural state. J Trauma 1994;37:255-61.

54. Jaicks RR, Cohn SM, Moller BA. Early fracture fixation may be deleterious after head injury. J Trauma 1997;42:1-6.

55. Pape HC, Rixen D, Morley J, Husebye EE, Mueller M, Dumont C, et al. Impact of the method of initial stabilization for femoral shaft fractures in patients with multiple injuries at risk for complications (borderline patients). Ann Surg 2007;246:491-501.

56. Talucci RC, Manning J, Lampard S, Bach A, Carrico CJ. Early intramedullary nailing of femoral shaft fractures: A cause of fat embolism syndrome. Am JSurg 1983;146:107-11.

57. Reynolds MA, Richardson JD, Spain DA, Seligson D, Wilson MA, Miller FB. Is the timing of fracture fixation important for the patient with multiple trauma? Ann Surg 1995;222:470-8; discussion 478-81.

58. Velmahos GC, Arroyo H, Ramicone E, Cornwell EE 3^rd, Murray JA, Asensio JA, et al. Timing of fracture fixation in blunt trauma patients with severe head injuries. Am J Surg 1998;176:324-30.

59. Nair R, Abdool-Carrim AT, Robbs JV. Gunshot injuries of the popliteal artery. Br J Surg 2000;87:602-7.

60. Hossny A. Blunt popliteal artery injury with complete lower limb ischemia: Is routine use of temporary intraluminal arterial shunt justified? J Vasc Surg 2004;40:61-6.

61. Green NE, Allen BL. Vascular injuries associated with dislocation of the knee. J Bone Joint Surg Am 1977;59:236-9.

62. Patterson BM, Agel J, Swiontkowski MF, Mackenzie EJ, Bosse MJ. Knee dislocations with vascular injury: Outcomes in the Lower Extremity Assessment Project (LEAP) Study. J Trauma 2007;63:855-8.

63. Nathens AB, McMurray MK, Cuschieri J, Durr EA, Moore EE, Bankey PE, et al. The practice of venous thromboembolism prophylaxis in the major trauma patient. J Trauma 2007;62:557-63.

64. Steele N, Dodenhoff RM, Ward AJ, Morse MH. Thromboprophylaxis in pelvic and acetabular trauma surgery. The role of early treatment with low-molecular-weight heparin. J Bone Joint Surg Br 2005;87:209-12.

65. The Brain Trauma Foundation. Available from; http://www.braintrauma.org/site/PageServer? pagename =Guidelines

66. Perel P, Yanagawa T, Bunn F, Roberts I, Wentz R, Pierro A. Nutritional support for head-injured patients. Cochrane Database Syst Rev 2006;(4):CD001530.

67. Haller G, Myles PS, Wolfe R, Weeks AM, Stoelwinder J, McNeil J. Validity of unplanned admission to an intensive care unit as a measure of patient safety in surgical patients. Anesthesiology 2005;103:1121-9.

68. Copes WS, Staz CF, Konvolinka CW, Sacco WJ. American College of Surgeons audit filters: Associations with patient outcome and resource utilization. J Trauma 1995;38:432-8.

69. Birkmeyer JD, Hamby LS, Birkmeyer CM, Decker MV, Karon NM, Dow RW. Is unplanned return to the operating room a useful quality indicator in general surgery? Arch Surg 2001;136:405-11.

70. Kroon HM, Breslau PJ, Lardenoye JW. Can the incidence of unplanned reoperations be used as an indicator of quality of care in surgery? Am J Med Qual 2007;22:198-202.

71. American College of Surgeons, Committee on Trauma. Advanced trauma life support program for physicians: Instructor manual. Chicago: American College of Surgeons; 1993. p. 75-110.

72. Riddez L, Johnson L, Hahn RG. Central and regional hemodynamics during crystalloid fluid therapy after uncontrolled intra-abdominal bleeding. J Trauma 1998;44:433-9.

73. Sakles JC, Sena MJ, Knight DA, Davis JM. Effect of immediate fluid resuscitation on the rate, volume, and duration of pulmonary vascular hemorrhage in a sheep model of penetrating thoracic trauma. Ann Emerg Med 1997;29:392-9.

74. Stern SA, Dronen SC, Birrer P, Wang X. Effect of blood pressure on hemorrhage volume and survival in a near-fatal hemorrhage model incorporating a vascular injury. Ann Emerg Med 1993;22:155-63.

75. Kowalenko T, Stern S, Dronen S, Wang X. Improved outcome with hypotensive resuscitation of uncontrolled hemorrhagic shock in a swine model. J Trauma 1992;33:349-53; discussion 361-2.

76. Bickell WH, Bruttig SP, Millnamow GA, O'Benar J, Wade CE. The detrimental effects of intravenous crystalloid after aortotomy in swine. Surgery 1991;110:529-36.

77. Abu-Hatoum O, Bashenko Y, Hirsh M, Krausz MM. Continuous fluid resuscitation and splenectomy for treatment of uncontrolled hemorrhagic shock after massive splenic injury. J Trauma 2002;52:253-8.

78. Matsuoka T, Hildreth J, Wisner DH. Uncontrolled hemorrhage from parenchymal injury: Is resuscitation helpful? J Trauma 1996;40:915-22.

79. Bickell WH, Wall MJJr, Pepe PE, Martin RR, Ginger VF, Allen MK, et al. Immediate versus delayed fluid resuscitation for hypotensive patients with penetrating torso injuries. N Engl J Med 1994;331:1105-9.

80. Kashuk JL, Moore EE, Millikan JS, Moore JB. Major abdominal vascular trauma--A unified approach. J Trauma 1982;22:672-9.

81. Jurkovich GJ, Greiser WB, Luterman A, Curreri PW. Hypothermia in trauma victims: An ominous predictor of survival. J Trauma 1987;27:1019-24.

82. Steinemann S, Shackford SR, Davis JW. Implications of admission hypothermia in trauma patients. J Trauma 1990;30:200-2.

83. Ferrara A, MacArthur JD, Wright HK, Modlin IM, McMillen MA. Hypothermia and acidosis worsen coagulopathy in the patient requiring massive transfusion. Am J Surg 1990;160:515-8.

84. Bernabei AF, Levison MA, Bender JS. The effects of hypothermia and injury severity on blood loss during trauma laparotomy. J Trauma 1992;33:835-9.

85. Gunning KA, Sugrue M, Sloane D, Deane SA. Hypothermia and severe trauma. Aust N Z J Surg 1995;65:80-2.

86. Cosgriff N, Moore EE, Sauaia A, Kenny-Moynihan M, Burch JM, Galloway B. Predicting life-threatening coagulopathy in the massively transfused trauma patient: Hypothermia and acidoses revisited. J Trauma 1997;42:857-62.

87. Gentilello LM, Jurkovich GJ, Stark MS, Hassantash SA, O'Keefe GE. Is hypothermia in the victim of major trauma protective or harmful? A randomized, prospective study. Ann Surg 1997;226:439-49.

88. Krishna G, Sleigh JW, Rahman H. Physiological predictors of death in exsanguinating trauma patients undergoing conventional trauma surgery. Aust N Z J Surg 1998;68:826-9.

89. Shafi S, Elliott AC, Gentilello L. Is hypothermia simply a marker of shock and injury severity or an independent risk factor for mortality in trauma patients? Analysis of a large national trauma registry. J Trauma 2005;59:1081-5.

90. Wang HE, Callaway CW, Peitzman AB, Tisherman SA. Admission hypothermia and outcome after major trauma. Crit Care Med 2005;33:1296-301.

91. Martin RS, Kilgo PD, Miller PR, Hoth JJ, Meredith JW, Chang MC. Injury-associated hypothermia: An analysis of the 2004 National Trauma Data Bank. Shock 2005;24:114-8.

92. Aitken LM, Hendrikz JK, Dulhunty JM, Rudd MJ. Hypothermia and associated outcomes in seriously injured trauma patients in a predominantly sub-tropical climate. Resuscitation 2009;80:217-23.

93. Inaba K, Teixeira PG, Rhee P, Brown C, Salim A, DuBose J, et al. Mortality impact of hypothermia after cavitary explorations in trauma. World J Surg 2009;33:864-9.

94. Beilman GJ, Blondet JJ, Nelson TR, Nathens AB, Moore FA, Rhee P, et al. Early hypothermia in severely injured trauma patients is a significant risk factor for multiple organ dysfunction syndrome but not mortality. Ann Surg 2009;249:845-50.

95. Carr BG, Kaye AJ, Wiebe DJ, Gracias VH, Schwab CW, Reilly PM. Emergency department length of stay: A major risk factor for pneumonia in intubated blunt trauma patients. J Trauma 2007;63:9-12.

96. Richardson JD, Franklin G, Santos A, Harbrecht B, Danzl D, Coleman R, et al. Effective triage can ameliorate the deleterious effects of delayed transfer of trauma patients from the emergency department to the ICU. J Am Coll Surg 2009;208:671-81.

97. Nayduch D, Moylan J, Snyder BL, Andrews L, Rutledge R, Cunningham P. American College of Surgeons trauma quality indicators: An analysis of outcome in a statewide trauma registry. J Trauma 1994;37:565-75.

98. Clarke JR, Trooskin SZ, Doshi PJ, Greenwald L, Mode CJ. Time to laparotomy for intra-abdominal bleeding from trauma does affect survival for delays up to 90 minutes. J Trauma 2002;52:420-5.

99. Choi KC, Peek-Asa C, Lovell M, Torner JC, Zwerling C, Kealey GP. Complications after therapeutic trauma laparotomy. J Am Coll Surg 2005;201:546-53.

100. Sorensen VJ, Mikhail JN, Karmy-Jones RC. Is delayed laparotomy for blunt abdominal trauma a valid quality improvement measure in the era of nonoperative management of abdominal injuries? J Trauma 2002;52:426-33.

101. Stewart BT, Lee V, Danne PD. Laparotomy for trauma in a regional centre: The effect of delay on outcome. Aust N Z J Surg 1994;64:484-7.

102. Inaba K, Demetriades D. The nonoperative management of penetrating abdominal trauma. Adv Surg 2007;41:51-62.

103. Renz BM, Feliciano DV. The length of hospital stay after an unnecessary laparotomy for trauma: A prospective study. J Trauma 1996;40:187-90.

104. Niederee MJ, Byrnes MC, Helmer SD, Smith RS. Delay in diagnosis of hollow viscus injuries: Effect on outcome. Am Surg 2003;69:293-9.

105. Fakhry SM, Brownstein M, Watts DD, Baker CC, Oller D. Relatively short diagnostic delays (<8 hours) produce morbidity and mortality in blunt small bowel injury: An analysis of time to operative intervention in 198 patients from a multicenter experience. J Trauma 2000;48:408-15.

106. Fang JF, Chen RJ, Lin BC, Hsu YB, Kao JL, Kao YC, Chen MF. Small bowel perforation: Is urgent surgery necessary? J Trauma 1999;47:515-20.

107. Willis CD, Stoelwinder JU, Cameron PA. Interpreting process indicators in trauma care: Construct validity versus confounding by indication. Int J Qual Health Care 2008;20:331-8.

108. Demetriades D, Velmahos GC, Scalea TM, Jurkovich GJ, Karmy-Jones R, Teixeira PG, et al. Blunt traumatic thoracic aortic injuries: Early or delayed repair--results of an American Association for the Surgery of Trauma prospective study. J Trauma 2009;66:967-73.

109. Moini M, Takyar MA, Rasouli MR. Revascularisation later than 24h after popliteal artery trauma: Is it worthwhile? Injury 2007;38:1098-101.

110. Pfeifer R, Pape HC. Missed injuries in trauma patients: A literature review. Patient Saf Surg 2008;2:20.

111. Enderson BL, Reath DB, Meadors J, Dallas W, DeBoo JM, Maull KI. The tertiary trauma survey: A prospective study of missed injury. J Trauma 1990;30:666-9; discussion 669-70.

112. Aaland MO, Smith K. Delayed diagnosis in a rural trauma center. Surgery 1996;120:774-9.

113. Janjua KJ, Sugrue M, Deane SA. Prospective evaluation of early missed injuries and the role of tertiary trauma survey. J Trauma 1998;44:1000-7.

114. Vles WJ, Veen EJ, Roukema JA, Meeuwis JD, Leenen LP. Consequences of delayed diagnoses in trauma patients: A prospective study. J Am CollSurg 2003;197:596-602.

115. Soundappan SV, Holland AJ, Cass DT. Role of an extended tertiary survey in detecting missed injuries in children. J Trauma 2004;57:114-8.

116. Frawley PA. Missed injuries in the multiply traumatized. Aust N Z J Surg 1993;63:935-9.

117. Rizoli SB, Boulanger BR, McLellan BA, Sharkey PW. Injuries missed during initial assessment of blunt trauma patients. Accid Anal Prev 1994;26:681-6.

118. Robertson R, Mattox R, Collins T, Parks-Miller C, Eidt J, Cone J. Missed injuries in a rural area trauma center. Am J Surg 1996;172:564-8.

119. Furnival RA, Woodward GA, Schunk JE. Delayed diagnosis of injury in pediatric trauma. Pediatrics 1996;98:56-62.

120. Peery CL, Chendrasekhar A, Paradise NF, Moorman DW, Timberlake GA. Missed injuries in pediatric trauma. Am Surg 1999;65:1067-9.

121. Buduhan G, McRitchie DI. Missed injuries in patients with multiple trauma. J Trauma 2000;49:600-5.

122. Connors JM, Ruddy RM, McCall J, Garcia VF. Delayed diagnosis in pediatric blunt trauma. Pediatr Emerg Care 2001;17:1-4.

123. Houshian S, Larsen MS, Holm C. Missed injuries in a level I trauma center. J Trauma 2002;52:715-9.

124. Brooks A, Holroyd B, Riley B. Missed injury in major trauma patients. Injury 2004;35:407-10.

125. Perno JF, Schunk JE, Hansen KW, Furnival RA. Significant reduction in delayed diagnosis of injury with implementation of a pediatric trauma service. Pediatr Emerg Care 2005;21:367-71.

126. Kalemoglu M, Demirbas S, Akin ML, Yildirim I, Kurt Y, Uluutku H, et al. Missed injuries in military patients with major trauma: Original study. Mil Med 2006;171:598-602.

127. Esteban A, Alia I, Gordo F, Fernández R, Solsona JF, Vallverdú I, et al. Extubation outcome after spontaneous breathing trials with T-tube or pressure support ventilation. The Spanish Lung Failure Collaborative Group. Am J Respir Crit Care Med 1997;156:459-65.

128. Gowardman JR, Huntington D, Whiting J. The effect of extubation failure on outcome in a multidisciplinary Australian intensive care unit. Crit Care Resusc 2006;8:328-33.

129. Nisim AA, Margulies DR, Wilson MT, Alban RF, Dang CM, Allins AD, et al. A 2-minute pre-extubation protocol for ventilated intensive care unit patients. Am J Surg 2008;196:890-5.

Appendix: 32 Process performance indicators