Failure-to-Rescue as a Center-Level Metric in Pediatric Trauma

Abstract

BACKGROUND

Failure-to-rescue (FTR) is defined as death after a complication and has been used to evaluate quality of care in adult trauma patients, but there are no published studies on FTR in pediatric trauma. The aim of this study was to define the relationship between rates of mortality, complications, and FTR at centers caring for pediatric (<18 years of age) trauma patients in a nationally representative database.

METHODS

We performed a retrospective cohort study of the 2015 and 2016 National Trauma Data Bank. We included patients <18 years with an Injury Severity Score of ≥9. We excluded centers with <50 pediatric patients and/or that reported no complications. We calculated the complication, FTR, mortality, and precedence rates by center and divided centers into tertiles of mortality. We compared complication and FTR rates between high and low tertiles of mortality using the Kruskal-Wallis test.

RESULTS

Of 62,190 patients from 284 centers, 2,204 patients had at least one complication for an overall complication rate of 4% (center-level 0–15%), while 120 patients died after a complication for an overall FTR rate of 5% (center-level 0–67%). High-mortality centers had both higher FTR rates (10% vs 0.6%, p<.001) and higher complication rates (5% vs 4%, p=.001) than lower-mortality hospitals. The overall precedence rate was 15% with a median rate of 0% (IQR 0%−25%).

CONCLUSIONS

Both complication and FTR rates are low in the pediatric injury population, but both complication and FTR rates are higher at higher mortality centers. The low overall complication rates and precedence rates likely limit the utility of FTR as a valid center-level metric in this population, but further investigation into individual FTR cases may reveal important opportunities for improvement.

TOC Statement- 19-sus-07

The goal of this study was to define the relationship between mortality, complication, and failure-to-rescue (FTR) rates at centers caring for pediatric trauma patients. The importance of this finding is a better understanding of FTR and its validity as a center-level metric for quality of care in the pediatric trauma population.

Introduction:

Failure-to-rescue (FTR) is defined as death after a complication (1) and has proven to be a useful quality metric in adult surgical populations. Historically, center-level quality of care has been evaluated using complication and mortality rates, but there are several attractive properties of the FTR metric. First, there is an established correlation between FTR rates and in-hospital mortality rates across a variety of surgical conditions in adult patients (≥18 years of age) (2–7). Second, while complication and mortality rates tend to be strongly associated with non-modifiable patient characteristics, FTR rates are more strongly associated with hospital characteristics, such as teaching status (4), nurse education levels (8), bed volume (4), and nurse-to-patient ratio (4, 9). These factors are subject to modification at the level of the institution which suggests a path towards improving patient care.

The initial description of FTR was in adult patients undergoing elective surgery (1), but findings have been expanded to acute care surgery populations (2–7). In addition to associating significantly with several hospital characteristics (4), FTR rate has been shown to correlate better with in-hospital mortality than complication rates alone. In line with this, FTR rate has been shown to follow a stepwise increase from low- to high-mortality centers, while complication rates do not vary significantly between high and low volume centers (5–7, 10).

FTR has been useful in evaluating pediatric quality of care in pediatric liver transplantation (11), cardiac surgery (12), obstetrics (13–15), for children’s hospitals (16, 17), and as a measure of racial or ethnic disparities among congenital heart surgery pediatric patients (18). However, this metric is poorly described for pediatric trauma. There is a critical need to compare quality of care in pediatric trauma, but this is challenging given low mortality rates in pediatric populations.

In this study, we sought to define the relationship between rates of mortality, complications, and FTR at centers managing pediatric (<18 years of age) trauma using the 2016 National Trauma Database. We hypothesized that FTR rates would be lowest at low mortality centers and would show a stepwise increase across increasing tertiles of center level mortality. We further hypothesized that complication rates would be similar between high- and low-mortality centers.

Methods:

We performed a retrospective cohort study of the 2015 and 2016 National Trauma Data Bank (NTDB). We included all inpatient trauma patients during the study period who were <18 years of age and had an Injury Severity Score (ISS) of ≥9. Demographic information (age, race, sex), physiologic data (admission Glasgow Coma Score [GCS] and subscores, systolic blood pressure), complications and ISS were obtained from the 2015 and 2016 NTDB. Due to known issues with underreporting of complications in the NTDB (19), we excluded centers that reported no complications and/or that treated fewer than 50 pediatric patients per year (Figure 1).

Figure 1.

Flow diagram of patients in the study. Abbreviations: ISS = Injury Severity Score; FTR = Failure-to-rescue

The cohort of interest in this study was pediatric trauma patients who sustained inhospital complications (Table 1). The definitions of the 2015 and 2016 NTDB National Trauma Data Standard, Appendix 3 were used as complications (20). Out of the group of patients who sustained complications, the patients who died comprised the total number of FTR cases. This study was approved by the University of Pennsylvania’s Institutional Review Board.

Table 1.

Complications evaluated in this study.

Hospital Complications
Acute kidney injury (AKI)
Acute respiratory distress syndrome (ARDS)
Cardiac arrest with CPR
Catheter-associated urinary tract infection (CAUTI)
Central-line-associated bloodstream infection (CLABSI)
Decubitus ulcer
Deep surgical infection
Deep vein thrombosis (DVT)
Drug or alcohol withdrawal syndrome
Extremity compartment syndrome
Myocardial infarction
Organ/space surgical site infection
Osteomyelitis
Other
Pulmonary embolism
Severe sepsis
Stroke/CVA
Superficial incisional surgical site infection
Unplanned admission to ICU
Unplanned intubation
Unplanned return to OR
Ventilator-associated pneumonia (VAP)

Abbreviations: CVA = cerebrovascular accident; ICU = Intensive care unit; OR = Operating room

First, to better understand pediatric trauma patients at risk for FTR, we divided patients with complications into those who survived and those who did not. Within each outcome category, we calculated the proportions of patients with 1, 2, and ≥3 complications. We then compared mortality rates according to complication type. Next, we calculated complication, mortality, FTR, and precedence rates for each center. The precedence rate was defined as the proportion of deaths preceded by complications, and was included because it has been used as a proxy for the reliability and validity of the FTR metric (21). Finally, we divided the centers into tertiles of mortality to conduct comparisons of center-level characteristics, such as pediatric trauma center level, teaching status, total bed volume, and pediatric bed volume. We then compared FTR rates among these tertiles using the Kruskal-Wallis test. All statistical analyses were performed using Stata version 14.1 (College Station, TX).

Results:

We included 62,190 patients from 284 centers in the study (Figure 1). The demographics of our study population are described in Table 2. The overall mortality rate was 2%. Complications occurred in 2,204 patients for an overall complication rate of 4% (center-level range 0–15%). Among patients with complications, 120 died for an FTR rate of 5% (center-level 0–67%). The most frequent complications were cardiac arrest (24%), unplanned return to ICU (15%), and unplanned return to OR (12%). The overall precedence rate was 15% with a median rate of 0% (IQR 0%−25%).

Table 2.

Demographics and other patient characteristics.

Patients included in study (n=62,190)
Age	11 (5–15)
Sex
Male	41,850 (67%)
Race
White	39,357 (63%)
Black	11,462 (18%)
Other	9,512 (15%)
Missing	1,859 (3%)
Physiology
GCS motor score	6 (6–6)
SBP (mmHg)	120 (109–132)
Injury Mechanism
Blunt	64,35 (85%)
Penetrating	4,789 (6%)
Other	4,264 (6%)
Missing	2,574 (3%)
Injury Severity Score	10 (9–17)

Continuous values expressed as median (Interquartile Range);

Categorical values expressed as n (%).

Abbreviations: GCS = Glasgow Coma Score, SBP = systolic blood pressure, ISS = Injury Severity Score

When comparing deaths and survivors with complications, we found that patient deaths were significantly associated with preceding cardiac arrest (p<.001) and acute respiratory distress syndrome (p<.001) (Table 3). Overall, survivors had more complications than patients who died (6% vs 0%, p<.001).

Table 3.

Evaluating patients by total complications and complication type.

Patients with complications (n=2,204)
	Lived n=2,084	Died n=120	p
Total complications			<.001
1	1644 (79%)	117 (98%)
2	316 (15%)	3 (3%)
>3	124 (6%)	0 (0%)
Complication type
Cardiac arrest	406 (19%)	116 (97%)	<.001
Other	385 (18%)	8 (7%)	<.001
Unplanned return to ICU	324 (16%)	0 (0%)	<.001
Unplanned return to OR	273 (13%)	0 (0%)	<.001
Acute respiratory distress syndrome	262 (13%)	2 (2%)	<.001
Decubitus ulcer	256 (12%)	0 (0%)	<.001
Deep vein thrombosis	244 (12%)	2 (2%)	<.001
Unplanned intubation	206 (10%)	3 (3%)	.01
Acute kidney injury	115 (6%)	0 (0%)	.01
Severe sepsis	101 (5%)	0 (0%)	.01
Extremity compartment syndrome	100 (5%)	0 (0%)	.01
Stroke	96 (5%)	0 (0%)	.02
Superficial surgical site infection	92 (4%)	0 (0%)	.02
Deep surgical infection	85 (4%)	0 (0%)	.02
Organ space surgical site infection	72 (3%)	0 (0%)	.04
Pulmonary embolism	42 (2%)	0 (0%)	.12
Drug or alcohol withdrawal syndrome	20 (1%)	0 (0%)	.28
Osteomyelitis	7 (0%)	0 (0%)	.53
Myocardial infarction	2 (0%)	0 (0%)	.73

Patients binned by total complications and complication type.

Categorical values expressed as n (%).

P values result from χ²-squared test.

Abbreviations: CVA = cerebrovascular accident; ICU = Intensive care unit; OR = Operating room

In our center-level comparisons, we divided the 284 centers into tertiles of low, medium and high mortality, each with approximately 94 centers (Table 4). Pediatric trauma level was significantly different between tertiles of hospital mortality, such that Level I trauma centers were less likely to be in the highest tertile of mortality than Level II centers (22% vs 46%, p<.001). Total pediatric bed volume was significantly associated with mortality. Low-mortality centers were more likely to have a greater total pediatric bed volume of at least 31 beds (63% vs 44%, p=.001). Center teaching status was also significantly associated across tertiles. University teaching centers were less likely than non-teaching centers to be in the highest tertile of mortality (26% vs 52%, p=.002). Across the tertiles, we found no significant association with total hospital bed volume (p=.264).

Table 4.

Center-level characteristics by tertiles of mortality.

	Tertiles of Mortality
	Low n=95	Middle n=95	High n=94	p
Trauma center peds level				<.001
Level I	55 (35%)	68 (43%)	35 (22%)
Level II	38 (31%)	27 (22%)	56 (46%)
Other	2 (40%)	0 (0%)	3 (60%)
Teaching status				.002
University	48 (33%)	61 (42%)	38 (26%)
Community	43 (40%)	25 (23%)	42 (38%)
Non-teaching	4 (15%)	9 (33%)	14 (52%)
Bed volume				.259
≤200 beds	8 (47%)	5 (29%)	4 (24%)
201–400 beds	27 (39%)	24 (35%)	18 (26%)
401–600 beds	25 (26%)	31 (32%)	41 (42%)
>600 beds	35 (35%)	35 (35%)	31 (31%)
Number of pediatric beds				.001
0–10 beds	14 (26%)	11 (21%)	29 (54%)
11–30 beds	21 (36%)	16 (27%)	22 (37%)
>31 beds	60 (36%)	68 (40%)	41 (24%)

Continuous values expressed as median (Interquartile Range);

Categorical values expressed as n (column %).

P values result from χ²-squared test.

Finally, we used the Kruskal-Wallis test to assess significant differences in our complication and FTR rates by tertiles of mortality. High-mortality centers had both higher FTR rates (10% vs 0.6%, p<.001) and higher complication rates (5% vs 4%, p=.001) than lower-mortality hospitals (Figure 2).

Figure 2.

Differences in FTR and complications rates by tertile of mortality. FTR, Failure-to-rescue. P-values result from Kruskal-Wallis test. High-mortality centers had both higher FTR rates (10% vs 0.6%, p = <.001) and higher complication rates (5% vs 4%, p = .001) than lower-mortality hospitals. Abbreviations: FTR = Failure-to-rescue

Discussion:

In this study, we sought to evaluate the utility of FTR as a quality metric in the pediatric trauma population. We found that both complication and FTR rates are low in this cohort. However, consistent with studies in adult populations, FTR rates increased in a stepwise fashion across increasing tertiles of mortality. In contrast to literature from other populations demonstrating that complication rates are similar between high and low mortality centers, we found that complication rates are also higher at high-mortality centers.

Existing literature on the effectiveness of FTR as a quality-care metric in other pediatric patient populations has shown both different and similar conclusions compared to our study. In pediatric populations for cardiac surgery, FTR rates but not complication rates were correlated with center-level mortality, which was similar to findings from the adult literature (12). However, in the pediatric liver transplant population, the low-mortality tertile had both significantly lower FTR and complication rates, which is similar to the findings of our study (22).

Our findings also differ from the adult trauma literature with respect to the complications found. The most frequent complications for the adult trauma population are pneumonia, acute respiratory distress syndrome (ARDS), and sepsis, while one of the least common is cardiovascular complications (23). In the pediatric trauma population, the most frequent complication was cardiac arrest (0.27%). This parallels the complication frequency of pediatric FTR patients, where the most common are cardiac arrest (97%) and unplanned intubation (3%).

We found that the percentage of deaths meeting the definition of FTR (death following a complication) was only 5%, indicating that the majority of injured children who died in our study are not recorded as having experienced complications. There are several possible explanations for this finding. First, missing data is a known issue with the NTDB and it is possible that the low precedence rate observed here is a function of underreporting of complications. However, precedence rates are similarly low in adult trauma populations both using the NTDB and other data sources (24). Therefore, including patients who died without recorded complications in the FTR rate (as described in the original FTR methodology for elective surgical patients (1, 21)) is probably not appropriate in trauma populations.

Seeking variability in outcomes between centers is a key first step in many quality improvement programs, including the American College of Surgeons’ Trauma Quality Improvement Program (25). From a methodologic perspective, the ability to detect differences in outcomes between centers is contingent on both the number of centers in the population and the number of events at these centers. Even using two years of NTDB data with over 62,000 patients at 284 centers included in our study, the number of patients sustaining a complication (and therefore at risk for FTR) was only a little over 2,000 (4%). In this subset, the mortality (overall FTR) rate was only 2%. Given these low event rates, the likelihood that the FTR metric will be useful as a means to compare performance between trauma centers caring for injured children is low. However, from an institutional standpoint, examination of deaths meeting the definition of FTR may still yield useful insights into opportunities for improvement. While more sensitive than specific for preventability, mortality cases meeting the definition of FTR are more likely to be judged preventable or potentially preventable than those that do not in peer review (26).

Limitations:

As with any retrospective study analysis, our work has limitations to address. First, an FTR case is canonically defined as a death preceded by any “major” complication. There is a precedent for “major” complications being defined as those noted in the sentinel publication on FTR (1) and in the case of adult trauma patients (26). In our study of pediatric patients, we expanded this precedence definition to include deaths preceded by any complication in order to increase the precedence rate and thus include a greater proportion of total deaths (27).

Second, underreporting of complications is a well-known phenomenon and limitation of the NTDB. We did our best to limit the effects through our exclusion criteria of centers with volume lower than 50 patients per year and/or centers that had reported no complications. We recognize that, by implementing these criteria, we potentially bias our data and analysis by concentrating the total proportion of patients who died with complications and affecting the overall FTR rate, when it might be possible that in fact these centers had a 0% FTR rate and no complications. Though the precedence rates we reported here are lower than have been reported in adult trauma populations, low precedence rates after injury do not necessarily reflect underreporting of complications (27). For instance, consider the case of a patient who presents with a transcranial gunshot wound and dies without a recorded complication. It is possible that complications occurred but were not recorded, but it is also possible the patient died without a complication due to progression of an unsurvivable injury. Unlike in elective surgery cohorts, non-precedented deaths after injury likely reflect a mixture of unsurvivable injuries and unrecorded complications, and methods for dealing with this issue remain an area of active investigation.

Third, we found that the most common complication leading to FTR was cardiac arrest. Although this is listed as a complication, we are unable to definitively say that this is a true complication and not just the terminal mechanism of death. Addressing the differences between adult and pediatric trauma complication rates, we note that overall complication rates are much lower in pediatric populations than adult populations. This raises the possibility that the cardiac arrest appears to be relatively more common in pediatric populations only because complications that tend to occur most frequently in adult trauma populations (such as pneumonia and ARDS) are relatively less common. However, due to the retrospective nature of this study, we remain limited in our ability to speak to the exact reasons that cardiac arrest occurs most frequently.

Finally, we found only 120 FTR cases out of the total 62,190 patients in our study. As a result, when we try to evaluate any significance between FTR rate and patient characteristics, the small sample size does not allow for risk-adjusted center-level comparisons. However, we believe that, while certainly not without flaws, the NTDB is currently the dataset that best reflects the pediatric trauma patient population. Since this represents the gold standard for measuring outcomes in pediatric trauma, it is difficult to disentangle the dataset from the population. We hope that with increased focus on quality of care that has been endorsed by the ACS Pediatric Quality Improvement Program (ACS TQIP Pediatric), the quality of data will continue to improve to the point that there will be better complication records, which can reduce the consequences of underreported complications, and better overall representation of the patient population.

Conclusion:

In summary, although we found evidence that FTR rates and mortality rates are strongly correlated in this pediatric trauma cohort, the overall proportion of FTR cases observed in this population is too small for us to proceed with using FTR as a center-level quality metric. Because the majority of pediatric trauma mortalities did not meet the definition for FTR, efforts to improve quality of care using the FTR metric may require a modified definition of FTR accounting for the proportion of ‘non-FTR’ deaths that could have been prevented by trauma centers caring for pediatric populations. Failing this, the utility of FTR as a quality metric in pediatric trauma patients may be so limited as to preclude useful comparisons between centers.