Simulation-Based Training is Associated with Lower Risk-Adjusted Mortality in ACS Pediatric TQIP Centers

Abstract

Background:

Although use of simulation-based team training for pediatric trauma resuscitation has increased, its impact on patient outcomes has not yet been shown. The purpose of this study was to determine the association between simulation use and patient outcomes.

Methods:

Trauma centers that participate in the American College of Surgeons (ACS) Pediatric Trauma Quality Improvement Program (TQIP) were surveyed to determine frequency of simulation use in 2014 and 2015. Center-specific clinical data for 2016 and 2017 were abstracted from the ACS TQIP registry (n=57,916 patients) and linked to survey responses. Center-specific risk-adjusted mortality was estimated using multivariable hierarchical logistic regression and compared across four levels of simulation-based training use: no training, low-volume training, high-volume training, and survey non-responders (unknown training use).

Results:

Survey response rate was 75% (94/125 centers) with 78% of the responding centers (73/94) reporting simulation use. The average risk-adjusted odds of mortality was lower in centers with a high volume of training compared to centers not using simulation (OR 0.58, 95% CI 0.37-0.92). The times required for resuscitation processes, evaluations, and critical procedures (endotracheal intubation, head computed tomography, craniotomy, and surgery for hemorrhage control) were not different between centers based on levels of simulation use.

Conclusions:

Risk-adjusted mortality is lower in TQIP-Pediatric centers using simulation-based training, but this improvement in mortality may not be mediated by a reduction in time to critical procedures. Further investigation into alternative mediators of improved mortality associated with simulation use is warranted, including assessment of resuscitation quality, improved communication, enhanced teamwork skills, and decreased errors.

Level of Evidence:

Level III therapeutic / care management

INTRODUCTION

Injury remains the leading cause of mortality in children 1-18 years old.^1,2 More than 50% of deaths after injury are within the first 24 hours.³ This early mortality shows the importance of improving the quality of the resuscitation phase of care for critically injured children. Children may have better outcomes when treated at pediatric trauma centers,^4-8 but a minority of severely injured children are initially resuscitated at pediatric centers.^9,10 Due to the rarity of severe injury in children, providers in pediatric centers may lack experience caring for an injured child in extremis.¹¹ Trauma resuscitation is a time-dependent process, and time to completion of critical evaluation and intervention may play a critical role in improving outcomes.^12-17 Achieving a timely and high-quality multidisciplinary resuscitation requires experienced trauma providers working as a coordinated team.

Simulation-based training has been associated with improved team performance during trauma resuscitation.^18-22 Simulation use has also been associated with improved outcomes for pediatric in-hospital cardiac arrest.²³ The use of simulation-based training for pediatric trauma resuscitation has been reported by several single-center studies and is being increasingly utilized,^18,22,24,25 but not in a standardized fashion.²⁶ Demonstrated benefits of multidisciplinary simulation-based team training for trauma resuscitation also include faster time to completion of a) the primary survey, b) critical procedures (time to endotracheal intubation and time to computed tomography completion), and c) emergency surgery in single-center studies of adult trauma patients.²⁷ The impact of simulation use on patient outcomes, however, has not been studied widely and has not been studied specifically for injured children.

The purpose of this study was to determine if the use of simulation-based training for trauma resuscitation is associated with improved performance measured by 1) time to critical evaluations and procedures, and 2) risk-adjusted mortality in pediatric trauma patients. We hypothesized that pediatric trauma centers that use simulation-based training would have lower risk-adjusted mortality and faster times to critical evaluations and procedures for injured children.

METHODS

Study Population

All trauma centers participating in the American College of Surgeons Pediatric Trauma Quality Improvement Program (ACS Pediatric TQIP) in 2016 (N=125) were selected for inclusion in this cross-sectional cohort study. Participation in the ACS Pediatric TQIP Program is limited to centers with either state designation or ACS verification as a pediatric trauma center. Trauma centers participating in the ACS Pediatric TQIP program contribute data to a national registry in accordance with the National Trauma Data Standard. The Pediatric TQIP registry includes patients eighteen years and under with at least one injury with an Abbreviated Injury Scale (AIS) severity score of two or higher. Patients are excluded from the registry if presenting without signs of life, are discharged home from the emergency department, have a pre-existing advanced directive, or have a major burn injury.²⁸ Transfer-in patients were excluded from the primary mortality analyses and from the secondary time-to-procedure analyses, as simulation use would not be expected to impact outcomes in patients that underwent initial resuscitation at a referring facility. Transfer-in patients were, however, analyzed independently and used as an internal center-specific control for risk-adjusted modeling of mortality, as these patients should not be affected by any impact of simulation-based training. Human Subjects approval was obtained from the Children’s Hospital Los Angles Institutional Review Board.

Survey Development, Implementation, and Exposure Definition

A seventeen-item survey was developed by the study team and piloted for readability by a group of trauma program managers. Full details of the survey have been previously described.²⁶ The survey was administered electronically via Qualtrics online software to trauma program managers at each participating center and subsequently sent to trauma medical directors at initially nonresponding centers. Additional follow-up by direct phone survey was attempted by ACS TQIP program staff, with verbal survey administration of the survey. Survey items included annual number of simulation-based training sessions for calendar years 2014 and 2015 (scaled 0-12 and 13+). Trauma center simulation use was categorized as no training (zero simulation-based training sessions in two years), low-volume training (1-10 simulation-based training sessions over two years), high-volume training (11 or more simulation-based training sessions over two years), or unknown training (survey non-responders). Categorization of ‘high’ and ‘low’ volume simulation use was arbitrarily defined based on the median number of reported pediatric simulation sessions in 2014 (5.5) for centers reporting simulation use.

Outcome Measures and Cohort Definitions

The primary outcome measure for this study was risk-adjusted mortality compared across levels of training. Secondary outcomes included time to critical interventions (endotracheal intubation), evaluations (head CT), and procedures (emergent craniotomy and surgery for hemorrhage control). For secondary analyses, ‘traumatic brain injury (TBI)’ was defined as any patient with an AIS head severity score ≥ 1, excluding scalp laceration or skull fracture codes. ‘Isolated TBI’ was defined as any patient with TBI with no other AIS severity score ≥ 2 other than facial injuries. ‘Emergent craniotomy’ was defined as any craniotomy in a patient that had an ED disposition of ‘Operating Room’ and only received one head CT before ED discharge. Frequency of endotracheal intubation (as a marker of resuscitation quality) and time to endotracheal intubation were assessed for all trauma patients with initial known GCS total ≤ 8, as well as for patients with confirmed TBI and initial known GCS total ≤ 8. Time to first head CT was assessed for isolated TBI patients with an initial known GCS total ≤ 8, and for polytrauma patients with TBI and an initial known GCS total ≤ 8. Time to emergent craniotomy was assessed for isolated TBI patients and for polytrauma patients with TBI. Time to surgery for hemorrhage control was assessed for any patient that received packed red blood cells within four hours of ED admission, had a surgical (non-angiographic) procedure for control of bleeding (laparotomy, thoracotomy, sternotomy, neck exploration, extremity exploration, skin or soft tissue operation, or mangled extremity procedure), and had an ED disposition of ‘Operating Room’.

Patients were excluded from analyses of time to intubation if they had prehospital intubation (defined by time to procedure variable as 0), tracheostomy (defined by ICD-10 procedure codes) performed in the emergency room, or endotracheal intubation performed after discharge from the emergency room. Patients were excluded from analyses of time to head CT if the scan time was the same as the patient arrival time or if the scan occurred after ED discharge. The location and timing of endotracheal intubation or head CT origination was determined using time to procedure variables and ED length of stay variables.

Confounding Covariates

The TQIP program uses a validated multivariable model for risk-adjusted benchmarking of mortality between pediatric trauma centers.²⁹ Covariates in this model include gender, race (white, black, Asian, other), age, comorbidities (respiratory diseases, substance abuse, major psychiatric illness, bleeding disorder, functional dependence, diabetes mellitus, hypertension, congenital anomalies, and prematurity), injury-specific survival risk ratios calculated and validated based on historic datasets, age-normalized initial ED systolic blood pressure, age-normalized initial ED heart rate, initial ED GCS motor score, maximum AIS severity scores by body region (head, face, neck, chest, abdomen, spine, upper extremity, lower extremity), mechanism of injury (fall, motor vehicle occupant, motorcyclist, struck by object, firearm, cut/pierce, pedestrian struck by bicycle, other), pre-hospital cardiac arrest, and interaction terms for AIS head by age, AIS head by infant, and systolic blood pressure by firearm injury. Trauma center state designation status, ACS verification level, and annual admission volume are specifically not included in the TQIP mortality model. We therefore performed sensitivity analyses including these covariates to assess for unmeasured confounding due to trauma center resources and due to trauma admission volume as a proxy for provider and team experience.

Adjusted analyses for all secondary outcomes included the same covariates in in the TQIP model in addition to several other specific factors. Additional covariates considered included ED respiratory rate, ED respiratory assistance, ED oxygen saturation, ED supplemental oxygen, and injury related to child abuse. Adjusted analyses for time to head CT, time to craniotomy, and time to surgery for hemorrhage control also included factors that may impact ED length of stay, including prehospital intubation, chest tube placement in the emergency room, transfusion in the emergency room, placement of a surgical airway in the emergency room, and CT of the abdomen and pelvis in the emergency room. Transfer-in patients were excluded from all secondary analyses.

Statistical Analyses

Baseline center characteristics assessed for differences across levels of simulation with omnibus tests of significance, including ANOVA for continuous variables and chi-squared test of independence for categorical variables. Missing physiological data for center-level mortality comparisons were managed using multiple imputation. The imputation model included age, gender, race, transfer status, the presence of a serious body region injury (AIS≥2 of the spine, abdomen, lower extremity, and upper extremity), injury mechanism (fall, firearm, motorcyclist, motor vehicle occupant, pedestrian, struck, and other mechanism), and age by vital interactions in the imputation model. Mortality was modeled using data from Fall 2017 TQIP report as outcomes in an hierarchical logistic regression model. Because of positive skew, resuscitation process times were normalized by log transformation. Missing data for multivariable time-to-event analyses were imputed by drawing from a random distribution with sample mean and standard deviation or from a binary distribution for proportions to minimize bias and preserve variability. To account for clustering of patients within centers, time-to-event outcomes were analyzed using hierarchical linear regression across levels of simulation-based training use. All significance tests were two-tailed, with α=0.05. All analyses were performed using SAS software v. 9.4 (SAS Institute Inc., Cary, NC).

RESULTS

One hundred twenty-five ACS TQIP-Pediatric trauma centers were surveyed (Table 1a). Survey response rate was 75% (N = 94/125 centers). One center that responded to the survey did not submit registry data for 2016 and was therefore not included in the clinical outcomes analysis, leaving 124 centers for our analysis sample. Simulation use in 2014-15 was reported in 54 centers (43% of all centers and 58% among respondents). Among centers reporting simulation use in 2014-15, 19 (15% of all centers and 20% among respondents) centers reported low volume simulation use (median [IQR] of 6 [3-8] sessions over two years). Thirty-five (28% of all centers and 37% among respondents) centers reported high-volume simulation use (22 [14-26] sessions over two years). High-volume simulation centers were found to have significantly lower overall trauma volume and annual trauma admissions with serious injury (ISS≥16). No significant differences in pediatric (age ≤ 14) admissions were observed across levels of training. Centers reporting any level of training were more likely to be an ACS-verified or state-designated level 1 pediatric trauma center and were more likely to have a pediatric intensive care unit.

Table 1a.

Center Characteristics by Level of Simulation Use.

	Total Number of Pediatric Simulation Sessions, 2014-2015				Omnibus p-value†
	Unknown	None	1-10	11+
Number of Facilities	30	40	19	35
Quantification of Clinical Trauma Volume
Annual Trauma Admissions	1,682.5 [873-2,690]	1,829.5 [1,233-2,460]	1,462 [747-2,847]	1,367 [776-1,987]	0.01
Annual Admissions with ISS ≥16, Excluding Transfers	297.5 [162-543]	346 [204.5-622.5]	289 [105-855]	182 [116-500]	0.02
Annual Admissions, Age ≤ 14yo	398.5 [169-607]	229.5 [155.5-588.5]	347 [261-555]	355 [185-922]	0.32
Annual Admissions, Age ≤ 14yo with ISS ≥16	57 [24-83]	37 [21.5-60]	54 [33-82]	46 [18-115]	0.82
Annual Admissions, Age ≤ 14yo with ISS≥16, excluding transfers	21 [10-34]	17 [11.5-31]	24 [11-33]	16 [10-47]	0.65
Hospital Teaching Status
University Hospital	19 (63.3)	25 (62.5)	13 (68.4)	26 (74.3)	0.75
Community Teaching Hospital	10 (33.3)	12 (30.0)	5 (26.3)	9 (25.7)
Nonteaching Hospital	1 (3.3)	3 (7.5)	1 (5.3)	0 (0.0)
Hospital Type
For Profit	3 (10.0)	3 (7.5)	1 (5.3)	0 (0.0)	0.28
Not for Profit	27 (90.0)	37 (92.5)	18 (94.7)	35 (100.0)
ACS Pediatric Verification
Level 1 Pediatric Verification	8 (28.6)	7 (18.0)	9 (47.4)	17 (50.0)	0.01
Level 2 Pediatric Verification	6 (21.4)	16 (41.0)	7 (36.8)	5 (14.7)
Not Verified for Pediatric Trauma	14 (50.0)	16 (41.0)	3 (15.8)	12 (35.3)
State Pediatric Designation
Level 1 State Designation	12 (40.0)	15 (37.5)	8 (42.1)	23 (65.7)	0.03
Level 2 State Designation	5 (16.7)	16 (40.0)	4 (21.1)	4 (11.4)
No State Designation	13 (43.3)	9 (22.5)	7 (36.8)	8 (22.9)
Patient Care Characteristics
Associated with a Pediatric Hospital	24 (80.0)	32 (80.0)	16 (84.2)	29 (82.9)	0.99
Have a Pediatrics Ward	30 (100.0)	39 (97.5)	19 (100.0)	35 (100.0)	1.00
Have a Pediatric Intensive Care Unit	28 (93.3)	40 (100.0)	19 (100.0)	35 (100.0)	0.08
Transfer severely injured children to other centers	5 (16.7)	3 (7.5)	1 (5.3)	1 (2.9)	0.25
Provide all Acute Care Services to Injured Children	29 (96.7)	37 (92.5)	18 (94.7)	35 (100.0)	0.40

Data expressed as median [IQR] or N(%).

^†One-way analysis of variance (ANOVA) for continuous variables and Fisher’s exact test for categorical variables.

The mortality analysis included 57,916 patients treated at 124 centers. There were statistically significant differences in most patient-level characteristics across levels of simulation use, but the differences were small (effect size ≤0.20) for all variables except mechanism of injury (Table 1b). Centers with a high-volume of simulation-based training use had significantly less motor vehicle crash occupants (14% versus 21-23%, p<0.01, d=−0.25) and significantly more patients treated after a fall (53% versus 41%, p<0.01, d=0.24) when compared to centers with a low volume of or no simulation-based training use.

Table 1b:

Demographic, Physiologic, and Injury Characteristics for Children Treated at Pediatric TQIP Centers by Level of Simulation Use.

	Unknown	No Sim Use	0-10 hours	11+ hours
Number of Facilities	30	39	19	35
Patients (All)	14,576	17,118	8,645	19,114
Transfer In	7,307 (50.1)	7,434 (43.4)	4,556 (52.7)	8,976 (47.0)
Patients (No Transfers)	7,269	9,684	4,089	10,138	p-value	effect size
Race - White	4263 (59.6)	6031 (65.7)	2296 (57.1)	6292 (64.1)	<0.01	≤0.18
Gender - Male	4619 (63.5)	6267 (64.7)	2620 (64.1)	6387 (63)	0.08
Age	8 (4-14)	9 (4-15)	10 (4-15)	8 (4-13)	<0.01	≤0.17
ED GCS Motor Score	6 (6-6)	6 (6-6)	6 (6-6)	6 (6-6)	<0.01	≤0.09
ED Systolic Blood Pressure	120 (110-132)	120 (110-132)	122 (111-134)	119 (108-130)	<0.01	≤0.20
ED Heart Rate	103 (88-121)	103 (88-121)	104 (88-123)	104 (88-121)	0.36	≤0.04
Prehospital Cardiac Arrest	62 (0.9)	83 (0.9)	29 (0.7)	51 (0.5)	<0.01	≤0.09
Comorbidity
Functional Dependence	18 (0.2)	45 (0.5)	14 (0.3)	35 (0.3)	0.11
Substance Abuse	194 (2.7)	302 (3.2)	169 (4.1)	172 (1.7)	<0.01	≤0.15
Congenital Anomalies	73 (1)	160 (1.7)	62 (1.5)	182 (1.8)	<0.01	≤0.06
Prematurity	111 (1.5)	162 (1.7)	60 (1.5)	173 (1.7)	0.62
Mechanism
Firearm	206 (2.8)	376 (3.9)	164 (4)	228 (2.2)	<0.01	≤0.10
Motorcyclist	42 (0.6)	121 (1.2)	40 (1)	60 (0.6)	<0.01	≤0.07
Motor Vehicle Crash - Occupant	1247 (17.2)	2069 (21.4)	962 (23.5)	1407 (13.9)	<0.01	≤0.25†
Cut/pierce	109 (1.5)	149 (1.5)	51 (1.2)	151 (1.5)	0.62
Fall	3495 (48.1)	4013 (41.4)	1661 (40.6)	5341 (52.7)	<0.01	≤0.24‡
Pedestrian	704 (9.7)	1105 (11.4)	429 (10.5)	1043 (10.3)	<0.01	≤0.06
Other	657 (9)	765 (7.9)	374 (9.1)	797 (7.9)	<0.01	≤0.05
Worst AIS Severity Score
Head	0 (0-2)	0 (0-2)	0 (0-2)	0 (0-2)	<0.01	≤0.19
Face	0 (0-0)	0 (0-1)	0 (0-0)	0 (0-0)	<0.01	≤0.10
Neck	0 (0-0)	0 (0-0)	0 (0-0)	0 (0-0)	<0.01	≤0.04
Chest	0 (0-0)	0 (0-0)	0 (0-0)	0 (0-0)	<0.01	≤0.18
Abdomen	0 (0-0)	0 (0-0)	0 (0-0)	0 (0-0)	<0.01	≤0.09
Spine	0 (0-0)	0 (0-0)	0 (0-0)	0 (0-0)	<0.01	≤0.04
Upper Extremity	0 (0-2)	0 (0-2)	0 (0-2)	0 (0-2)	<0.01	≤0.15
Lower Extremity	0 (0-2)	0 (0-2)	0 (0-2)	0 (0-2)	0.04	≤0.03
Severe TBI1	266 (3.7)	422 (4.4)	198 (4.8)	287 (2.8)	<0.01	≤0.10
Infant with Severe TBI1	13 (4.9)	10 (2.4)	6 (3.0)	10 (3.5)	0.35
Firearm Injury with Hypotension	3 (1.1)	10 (2.4)	5 (2.5)	8 (2.8)	0.53

Data expressed as frequency (%) or median (interquartile range).

¹AIS Head ≥3 and GCS≤8. Reported effect sizes represent the largest post-hoc pairwise difference between groups. Variables with effect size differences >0.20 are detailed as follows: Mechanism-MVC Occupant: unknown simulation versus no simulation, p<0.01, d=0.11; no simulation versus 0-10 hours, p<0.01, d=0.05; 0-10 hours versus 11+ hours of simulation, p<0.01, d=0.25. Mechanism-Fall: unknown simulation versus no simulation, p<0.01, d=0.13; no simulation versus 0-10 hours, p=0.37; 0-10 hours versus 11+ hours of simulation, p<0.01, d=0.24.

Average center-specific unadjusted mortality rate was lowest in centers with a high volume of simulation-based training use (Table 2). Centers using either low- or high-volume training had significantly lower mortality when compared to centers that do not use simulation. Additional adjustment for ACS trauma center verification level and for annual trauma admission volume did not impact these results. Using transfer patients within centers as internal controls, we did not see a significant impact of simulation use on risk-adjusted mortality for these patients that underwent initial resuscitation at a referring center (Figure 1). Adjusted odds ratios for covariates included in the multivariable mortality model are shown in Supplemental Table 1.

Table 2:

Center-Specific Mortality, Risk-Adjusted Center-Specific Mortality, and Sensitivity Analysis of Center-Specific Risk-Adjusted Mortality Including Additional Adjustment for ACS Pediatric Trauma Center Verification and for Annual Trauma Volume for Pediatric TQIP Centers by Level of Simulation Use.

	Total Number of Pediatric Simulation Sessions, 2014-2015
	Unknown	None	1-10	11+
	N=30	N=40	N=19	N=35
Unadjusted center-specific mortality rate	1.37%	1.88%	1.59%	1.03%
Risk-adjusted mortality (TQIP model)	0.80 [0.51-1.26]	Ref	0.55 [0.32-0.96]	0.58 [0.37-0.92]
Risk-adjusted mortality with additional adjustment for ACS verification level and annual trauma volume	0.80 [0.51-1.27]	Ref	0.56 [0.33-0.94]	0.65 [0.41-1.03]

Data presented as percentages or odds ratio with 95% confidence interval. TQIP: Trauma Quality Improvement Program.

Figure 1:

Risk-Adjusted Mortality in Transferred and Non-Transferred Trauma Patients Treated at Centers Using No Simulation, Low-Volume (0-10 hrs) Simulation, High-Volume (11+ hrs) Simulation, and Unknown Simulation Use (Survey Non-Respondents).

We observed no significant difference in time to intubation, head CT, emergent craniotomy, or surgery for hemorrhage control between centers of differing levels of simulation-based training use (Table 3). Trauma patients with an initial ED GCS total ≤ 8 had a slightly higher frequency of intubation at centers using high-volume training compared to centers not using simulation-based training (94% versus 91%, p=0.02, Table 4).

Table 3:

Resuscitation Process Times for Centers of Varying Levels of Simulation Use and Adjusted Odds Ratios Comparing Centers Using High-Volume Simulation to Centers Using No Simulation.

	Unknown Simulation		No Simulation		Low Volume Simulation		High Volume Simulation		Univariate	Multivariable
	N	Mean [95% CI]	N	Mean [95% CI]	N	Mean [95% CI]	N	Mean [95% CI]	Omnibus p-value	Adjusted OR [95% CI]
Time to Endotracheal Intubation
All Patients with GCS 8 or less	111	8 [7-10]	175	9 [8-11]	83	9 [7-11]	117	10 [8-12]	0.44	1.00 [0.80-1.26]
Isolated TBI with GCS 8 or less	41	9 [6-11]	48	8 [6-11]	21	8 [5-11]	33	12 [8-16]	0.31	0.80 [0.55-1.16]
Time to Head CT
All Patients with GCS 8 or less	183	21 [18-26]	296	27 [23-33]	136	27 [21-34]	220	25 [21-29]	0.30	1.17 [0.91-1.50]
Isolated TBI with GCS 8 or less	53	21 [16-27]	68	23 [18-29]	38	22 [16-29]	61	24 [19-31]	0.84	1.0 [0.74-1.4]
Time to Emergent Craniotomy
All patients	43	107 [91-128]	83	128 [113-146]	35	113 [93-136]	53	128 [109-150]	0.32	1.07 [0.84-1.35]
Isolated TBI	24	107 [85-135]	33	122 [101-148]	21	117 [92-148]	28	124 [101-152]	0.80	0.92 [0.72-1.17]
Time to Surgery for
Hemorrhage Control	55	70 [54-90]	101	53 [43-66]	54	51 [39-68]	70	61 [48-76]	0.31	0.97 [0.64-1.21]

Table 4:

Frequency of Endotracheal Intubation for Pediatric Trauma Patients that Present with a Glasgow Coma Scale of 8 or Less by Trauma Center Level of Simulation Use.

	Total Number of Pediatric Simulation Sessions, 2014-2015
	No Survey	No Simulation	1-10	11+	TOTAL	Omnibus p- value†
N	784	932	486	882	3,084
Not intubated	72 (9.1)	84 (9.0)	55 (11.3)	53 (6.0)	264	0.02
Intubated	712 (90.8)	848 (91.0)	431 (88.7)	829 (94.0)	2,820

Data expressed as N(%).

^†Chi-square test of independence.

DISCUSSION

This retrospective study of simulation-based training use in ACS Pediatric TQIP centers found an association between increased use of simulation-based training and lower risk-adjusted mortality for acutely injured children. We did not find a difference in time to critical interventions, evaluations, or procedures, but found an increased rate of intubation for patients with a GCS ≤ 8 in centers with a high volume of training. These findings suggest that simulation-based training may improve resuscitation quality and outcomes in pediatric trauma patients, but improved outcomes may not be mediated by faster time to critical interventions, evaluations, and procedures.

Simulation-based training has been shown to improve trauma team performance, mainly measured by number of tasks completed and time to task completion in acute resuscitation.^18-22 Few studies have found an association between training with simulation and improved trauma resuscitation performance in a real-world setting.^19,21,27 Simulation-based training has been associated with faster times to critical procedures and evaluations, including time to endotracheal intubation, time to head CT, and time to the operating room in experimental studies using this methodology as an educational intervention.²¹ Systematic implementation of simulation-based training for all providers in a trauma center led to faster time to critical operations after implementation.²⁷ In addition to faster resuscitation processes, simulation-based training has also been shown to decrease missed critical steps during trauma resuscitation.¹⁹ While none of these single-center studies demonstrated a mortality benefit from training with simulation, they all postulate benefit from improved resuscitation times and faster time to critical procedures.

Based on our literature review, we hypothesized that simulation-based training would be associated with a more efficient resuscitation with shorter times to critical evaluations and procedures, but we found no difference in time to intubation, head CT, emergent craniotomy or emergent surgery for hemorrhage control. Conversely, we did demonstrate an association with simulation use and decreased risk-adjusted mortality, which brings into question the mechanism by which training with simulation may improve mortality. Our findings suggest that earlier performance of these procedures is not necessarily the only factor associated with lower mortality and are consistent with other evidence where quality of care for severely injured patients might be of greater value than small differences in time to intervention.³⁰ Faster resuscitation time has been associated with improved survival,³¹ while early intubation¹³ and faster time to laparotomy^12,14,15 have been shown to improve outcomes. Among patients with severe TBI requiring craniotomy, however, the association between time to surgery and outcome is not certain.^{12,16,17,32-34} The outcome of some reversible clinical scenarios, such as hemorrhagic shock, may be more dependent on rapid intervention.^14,15,35 We did not find an association between the use of simulation-based training and time to surgery for hemorrhage control, but the median time to laparotomy was much longer than observed by previous adult data (10-36 min).^14,15 This longer time to surgery for hemorrhage control in our cohort suggests that many of the patients we defined as ‘emergent’ may not have truly had immediate life-threatening hemorrhage, and we would not expect simulation-based training to have a profound impact on less urgent operative times. This longer time to laparotomy may also be a pediatric center-specific effect, with a greater emphasis on non-operative management. Furthermore, using our definition of ‘emergent craniotomy’, we demonstrated times to OR of approximately two hours – again questioning the true emergent nature of these operations, as one would expect a “crash” craniotomy to be in the operating room in 30-60 minutes from arrival to the trauma bay. In the absence of more granular physical exam findings and GCS trends over time, we are not able to better define a cohort of ‘emergent’ craniotomy patients using the TQIP dataset which may have led to our lack of demonstrating an impact on time to critical operations.

The survival advantage found in our study may be attributed to center-specific factors that are not measured in this study. While we have attributed the impact on mortality to simulation-based training, the use of simulation may alternatively serve as a proxy for other factors such as organizational culture that embraces teamwork, communication, and quality improvement. These latter factors may directly improve outcomes in the centers that have adopted simulation use. The use of simulation may also be a marker for more institutional resources, as the use of simulation requires significant financial investment. Children treated at pediatric trauma centers may have improved mortality compared to those treated at adult trauma centers,^4,6,7,36 but we found no impact of center verification level (as a marker for resources) on the association of simulation use with improved mortality. Centers that use a higher-volume of simulation may also be more likely to adopt evidence-based practices. We attempted to control for these unmeasured factors using transfer-in patients from the same institutions as internal controls, assuming that hospital-wide factors would have an impact on both acutely resuscitated patients and transferred-in patients equally. We only saw an association with a mortality benefit in acutely resuscitated patients and not in transfer-in patients that were resuscitated at referring institutions – suggesting that whatever factor we are measuring does lead to some improvement in the initial resuscitation.

There are several limitations to this study. Our analysis was limited to centers that participate in ACS Pediatric TQIP – centers that, by definition, have significant resources. Our findings, therefore, may not be generalizable to all trauma centers. We would expect, however, the impact of simulation would be larger in lower-resourced centers that have more opportunity for improvement in initial resuscitation practices. Response bias may have contributed to the results as the frequency of simulation use was based on a survey in which we had a 25% non-response rate. Outcomes in non-response centers were similar to those in centers that do not use simulation suggesting the effect size may indeed be larger than we have shown. We accounted for this limitation by including non-responders as a separate category in our multivariable analysis. Outcomes in non-response centers were similar to those in centers that do not use simulation, suggesting the effect size may indeed be larger than we observed.

Our findings are inherently subject to issues common to a retrospective design, most prominent of which is that the level of simulation-based training was not randomly assigned. High-volume simulation centers had lower annual trauma admission volumes, suggesting the use of simulation-based training may be an attempt to supplement provider and team experience in the presence of lower clinical volumes. We noted minor differences in patient demographics and injury characteristics and significant differences in mechanism of injury between trauma centers using differing levels of simulation-based training. These variables were all adjusted for in the multivariable model, which should limit the impact of these differences. The cross-sectional design limits our ability to assess the impact of simulation on individual programs over time. Low-volume and lower-performing centers may have implemented simulation-based training to address inefficiencies in resuscitation. The lack of difference in time to procedures, therefore, may reflect a true improvement from baseline at these centers. Unmeasured confounding likely remained despite controlling for known independent predictors of mortality. For instance, we could not measure change in condition over time during the initial resuscitation. An initial GCS of 8 may rapidly improve during resuscitation and thus obviate the need for intubation. Experienced providers may elect to avoid intubation for a brief period to assess for response, which may provide an alternative explanation of the differences in intubation frequency and in the less than 100% intubation frequency for GCS of 8 or less. It also should be noted that ‘high volume’ simulation was arbitrarily defined for this study as over ten simulated scenarios in two years, and that median frequency of simulation use in ‘high-volume’ centers was 22 sessions over two years – or slightly less than once per month. While the ideal frequency with which teams should undergo simulation-based training is unknown, the expected effect size from this infrequent use may be modest.

CONCLUSIONS

Risk-adjusted mortality is lower among children treated at pediatric TQIP centers that use simulation. Whether this effect is directly attributable to simulation use or to a center-level factor that simulation-based training use serves as a proxy for remains unknown. Simulation use was not associated with faster time to critical evaluations and procedures, but intubation frequency for patients with low GCS was higher in centers using high-volume simulation, suggesting that quality (or fewer missed steps) may be a greater mediator of mortality than the speed of the resuscitation or its critical components. Prospective evaluation of simulation-based training within trauma centers is needed to show a casual improvement in mortality over time, as is further delineation of the mechanisms by which simulation contributes to a clinical benefit for trauma patients.