Table 2.

Included studies with Kirkpatrick level 4 outcomes (patient outcomes)

Primary author (Year)	Clinical context and participants	Study design	Results	Conclusions
Capella et al. (2010)*	Trauma (114; 28 surgery residents, 6 faculty surgeons, 80 ED nurses)	Uncontrolled Before-and-After study; Pre/post training intervention study design Compared clinical outcome and efficiency of care pre and post team training	A significant decreased time was observed after the training for: arrival to CT scanner (26.4-22.1 min, P = 0.005), endotracheal intubation (10.1-6.6 minutes, P = 0.049), and operating room (130.1-94.5 min, P = 0.021) No significant difference was observed after the training for: intensive care unit LOS (5.5-6.3 days, P = 0.445), hospital LOS (7.6-6.3 days, P = 0.210), absence of complication rate (70.5-76.8, P = 0,113), and survival rate (86.9-91.5, P = 0,121) and times from arrival to FAST examination (8.3-9.6 min, P = 0.131) and time in the ED (186.1-187.4, P = 0.93)	Structured trauma resuscitation team training augmented by simulation resulted in improved efficiency of patient care in the trauma bay
Steinemann et al. (2011)*	Trauma (137; 9 staff surgeons, 21 staff ED physicians, 24 residents, 3 physician assistants, 44 RNs, 23 RTs, 13 technicians)	Uncontrolled Before-and-After study; Pre/post training intervention study design compared clinical outcome pre and post team training during actual trauma resuscitations	Significant improvements in speed (reduction by 16% of mean overall ED resuscitation time, P < 0.05) and completeness of resuscitation (76% increase in the frequency of near-perfect task completion, P < 0.001) The mortality rate, mean ICU and hospital LOS were not significantly different before and after the training all P > 0.05)	A relatively brief (4 hr) simulation-based curriculum can improve clinical performance and patients’ outcomes
Riley et al. (2011)	Obstetrics and perinatal (134 from 3 hospitals; 13 obstetricians, 23 family practitioners, 14 pediatricians, 65 registered nurses, 18 certified RNs anesthetist, 1 physician assistant	RCT – Cluster randomization of hospitals Randomized to simulation-based, didactic-based, or no intervention. Groups were compared using clinical outcome scores.	A statistically significant and persistent improvement of 37% (P < 0.05) in perinatal morbidity was observed between the pre- and post-intervention for the hospital exposed to the simulation program. There were no statistically significant differences in the didactic-only or the control hospitals (P > 0.05). No significant change in the perception of culture of safety (P > 0.05) at the three hospitals	Interdisciplinary in situ simulation training is effective in decreasing perinatal morbidity and mortality for perinatal emergencies Didactics alone were not effective in improving perinatal outcomes
Andreatta et al. (2011)	Pediatrics resuscitation (228, junior and senior pediatric medicine resident with code team members: RNs, medical students, pediatric hospitalists, pharmacists)	Longitudinal cohort study for 4 years Observed patients’ outcome as the training occurred over several years	After the routine integration of the formal mock code program into residency curriculum, resuscitation survival rates significantly increased from 33% to 50% within 1 year, in increments that correlated with the increasing number of mock code events (r =0.87) and held steady for three consecutive years	Simulation-based mock codes can provide a sustainable and transferable learning context for advanced clinical training and assessment that ultimately decreased mortality for pediatric resuscitations
Phipps et al. (2012)	Obstetrics and perinatal (~185; obstetricians, perinatologists, labour and delivery RNs, certified nurse midwives, anesthesiologists, certified RN anesthetists, resident physician /fellows)	ITS - Patient outcomes were assessed using data collected quarterly for 8 quarters prior to initiating the program and for the 6 quarters after implementing the program hospital wide, multidisciplinary simulation -based CRM intervention was applied to assess clinical outcome data collected 8 quarters pre-interruption and 6 quarters post interruption.	AOIs significantly decreased from 0.052 (95%CI: 0.048 to 0.055) at baseline to 0.043 (95%CI: 0.04 to 0.047). Overall, the frequency of event reporting and the overall perception of safety did not change significantly. No change in patient perception but were satisfied > 90% even before the intervention	Using the combination of a didactic and simulation-based CRM training was noted to improve patient outcomes

AOI = adverse outcome index; CT = computed tomography; CI = confidence interval; CRM = crisis resource management; ED = emergency department; ICU = intensive care unit; ITS = interrupted time series; LOS = length of stay; FAST = focused assessment with sonography for trauma; MD = medical doctor; RCT = randomized controlled trial; RN = registered nurse; RT = respiratory therapist

*Study includes both Kirkpatrick Level 3 and Level 4 outcomes. Please see Table 1 for details on Kirkpatrick Level 3 outcomes