Abstract
Objective
Massive hemorrhages (MHs) are rare but serious complications of pediatric trauma and obstetric cases. This study aimed to evaluate the impact of interprofessional simulation to improve adherence to a MH protocol (MHP), teamwork skills and confidence levels during a hemorrhagic crisis situation.
Methods
This was a pre‐post experimental study conducted at a tertiary care mother–child simulation center. Pediatric emergency and obstetric teams were submitted to simulated trauma and postpartum MH scenarios. Training consisted of two case scenarios followed by debriefing sessions and a lecture on the MHP. The primary outcome was adherence to MHP processes (checklist) measured prior to and 2 weeks following training sessions. Other outcomes were the measure of teamwork skills (Mayo High Performance Teamwork Scale) and confidence of the participants.
Results
Sixty‐two health care professionals were involved in eight interprofessional teams. Mean scores for adherence to the MHP improved from 19.1 in the pretraining phase to 25.8 in the posttraining phase (difference of 6.7; 95% confidence interval [CI] = 4.4 to 8.9). Mean scores pertaining to teamwork skills also improved significantly between pre‐ and posttraining phases (difference = 3.9; 95% CI = 1.5 to 6.4). Confidence questionnaires showed significant improvements in the posttraining phase (difference = 6.9; 95% CI = 5.3 to 8.3).
Conclusions
Targeted training involving simulation and protocol review improved participant adherence to MHP processes and teamwork skills. Confidence levels improved across all disciplines.
Massive hemorrhage (MH) is defined as blood loss exceeding one or more circulating blood volumes within 24 hours or rapid blood loss of more than 50% of circulating volume. 1 This condition may arise from traumatic, surgical, or obstetric complications. 2 During MH, blood volume loss combined with decreased hemoglobin potentiate shock state and cellular hypoxia, making MH a time‐critical emergency. 3 As in all resuscitations, prompt medical and surgical management is required along with crisis resource management (CRM) skills. The latter refers to optimization of teamwork and behaviors during crisis situations. 4 Furthermore, efficiently managing distribution and administration of various blood products is mandatory to ensure successful outcomes. MH protocols (MHPs) allow for better coordination between the team caring for the patient and the blood bank, leading to timely and sustained access to blood products that are distributed in predetermined ratios. 5 , 6 , 7
Audits of cases 8 months after MHP implementation in our center revealed difficulties in launching and applying the MHP (personal data). It is possible that, despite initial MHP implementation strategies, the fact that MH cases happen infrequently provides little opportunity for professionals to put knowledge into practice. Simulation represents an interesting tool to facilitate protocol implementation and has been proven to increase adherence to critical procedural protocol. 8 Thus, we used high‐fidelity simulation to facilitate a structured observation of health care professionals managing MH events. The primary objective of this study was to evaluate adherence to MHP and teamwork skills using simulation in a mother–child center offering tertiary and quaternary care in pediatric and obstetric medicine.. Secondary objectives were to determine confidence levels of various team members when applying the MHP and fulfilling their role within an interprofessional team during a simulated crisis situation. We hypothesized that educational material provided would improve adherence to MHP as well as teamwork skills and confidence levels of participants in fulfilling their roles in a simulated environment.
METHODS
Design
This was an experimental pre–post prospective study conducted in a simulated environment to assess the application of the MHP and interprofessional teamwork while caring for a patient with MH. The institutional research ethics board approved this project and written informed consent was obtained from all participants prior to enrollment.
Setting
The study was conducted at a tertiary care mother–child hospital, a designated Level 1 pediatric trauma and high‐risk maternity center in Montreal, Canada. Simulated cases were deployed in the institution's hospital‐based mother–child simulation center and in situ at the hospital blood bank.
Participants
A convenience sample of health care professionals (physicians, nurses, respiratory therapists [RTs], and orderlies) working at the pediatric emergency department, the pediatric intensive care unit or obstetric department were selected as participants. These were grouped into four interprofessional teams. Each team was composed of two nurses, one RT, one orderly, and two physicians (faculty members and fellows) who acted as team leaders. Participants were excluded if they were unable to attend both phases of the study. Hematology teams were composed of blood bank technologists and pediatric hematologists. They collaborated during simulated cases when blood bank technologists were notified by a resource person when a MHP was launched. Blood bank technologists prepared mock blood products according to their standard procedures. Pediatric hematologists assisted both the blood bank technologists and the clinical team at bedside upon request, answering questions over the phone.
Intervention
The intervention of interest consisted of 1) a high‐fidelity simulation session, 2) a brief oral presentation given by the head of the transfusion medicine committee at our institution summarizing the physiologic concepts and the application of the MHP, and 3) a key article related to the integration of CRM in simulated emergency situations. 9 The first simulation served as the evaluation of the pretraining. The second simulation session held 2 weeks later was aimed at evaluating performances following the intervention (posttraining). Simulation sessions were divided into four acute cases of MH (two obstetric postpartum hemorrhage and two pediatric trauma cases according to the involved participants; see Data Supplement S1, Appendix S1, available as supporting information in the online version of this paper, which is available at http://onlinelibrary.wiley.com/doi/10.1002/aet2.10513/full).
Obstetric postpartum hemorrhage cases occurred in an environment reproducing an operating room and used a maternal and neonatal birthing simulator (Noelle, Gaumard Scientific Company). Pediatric trauma cases were held in an environment reproducing the trauma bay and used two mannequins, (Noelle, Gaumard Scientific Company; and SimBaby, Laerdal Corporation). Mannequins were selected based on their features that rendered them able to physiologically simulate a hemorrhagic shock.
To enhance the level of physical and emotional realism, equipment was set up according to the clinical settings in both the emergency and the operating rooms. Mock blood products were physically handed to the runner to be transported from the blood bank back to the simulation center.
Content‐expert simulation instructors developed simulated scenarios for each field to ensure accuracy in case presentations, progression, and likely responses according to team interventions, actions, and behaviors. To ensure that scenarios were delivered in a uniform fashion, they were scripted in detail, programmed in advance, and pilot tested to be ran in identical fashion by the same simulation field experts within each discipline. The settings allowed transition from one clinical state to another based on participant interventions, but to avoid large differences in duration of each scenario, clinical cases were aborted after 12 minutes. Simulated cases were under direct observation by simulation experts in addition to being captured on video to assist the debriefing process and to subsequently rate team performances.
Each scenario was followed by a 20‐minute scripted debriefing session. Two physicians, field experts with training and experience in debriefing, facilitated debriefing sessions. For each session, the same transfusion medicine specialist was present.
Educational objectives were elaborated prior to the session with a focus on recognition and management of hemorrhagic shock, optimization of the MHP application, and key CRM elements such as teamwork skills including role responsibility, communication, and reference to cognitive aids. 4
Outcomes
The primary outcome was the scores from the MHP checklist (see Data Supplement S1, Appendix S2). The MHP checklist is a task‐based scoring instrument comprising 15 items (not done [0 point] or need improvement [1 point], well‐done [2 points]; total maximum of 30 points) derived from the institution's original protocol elaborated and conceived by a committee of experts according to evidence‐based literature. This instrument was designed to measure adherence to the MHP and is generalizable to any obstetric or pediatric MH situation.
Secondary outcomes included team work performances as measured using the Mayo High Performance Teamwork Scale (see Data Supplement S3, Appendix S3) and participant confidence. The Mayo High Performance Teamwork Scale is a 16‐item scale designed to provide a measure of teamwork skills that are part of CRM training in medical settings and has demonstrated validity evidence. 10 Each item is rated on a 3‐point scale indicating how often the behavior occurred in the simulation scenario (never or rarely, inconsistently, consistently).
Both MHP checklists and Mayo scales were filled by raters upon reviewing videos from simulation cases. Individual participant confidence levels during acute crisis were evaluated via the participant confidence questionnaire that was completed prior to the first simulation session and following the second simulation session 2 weeks later (see Data Supplement S4, Appendix S4). The 11 items surveyed, using a 5‐point Likert scale, addressed participant confidence when applying the MHP and when taking a role within an interprofessional team. All questionnaires were coded by discipline and anonymous to ensure participant confidentiality.
Raters
The MHP checklist and Mayo High Performance Teamwork Scale were measured in duplicate by raters who were pediatric critical care or emergency subspecialists (pediatric trauma scenarios) and anesthesiologists or obstetricians (postpartum hemorrhage scenarios) selected because of their credentials as experienced field specialists and simulation facilitators.
They were not involved in the study, in the elaboration of the MHP or in the development of the MHP checklist. Principal investigators trained raters by reviewing items of each instrument and discussing scoring. Once training was completed, raters were divided into two pairs, one for each MH scenario category (pediatric trauma and postpartum MH). They scored scenario video recordings. Each rater was responsible for the same scenario pair for all teams, during the pre‐ and posttraining phases. All raters were blinded to the phases of the study but were not blinded to the identity of the subjects because they were from the same academic center given the language restriction. The final score between raters was an average of both rater scores; there was no consensus between raters.
Data Analysis
For this experimental prospective study, we used a convenience sample size comprising eight teams. Assumption for normality was checked (Shapiro‐Wilk) and when verified a paired t‐test was performed to compare pre‐ and postintervention values. A two‐sided nominal value of 0.05 was used for all comparisons and a conservative Bonferroni correction was applied to account for multiplicity. Results were expressed as mean, standard deviation, and paired mean differences for the pre–post comparison; median and interquartile range (IQR) were also reported. Intraclass correlation coefficient (ICC) was estimated to assess inter‐rater reliability for both MHP management and teamwork performances. All analyses were performed by a statistician using SAS statistical software Version 9.3 (SAS Institute, Cary, NC).
RESULTS
Participants
A total of 62 health care professionals involved in eight interprofessional teams (four emergency/trauma/critical care and four obstetrics/anesthesia) participated in the study (Figure 1). All participants were involved in both pre‐ and posttraining simulations and therefore represented their own control. However, team member compositions may have occasionally differed between both phases.
Figure 1.
Participant distribution. MHP = massive hemorrhage protocol.
Adherence to MHP
Mean scores for adherence to MHP improved from 19.1 in the pretraining phase to 25.8 in the posttraining phases, with a mean score difference of 6.7 (95% confidence interval [CI] = 4.4 to 8.9, p < 0.0001) and a median difference of 7.0 (IQR = 4.5–9.5). All individual items improved between pre‐ and posttraining phases Figure 2 but the only statistical difference was for item 4, “The team asks for the designated massive hemorrhage box” (difference = 1.4, 95% CI = 0.8 to 2.0).
Figure 2.
Pre‐ and posttraining MHP checklist scores. MHP = massive hemorrhage protocol.
Teamwork Skills
Mean scores pertaining to teamwork skills also improved significantly between pre‐ and posttraining phases (23.8 vs 27.7); the mean score difference was 3.9 (95% CI = 1.5 to 6.4, p = 0.004) and the median difference was 3.2 (IQR = 0.8–4.4). All items significantly improved between pre‐ and posttraining phases, except for item 14 (Figure 3).
Figure 3.
Pre‐ and posttraining Mayo Teamwork Scale Scores.
Confidence in Applying MHP and Work in Teams
Results from participant confidence questionnaires also showed significant improvements in the posttraining phase (Figure 4). The mean score difference between the pre‐ and posttraining phases was 6.9 (95% CI = 5.3 to 8.3, p < 0.0001), and the median difference was 7 (IQR = 3–12).
Figure 4.
Pre‐ and posttraining confidence scores.
Inter‐rater Reliability
As mentioned, all simulations were evaluated in duplicate. The inter‐rater reliability was excellent for the MHP checklist with an ICC of 0.69 (95% CI = 0.13 to 0.92) for the pretraining phase and an ICC of 0.9 (95% CI = 0.64 to 0.98) for the posttraining phase.
DISCUSSION
This simulation study demonstrated participant improvements in adherence to the MHP, teamwork skills, and overall confidence in applying the MHP following our pedagogical intervention. A similar study focusing on the management of MH cases in various contexts demonstrated improvements in knowledge and confidence in MH management among participants. 11 Their evaluation was based on participant questionnaires, which was similar to our assessment of participant self‐confidence. However, our study was enhanced by including objective assessment of knowledge and teamwork skills. Participants in the study by Green and Curry 11 were trainees, whereas we included working professionals involved in interprofessional teams. A recent in situ simulation study involving pediatric critical care and blood bank health care professionals aimed to assess team training and principles related to massive transfusion protocol following spinal fusion surgery12. Team‐based simulation provided participants with an opportunity to become familiar with massive transfusion protocol, implement changes, and increase learner confidence.
Our study revealed improvements in MH checklist scores between pre‐ and posttraining phases. The MH checklist was available to use as a cognitive aid. Hilton et al. reported that cognitive aids could be used prospectively or retrospectively to ensure completion of all steps of any protocol 13 . In his study, 86% of multidisciplinary teams used the checklist when managing a severe postpartum hemorrhage compared to 100% of teams in our study. This difference could be explained by participants’ awareness of the checklist existence and knowledge of its content. Participants involved in our study knew that the project targeted adherence to the MH checklist and they were given the opportunity to access it prior to training. In contrast, training in the study by Hilton et al. 13 was geared toward the benefits of using checklists during critical obstetric events and used a shoulder dystocia checklist as an example. Therefore, none of the teams received or viewed the actual postpartum hemorrhage checklist prior to simulation.
To optimize checklist adherence, designating a cognitive aid reader improves management of simulated critical events and reduces diagnostic errors. 14 In our study, the reader was referred to as the MHP resource person. A critical step of MHP at our institution is to dedicate this role to a single team member whose sole responsibility within the team is to be the reader. Thus, a resource person was identified in all simulations but tasks related to this role was better understood during the posttraining phase. Further analysis revealed that the item that reached significant improvement was item 4, “The team asks for the MH box.” This refers to a plastic box containing a paper version of the MHP, prewritten orders for calcium bolus, laboratory and blood bank requests, and predetermined collecting tubes. This box is available on every unit where a MH case is susceptible to occur. Emphasizing MH box existence, content and location during teaching sessions proved beneficial. Requesting the MH box early led to prompt protocol access and to completion of critical steps. Another benefit from debriefing sessions following simulated cases was to gather qualitative feedback in order to improve the MHP itself. Namely, we added a front sheet to the protocol reminding professionals to seek the MH box available on each unit.
Results from our study showed significant improvements in mean scores of all items of the Mayo High Performance Teamwork Scale between pre‐ and posttraining phases. In fact, the item that improved the most is item 8, “All members of the team are appropriately involved and participate in the activity.” Based on existing teamwork competency and behavior framework, this item refers to “role responsibility” illustrated by role assignment, workload distribution, recognition of performance limitation, and monitoring actions of other team members. 15 Many factors could have contributed to this improvement: observation and documentation of teamwork behaviors during the prephase, targeted debriefing specific to teamwork skills including role responsibility and communication, and participant access to a key article related to CRM. 9 In addition, team performances improved despite the fact that team composition varied between both phases of our study, highlighting knowledge acquisition and transfer. According to Kirkpatrick's model of knowledge acquisition, these represent the first steps toward improvements in actual clinical settings and impact on positive patient outcome. 16 Few studies have been successful in documenting favorable patient outcome following simulation‐based team training. 17 , 18 , 19 At our institution, MH management and teamwork training led to a positive clinical outcome a few weeks after our study was conducted. A 36‐year‐old gravida 2, para 2 suffered cardiac arrest 48 hours post–caesarean section secondary to a massive pulmonary embolism. She underwent intravenous thrombolysis and she was successfully resuscitated. Within 2 hours of the initial cardiac arrest, she presented massive bleeding both vaginally and from the surgical site. Launching of the MHP and subtotal hysterectomy led to a successful postoperative stabilization. 20
Mean scores related to confidence of participants to apply MHP and work in teams improved significantly between pre‐ and posttraining phases. Similar simulation studies have shown that interprofessional team training improves confidence of team members during current or future crisis situations. 21 , 22 Similar to our study, Allan et al. 21 , 22 and Figueroa et al. 21 , 22 sought input from all team members, including physicians, nurses, RTs, and allied health professionals.
The highest score difference within the confidence questionnaire was noted for item 3, “Respect all MH protocol steps.” This is not surprising given that MH management steps are outlined in the actual MHP used in clinical settings. Given that our primary objective was to evaluate adherence to MHP, every step of the study was geared toward better usage of this cognitive aid. Indeed, the goals of using MHP were reviewed; the tool was disseminated to participants and was made available during simulated scenarios. Finally, focused debriefing was provided on the benefits of using cognitive aids during crisis situations. A recent review emphasizes the positive impacts of applying cognitive aids to improve provider satisfaction, adherence to evidence‐based care and, ultimately, patient safety and outcomes. 23
Our study has many strengths. First, our project was appealing to health care professionals because it accessed and reinforced prior knowledge related to MHP and its application. Second, the interprofessional nature of our research team led to active recruitment of participants from each specialty and discipline who may potentially be involved in managing MH cases in a mother–child center. Third, the same teams of instructors were present for all sessions. Consequently, the same experienced simulation experts within each discipline and specialty ran all scenarios in identical fashion and deployed debriefing sessions for all scenarios, thus decreasing possible biases resulting from running scenarios differently or individual differences in debriefing styles. Finally, our study is original because blood bank technologists and hematologists were involved in situ.
LIMITATIONS
Study limitations include the possible selection bias of having volunteer participants. However, this should be minimal considering that each participant was his or her own control. Another limitation is our sample size of eight teams due to simulation constraints. Even though it was small, the sample size permitted to demonstrate a difference. Also, the total number of teams could be misleading because it represented more than 60 participants.
In addition, the order of the assessment scenarios was not randomized. However, because scenarios were similar in difficulty, this led us to believe that improvements between the pre‐ and posttraining sessions truly represented the effect of our pedagogical intervention. Last, it would have been interesting to document knowledge and skill retention or long‐term effects of our training.
CONCLUSIONS
Targeted training involving simulated scenarios and protocol review improved participant adherence to MHP and teamwork skills. Confidence levels improved among participants from all disciplines. We provided a model for a pedagogical intervention combining simulation and didactic sessions that may be generalizable to train teams for other high‐risk low‐frequency events. Future research should focus on investigating MHP application in the clinical settings, linking performances to patient outcome.
Supporting information
Data Supplement S1. Supplemental material.
ACKNOWLEDGEMENTS
This work was supported by the Saint‐Justine Mother‐child Simulation Centre who provided technical support and free use of the simulation center for the study. The authors are very grateful for the support and involvement of Rachel Fisher, Catherine Farrell, Marisol Sanchez, and Jacques Bourque as raters in the study as well as Noémie Elfassy for data entry. The authors give special thanks to Bianca Brunet and Anne‐Marie Girouard for coordinating all the work at the blood bank and their support to the blood bank technologists and to Marie‐Pierre Pelletier for her pivotal role in transfusion medicine safety. The authors thank to Thierry Ducruet who carried out the initial analyses.
AEM Education and Training 2021;5:1–9
The authors have no relevant financial information or potential conflicts of interest to disclose.
REFERENCES
- 1. Rossaint R, Bouillon B, Cerny V, et al. Management of bleeding following major trauma: an updated European guideline. Crit Care 2010;14:R52. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Young PP, Cotton BA, Goodnough LT. Massive transfusion protocols for patients with substantial hemorrhage. Transfus Med Rev 2011;25:293–303. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Cannon JW. Hemorrhagic shock. N Engl J Med 2018;378:370–9. [DOI] [PubMed] [Google Scholar]
- 4. Cheng A, Donoghue A, Gilfoyle E, Eppich W. Simulation‐based crisis resource management training for pediatric critical care medicine: a review for instructors. Pediatr Crit Care Med 2012;13:197–203. [DOI] [PubMed] [Google Scholar]
- 5. Fuchs KM, Miller RS, Berkowitz RL. Optimizing outcomes through protocols, multidisciplinary drills, and simulation. Semin Perinatol 2009;33:104–8. [DOI] [PubMed] [Google Scholar]
- 6. Nunez TC, Young PP, Holcomb JB, Cotton BA. Creation, implementation, and maturation of a massive transfusion protocol for the exsanguinating trauma patient. J Trauma 2010;68:1498–505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7. Mitra B, O'Reilly G, Cameron PA, Zatta A, Gruen RL. Effectiveness of massive transfusion protocols on mortality in trauma: a systematic review and meta‐analysis. ANZ J Surg 2013;83:918–23. [DOI] [PubMed] [Google Scholar]
- 8. Peltan ID, Shiga T, Gordon JA, Currier PF. Simulation improves procedural protocol adherence during central venous catheter placement: a randomized controlled trial. Simul Healthc 2015;10:270–6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Ghazali AB, Scépi M, Mimoz O, Oriot D. La simulation: du Task‐Trainer au Crisis Resource Management, un défi pédagogique pour la médecine d’urgence. Ann Fr Med Urgence 2012;2:384–92. [Google Scholar]
- 10. Malec JF, Torsher LC, Dunn WF, et al. The Mayo high performance teamwork scale: reliability and validity for evaluating key crew resource management skills. Simul Healthc 2007;2:4–10. [DOI] [PubMed] [Google Scholar]
- 11. Green R, Curry N. Simulation training improves clinical knowledge of major haemorrhage management in foundation year doctors. Transfus Med 2014;24:379–84. [DOI] [PubMed] [Google Scholar]
- 12. Langston A, Downing D, Packard J, et al. Massive transfusion protocol simulation: an innovative approach to team training. Crit Care Nurs Clin North Am 2017;29:259–69. [DOI] [PubMed] [Google Scholar]
- 13. Hilton G, Daniels K, Goldhaber‐Fiebert SN, Lipman S, Carvalho B, Butwick A. Checklists and multidisciplinary team performance during simulated obstetric hemorrhage. Int J Obstet Anesth 2016;25:9–16. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MC. Does every code need a "reader?" Improvement of rare event management with a cognitive aid "reader" during a simulated emergency: a pilot study. Simul Healthc 2012;7:1–9. [DOI] [PubMed] [Google Scholar]
- 15. Gilfoyle E. Simulation‐based team training. In: Cheng VJ, editor. Comprehensive Healthcare Simulation: Pediatrics. Cham: Springer International Publishing, 2016:43–54. [Google Scholar]
- 16. Cheng A, Auerbach M, Hunt EA, et al. Designing and conducting simulation‐based research. Pediatrics 2014;133:1091–101. [DOI] [PubMed] [Google Scholar]
- 17. Andreatta P, Saxton E, Thompson M, Annich G. Simulation‐based mock codes significantly correlate with improved pediatric patient cardiopulmonary arrest survival rates. Pediatr Crit Care Med 2011;12:33–8. [DOI] [PubMed] [Google Scholar]
- 18. Knight LJ, Gabhart JM, Earnest KS, Leong KM, Anglemyer A, Franzon D. Improving code team performance and survival outcomes: implementation of pediatric resuscitation team training. Crit Care Med 2014;42:243–51. [DOI] [PubMed] [Google Scholar]
- 19. Weiner CP, Collins L, Bentley S, Dong Y, Satterwhite CL. Multi‐professional training for obstetric emergencies in a U.S. hospital over a 7‐year interval: an observational study. J Perinatol 2016;36:19–24. [DOI] [PubMed] [Google Scholar]
- 20. Shaulov T, David M, Pellerin M, Morin F. Massive hemorrhage following thrombolysis for postpartum pulmonary embolism with cardiac arrest. J Obstet Gynaecol Can 2014;36:498–501. [DOI] [PubMed] [Google Scholar]
- 21. Allan CK, Thiagarajan RR, Beke D, et al. Simulation‐based training delivered directly to the pediatric cardiac intensive care unit engenders preparedness, comfort, and decreased anxiety among multidisciplinary resuscitation teams. J Thorac Cardiovasc Surg 2010;140:646–52. [DOI] [PubMed] [Google Scholar]
- 22. Figueroa MI, Sepanski R, Goldberg SP, Shah S. Improving teamwork, confidence, and collaboration among members of a pediatric cardiovascular intensive care unit multidisciplinary team using simulation‐based team training. Pediatr Cardiol 2013;34:612–9. [DOI] [PubMed] [Google Scholar]
- 23. Hagerman NS, Varughese AM, Kurth CD. Quality and safety in pediatric anesthesia: how can guidelines, checklists, and initiatives improve the outcome? Curr Opin Anaesthesiol 2014;27:323–9. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Supplement S1. Supplemental material.