Improving nontechnical skills of an interprofessional emergency medical team through a one day crisis resource management training

Abstract

Errors are frequent in healthcare, but Emergency Departments are among the highest risk areas due to frequent changes in team composition, complexity and variety of cases, and difficulties encountered in managing multiple patients simultaneously.

Crisis resource management (CRM) training has been associated with decreased error rates in the aviation industry as well as in certain areas of acute medical care, such as anesthesia and emergency medicine. In this study, we assessed whether a single day CRM training, combining didactic and simulation sessions, improves nontechnical skills (NTS) of interprofessional emergency medical teams.

Seventy health professionals with different qualifications, working in an emergency department, were enrolled in the study. Twenty individual interprofessional teams were created. Each team was assessed before and after the training, through 2 in situ simulated exercises. The exercises were videotaped and were evaluated by 2 assessors who were blinded as to whether it was the initial or the final exercise. They used a new tool designed specifically for the assessment of emergency physicians’ NTS. The intervention consisted of one-day training, combining didactic and simulation sessions, followed by an instructor facilitated debriefing. All participants went through this training after the initial assessment exercises.

A significant improvement (P < 0.05) was shown for all the NTS assessed, in all professional categories involved, regardless of the duration of prior work experience in the Emergency Department.

This study shows that even a short intervention, such as a single day CRM training, can have a significant impact in improving NTS, and can potentially improve patient safety.

1. Introduction

An Emergency Department (ED) is a unique and fascinating workplace where one can see patients of any age, with a wide variety of clinical conditions and pre-existing pathologies. Patients arrive at any time, and they need or desire to be seen as soon as possible. Even more, the medical staff working in the ED has to deal with multiple patients, multiple interruptions and distractions, diagnostic ambiguity, problems due to the ED's and hospital's resources, as well as interactions with staff from other medical disciplines that may not understand the functioning of the ED.^[1]

These particularities place the medical staff: doctors, nurses, and technicians working in the ED, in a state of variable stress and pressure. Not surprisingly, this might increase the risk of errors and might have a negative impact on patient safety.

Unfortunately, errors in the medical field are quite widespread as shown by the report of the US Institute of Medicine published in 1999: To Err is Human. In this report, up to 98,000 deaths per year in US hospitals could be attributed to a preventable medical error,^[2] which means twice the annual highway death rate.^[3]

This huge number of lost lives due to medical errors has encouraged investigators to find ways to enhance patient safety by reducing the risks of errors. To do this, it is important first to understand the genesis of errors and then to find the way to prevent them, capture them, or simply mitigate their effects.^[4]

In a retrospective study of 636 cases referred from the ED for patient care management problems, Cosby and colleagues^[5] identified in >92% of cases multiple contributing factors, as often errors were not the result of an individual failure. Most of the problems were observed during the diagnosis phase and 3 leading contributing factors were identified: clinical task (99%), teamwork (61%) and patient factors (61%).

Similar results about the contribution of teamwork behaviors in the appearance of adverse events in ED were found in another retrospective review published in 1999 by Risser at al.

In anesthesia, 80% of mishaps were attributed to human error. Human error contributed to 70% of commercial aviation accidents.^[6] These figures have to be seen in the context of complex workplaces, and not merely as mistakes made by individual humans. The fact remains, however, that in such environments errors tend to have severe consequences.

As the majority of clinical errors (approximately 70%) are the result of human factors^[7] and they are not technical in nature (failure in team communication, situational awareness, resource utilization and leadership),^[1] it seems reasonable that the primary focus should be on human factor training. One way to achieve this goal is crew resource management (CRM) training, a tool that was developed initially by the aviation industry and is considered to be largely responsible for the decrease in aircraft accidents over the last 4 decades^[8]

One of the first medical specialties to adopt the CRM principle was anesthesia and Anesthesia CRM training has been shown to improve team performance and decrease error during medical and surgical crises in the operating room.^[1,9]

Despite the early interest shown by the specialty of emergency medicine to adapt CRM training, and the promising results of the MedTeams Project in error reduction and performance improvement through formal teamwork training,^[6] there remains a lack of formal training in human factors for professionals working in ED.

Most of the studies in the area of crisis resource management training for emergency medicine focused on emergency medicine residents or interdisciplinary trauma teams.

To evaluate the effectiveness of training, the Kirkpatrick model of educational outcomes can be used:^[10]

Level 1: Reaction—participants perception of the training program.

Level 2: Learning—acquisition of skills or knowledge, changes in attitude in the simulator.

Level 3: Behavior—behavioral changes/transfer of learning to the clinical settings.

Level 4: Results—effect of learners’action (improvement in patients’outcome).

This study is aimed to assess whether a single day CRM oriented team training combining didactic and simulation sessions improves interprofessional emergency medicine team performance of nontechnical skills.

2. Methods

2.1. Study design

We performed a prospective study assessing the impact of a single day CRM oriented training on the performance of interprofessional emergency medical teams.

Our study was conducted in the Tirgu-Mures Emergency Clinical County Hospital, Romania. The hospital has a 5-year emergency medicine residency training program as well as an affiliated simulation center dedicated to multidisciplinary and interprofessional training in emergency and disaster medicine. The annual census of this emergency department is approximately 77,000 patients of whom around 10% are critical patients.

All medical staff without prior CRM training, excluding first-year emergency medicine residents with <3 months experience in ED, were invited to participate.

Seventy staff members volunteered for this study between March and July 2016: 20 emergency medicine attending physicians, 10 emergency medicine residents, and 40 nurses.

The study protocol was approved by the Ethics Committee of the Tirgu-Mures Emergency Clinical County Hospital and informed consent was obtained from all participants.

After enrollment, participants were allocated to a team according to their work schedule.

Twenty mixed teams were formed. Each team was composed of an attending physician, a resident, and 2 nurses. This team composition was considered the ideal for management of critical patients in our ED. As the number of the residents eligible for the study was smaller than the number of attending physicians, each resident was assigned to 2 teams.

We designed 2 sets of scenarios. Each set consisted of 2 scenarios, a trauma case and a medical case (first set: a critical patient with severe pneumonia and acute exacerbation of COPD requiring noninvasive ventilation and then invasive ventilation, anticipation of difficult intubation was expected due to the morphology and history of previous tracheostomy and a polytrauma patient with severe brain injury and hemorrhagic shock due to abdominal bleeding requiring urgent surgery and massive transfusion protocol activation; second set: a critical patient with previous history of COPD now with pulmonary embolism requiring urgent thrombolysis due to rapid deterioration and a polytrauma patient with severe brain injury requiring urgent surgery and a femoral fracture that foster hypotension responsive to fluids, anticipation and management of a difficult intubation was expected). Sets were chosen pseudo randomly, the key element being the resident who was intended to be exposed to all of the cases, with the intention that the resident not repeat sets with the 2 teams. One team started with the medical case while the other started with the trauma case. The scenarios were run in the resuscitation room of the ED during work hours. To run the scenarios, we used a high-fidelity manikin, the medical equipment from the resuscitation room, and the usual emergency department documentation forms. The exercises were video recorded using 2 cameras: one fixed camera was placed on the ceiling of the resuscitation room, while the other camera was mobile. No debriefing was performed after the scenarios.

After the completion of the initial assessment, we established a period of 10 days dedicated to training sessions. The participants were invited to choose a day for the training session according to their availability. The training sessions took place in the simulation center, where we recreate the ED's resuscitation room. Each training session lasted one day (6–7 hours) and consisted of several elements. Initially participants received a lecture focused on medical errors and CRM principles. After the lecture, but before starting the simulation scenarios, participants had the opportunity to familiarize themselves with the manikin and the simulation setting.

For each session, we had 2 interprofessional teams consisting of one attending physician, one resident and 2 nurses. During the training sessions, participants were allowed to move between teams as long as the composition was the same. We ran 6 scenarios representing critical patients, different than those used for the initial assessment (2 trauma cases and 4 medical cases). One team ran through a scenario while the other team observed the exercise from a remote room via a high resolution real time video transmission system. Each team was exposed to all cases, participating actively in 3 scenarios and being observers for the other 3.

The simulation was run by 2 instructors with CRM background training (one doctor and one nurse) and an IT technician. Each scenario was followed by an instructor facilitated debriefing. Both technical and nontechnical issues relating to team performance and team work were discussed, all clinical errors were addressed either by the participants or by the instructors. Both teams were involved in each debriefing but the active participants had priority in providing feedback.

Two months after this training session, we performed a final assessment which consisted of the same 2 scenarios utilized for the initial assessment. The final assessment again took place in the resuscitation room of the ED and the teams were the same as for the initial assessment.

The scenarios were video recorded and no debriefings were performed.

The video recordings of the initial and the final assessment scenarios were independently analyzed by 2 assessors (one emergency physician and one anesthesiologist with broad experience in emergency medicine) with CRM training background. The assessors were blinded as to whether the scenarios being evaluated were the initial or final assessment. The evaluation was done using the scale proposed by Flowerdew et al^[11] (Fig. 1) as a tool for the assessments of emergency physicians’ nontechnical skills. This is a 9-point rating scale, divided into unacceptable (1 to 3), acceptable (3 to 6), and exemplary (7 to 9) with descriptive anchors to aid in scoring. The scale is composed by 4 domains: management and supervision, teamwork and cooperation, decision-making and situational awareness. Each domain is devised in 3 elements as seen in Figure 1.

Figure 1.

A tool for the assessment of emergency physicians’ nontechnical skills. Reproduced with permission of Lynsey Flowerdew.

As this tool was quite new and has not yet been validated in other studies, an agreement was made to accept a difference of no more than 2 scale points between the 2 assessors. If their ratings were >2 scale points apart, they had to meet and reach a consensus opinion to reduce their inter-rater difference to a maximum of 2 scale points. To help achieve best inter-rater agreement, the 2 raters participated in a rating training session before beginning their evaluations. In this training, they were acquainted with the rating scale and practiced its usage on different video footage from the simulation center.

2.2. Statistical analysis

All data were collected in a Microsoft Excel Spreadsheet and converted into an SPSS data file. The normality test used to assess each data series was Shapiro–Wilk test, since some authors consider it to have the greatest power compared to other tests of normality.^[12] This analysis revealed that the data series were non-Gaussian, with skewed distribution curves. Subsequently statistical analysis tests were performed taking into account the fact that our data were non-Gaussian, therefore median and IQR were used to describe each data series, and the Wilcoxon signed-rank test was used to assess the statistical significance for central tendency difference.^[13] The significance level used in all statistical tests was 0.05.

For attending physicians and nurses, the sum of the scores given by the 2 assessors for each skill for the initial/final assessment was used for the statistical analysis. For the residents, as they participated in 2 medical scenarios and 2 trauma scenarios in the initial/final assessment, a mean was generated between the scores of the same assessor for initial/ final medical case and similarly for the trauma case and then the sum of these means was used for the statistical analysis.

3. Results

Seventy participants were enrolled in the study (20 attending physicians, 10 residents, 40 nurses) and 69 completed the study. One nurse participant resigned her position and she was not available for the final assessment scenario. She was replaced by another nurse who had attended the training. The initial nurse was excluded from the final analysis and the nurse who replaced her was analyzed only once with the team which she was allocated to at the beginning of the study.

The male: female ratio was 13:17 for doctors and 11:29 for nurses. Professional experience was variable (average 70 months, minimum 8 and maximum 300 months). Newly employed nurses and first-year residents with less than 3 months experience in the ED were excluded.

The scale used for the assessment of nontechnical skills was originally designed for emergency physicians and covered 12 skills. We used the same scale for nurses but during the assessors’ training it became evident that not all the skills could be applied to nurses, as an attending physician was the team leader in all the scenarios. The scale was then adapted for nurses and 3 skills were omitted: supervising and providing feedback, generating options and selecting & communicating options.

Results for the 12 specific skills of the assessment tool for the twenty EM attending physicians are reported in Table 1 as well as for the 10 EM residents in Table 2.

Table 1.

Pre-and post-CRM training results for EM attending physicians. CRM = crisis resource management, EM = emergency medicine.

Table 2.

Pre-and post CRM training results for EM residents.

The mean experience for the EM attending was 149 months (minimum 72 and maximum 300 months).

For the residents mean experience was 30 months (minimum 12, maximum 52 months).

Thirty-nine nurses completed the study and were analyzed using the modified scale (9 skills). Their mean length of experience in the ED was 38 months (minimum 8 and maximum 156 months). Results are reported in Table 3.

Table 3.

Pre-and post CRM training results for the EM nurses.

In all 3 categories of participants, a significant improvement was shown in all of the skills assessed in both the medical and the trauma cases.

4. Discussion

CRM training has been widely use in the aviation industry as well as in different areas of acute care medicine and has been associated with improved teamwork skills and a decrease in adverse events/errors.^[1,8,9] CRM key points developed by the aviation sector were adapted for medicine by Gaba and colleagues^[14] and called “anesthesia crisis resource management” (ACRM). The fifteen ACRM key points, according to Gaba, are:

• 1.Know the environment
• 2.Anticipate and plan
• 3.Call for help early
• 4.Exercise leadership and followership
• 5.Distribute the workload
• 6.Mobilize all available resources
• 7.Communicate effectively
• 8.Use all available information
• 9.Prevent and manage fixation errors
• 10.Cross (double) check
• 11.Use cognitive aids
• 12.Re-evaluate repeatedly
• 13.Use good teamwork
• 14.Allocate attention wisely
• 15.Set priorities dynamically.

CRM training addresses a set of nontechnical skills, defined as “the cognitive, social and personal resource skills that complement technical skills, and contribute to safe and efficient task performance.”^[15]

As the majority of clinical errors are not the result of individual action, but rather a failure of teams, systems or processes, it seems reasonable that the focus should be on training in teams those who are supposed to work in teams, while also identifying necessary system improvements.^[16]

Our study shows that even 1-day training on CRM principles using a combination of didactic and simulation sessions may significantly improve nontechnical skills in all the categories involved in an interprofessional emergency team.

As the composition of teams in the emergency department changes frequently, we chose to assess each participant individually, and not only the team leader, or the entire team as a unit. We assume that if all participants show an improvement on nontechnical skills during this study, they should individually be able to apply these skills as the clinical team members change. Global teamwork performance score might be influenced by one person in the team. In their study on the effect of simulation-based workshops on multidisciplinary teamwork of newborn emergencies, Liisa Rovamo and colleagues showed that by using a global teamwork performance score known as TEAM (The Team Emergency Assessment Measure), the additional experience of team leaders improved teams’ performances over the course of CRM instruction. ^[7]

In the same study, they found that combined CRM training (lecture and simulations) improved the quality of teamwork only for inexperienced professionals such as pediatric register. In our study, a significant improvement was found not only in teamwork, but also in the other domains explored (management and supervision, decision-making and situational awareness). This was true not only for inexperienced residents, but also for experienced physicians without previous formal CRM training (P ≤ .001). This finding was surprising, as we supposed that by the very nature of their work, consistently practicing in interprofessional teams and having to assume constantly the role of leader in critical situations, EM doctors would develop over time a significant degree of NTS even without specific training. The experience of the attending physicians varied from 72 months to 300 months (mean experience 149) and the initial median for all the skills assessed was higher than that for the less experienced residents, yet they still show a significant improvement.

At the time the study was performed, our center had only 15 residents, only 10 of which were eligible to be enrolled in the study. Three of these were in the second year of their residency program, 3 in the third year, 2 in the fourth year and 2 in the fifth year. Their lesser experience is mirrored in their initial median, lower than the attendings’ and nurses’ median for the same skills. Nevertheless, through training they showed a marked increase in the measured values. Despite this, the P value is higher than in the cases of attendings and nurses, but still statistically significant (P ≤ .01, with the exceptions of “authority and assertiveness” with P = .012 for the medical case). For authority and assertiveness, the presence of an attending as a team leader may lead to a more passive role for residents.

In most of the studies where nurses were involved as a part of the interprofessional team, they were not individually assessed. In our study, we adapted the scale in order to assess them and 3 skills were omitted: supervising and providing feedback, generating options, selecting and communicating options. We do not believe these skills are solely the prerogatives of physicians, but in our study, as an attending physician was the team leader in each case, it was not possible for the assessors to evaluate the nurses appropriately for these skills. The training improved significantly (P ≤ .001) all of the other nontechnical skills in this professional category.

Yee and colleagues documented the effectiveness of a single anesthesia crisis resource management intervention, using a combination of didactic and simulation sessions followed by debriefing, but their study included only anesthesia residents.^[17] They evaluated the residents one and 2 months after the intervention and a significant improvement was shown in all of the NTS assessed with ANTS scale between the first and the second simulation sessions, as well as between the first and third sessions. No improvement was found between the second and third sessions, although the second session was followed by a debriefing that was not specifically oriented toward the identified weaknesses. That means that although they did not improve with the additional simulation session, they did retain the knowledge for the intervening 2 months.

There are contradictory findings in the literature regarding the length of CRM skills retention. In the MedTeams Project, one of the first studies to explore teamwork training in emergency medicine, knowledge retention was seen up to 8 months after the intervention (Kirkpatrick 4).^[6] There are also 2 additional studies (Kirkpatrick 4) that showed that benefits were maintained from 18 to 24 months after a single CRM simulation intervention.^[16] On the other hand, in Miller and colleague's study, despite the improvement of teamwork skills and communication during the training phase, benefits were lost one week after the intervention.^[18] In another study, retention of team behaviors, this time in a simulated environment, was seen up to 6 months post-training. ^[16]

In our study, 2 months after the intervention participants showed a significant improvement in all assessed skills (Kirkpatrick 3). Because working in a team is not an occasional, but rather a daily situation for emergency department staff, we know ED staff have opportunities to practice NTS every day. Further research is needed to address many remaining questions. Among these are, what is the length of the decay phase? Which specific skills are more prone to fade? How often should training be repeated? Additionally, the ideal method of refresher training is unclear. Do we need to repeat the didactic session or would simulations followed by debriefings be sufficient? Could we replace simulations with videotape exercise discussions?

In a systematic review, Boet and colleagues^[10] found 9 studies that target Kirkpatrick level 3 and 4 (transfer of learning to the work place and patient outcome) and their conclusion was that “CRM skills learned at the simulation center are transferred to the clinical settings and the acquired CRM skills may translate to improved patient outcomes, including a decrease in mortality.”

In another study, run in a pediatric ED where simulation-based teamwork was “part of the culture,” Couto et al^[19] compared teamwork scores between real interventions, in situ scenarios, and scenarios in a simulation center, and they found similar results on mean total Team Emergency Assessment Measure (TEAM) scores. No difference was found in between settings on global rating scores. In other words, if you are trained in NTS and you understand their importance, and you have institutional support, you will apply them in real life as well as in the simulation setting.

The association between greater NTS score and greater speed of completion of the care process has been demonstrated in previous studies. Pucher et al^[20] analyzed 50 real-trauma cases starting from the time of trauma team activation to the final patient disposition. Nontechnical skills were evaluated using the modified nontechnical skills scale for trauma (T-NOTECHS). A higher T-NOTECHS score was associated with significant decreases in time to the final disposition, a parameter that was used in others studies as a surrogate for the quality in trauma care.

As CRM training improved significantly the nontechnical skills in all the professions involved in our study, we hope for a positive impact on patient safety in our department. Further research is needed, however.

This study has several limitations.

The study was run in a single center and this limited the number of participants.

Due to the very small number of EM residents available during the study period, they were exposed twice to the initial and final assessment in 2 different teams. Although we used 2 sets of scenarios and we did not perform a debriefing afterwards, we cannot exclude that their performance with the second team was influenced by their experience with the first.

A bias might have been introduced by the agreement to accept a difference of no more than 2 scale points between the 2 raters. The reason for this agreement was the lack of validation of the scale used in previous studies. Most other studies used well-known and validated scales, such as the Ottawa CRM Global Rating Scale (OGRS), the Anesthetists’ Non-Technical Skills (ANTS), TEAM score, and the Observational Teamwork Assessment of Surgery (OTAS) for nontechnical skills assessment. These scales were not specifically designed for EM staff. Although most of the NTS necessary to achieve good performance with low risk of errors/adverse events in anesthesia, surgery or aviation are also applicable to emergency medicine, other skills might be mandatory to deal with distinctive challenges, not common in the other specialty fields, such as: multitasking, dealing with multiple patients and interruptions, high turnover of patients, supervising multiple junior physicians in diverse physical locations, and working with limited resources. ^[21]

To our knowledge, this is the first study to use this scale designed specifically for the assessment of emergency physicians, and it was not our objective to validate it.

In this study, we only evaluated the participants 2 months after the training. We did not assess them immediately after the training, so we do not know if their performance improved or declined over this time period.

Ideally, after a CRM training actual patient outcomes would be measured. This was our initial intention when we implemented this study. Unfortunately, we encountered a technical problem when we tried to analyze real cases. The quality of the image returned by the fixed camera was good, but the quality of the sound was too poor to permit proper evaluation of NTS.

5. Conclusion

Our study supports the use of combined CRM training for teaching and improving nontechnical skills for all the professional categories involved in an emergency team. Even a short intervention might have a significant impact, regardless of the experience of the ED staff.

In order to improve patient safety in the high-risk ED environment, the focus should be on training in CRM all those who are supposed to work within the emergency team, not only doctors but also nurses and technicians.

Ideally, CRM training should be a part of the curriculum for all members of the emergency medical team and patient safety should be an integral part of the ED culture. To achieve these goals, institutional support is mandatory.

Author contributions

Conceptualization: Teodora Sorana Truta, Sanda Maria Copotoiu, Marc Lazarovici.

Data curation: Teodora Sorana Truta.

Formal analysis: Cristian Marius Boeriu, Marius Petrisor, Dan George Vatau.

Investigation: Teodora Sorana Truta, Emilia Turucz, Dan George Vatau.

Methodology: Teodora Sorana Truta, Emilia Turucz.

Supervision: Cristian Marius Boeriu, Sanda Maria Copotoiu, Marc Lazarovici.

Validation: Emilia Turucz, Marc Lazarovici.

Visualization: Dan George Vatau.

Writing – original draft: Teodora Sorana Truta.

Writing – review & editing: Cristian Marius Boeriu, Sanda Maria Copotoiu, Marius Petrisor, Emilia Turucz, Dan George Vatau, Marc Lazarovici.