Advanced closed-loop communication training: the blindfolded resuscitation

Abstract

Closed-loop communication (CLC) improves task efficiency and decreases medical errors; however, limited literature on strategies to improve real-time use exist. The primary objective was whether blindfolding a resuscitation leader was effective to improve crisis resource management (CRM) skills, as measured by increased frequency of CLC. Secondary objectives included whether blindfolding affected overall CRM performance or perceived task load. Participants included emergency medicine (EM) or EM/paediatric dual resident physicians. Participants completed presurveys, were block randomised into intervention (blindfolded) or control groups, lead both adult and paediatric resuscitations and completed postsurveys before debriefing. Video recordings of the simulations were reviewed by simulation fellowship-trained EM physicians and rated using the Ottawa CRM Global Rating Scale (GRS). Frequency of CLC was assessed by one rater via video review. Summary statistics were performed. Intraclass correlation coefficient was calculated. Data were analysed using R program for analysis of variance and regression analysis. There were no significant differences between intervention and control groups in any Ottawa CRM GRS category. Postgraduate year (PGY) significantly impacts all Ottawa GRS categories. Frequency of CLC use significantly increased in the blindfolded group (31.7, 95% CI 29.34 to 34.1) vs the non-blindfolded group (24.6, 95% CI 21.5 to 27.7). Participant’s self-rated perceived NASA Task Load Index scores demonstrated no difference between intervention and control groups via a Wilcoxon rank sum test. Blindfolding the resuscitation leader significantly increases frequency of CLC. The blindfold code training exercise is an advanced technique that may increase the use of CLC.

Introduction

Effective teamwork and communication are cornerstones of patient safety in healthcare.^1–3 Communication failure is the third most common root cause of incident reports in emergency medicine (EM) and teamwork failure including communication errors are main factors in medical errors across specialties.^{4 5} Ineffective and inadequate communication among a healthcare team is the main aetiology of unsafe acts found in anaesthesia critical incidents.² All healthcare teams, regardless of experience, require deliberate communication strategies to perform effectively in crisis situations.⁶ Crisis resource management (CRM) and nontechnical skill training are the focus of strategies to mitigate preventable patient harm.^{7 8} Closed-loop communication (CLC) is a specific structure that improves task efficiency and decreases medical errors.^{9 10}

Although CLC is known to improve accuracy and timeliness of order completion, it remains difficult to apply in both real-time clinical situations and simulations. In simulated cardiac arrest scenarios, team leaders used CLC less than 30% of the time, while team members failed to communicate misunderstanding of orders, verbalise need for clarification and repeat back orders.¹¹ A recent study demonstrates this deficiency in the clinical environment with in-hospital paediatric cardiac arrest.¹² Communication was listed as a positive in less than half of in-hospital paediatric cardiac arrests and in only 24% of paediatric emergency department (ED) postevent debriefings.^{12 13} A prior study noted communication was commonly cited as requiring improvement in paediatric ED resuscitations.¹³

There is limited literature on CLC teaching techniques and successful strategies to improve real-time use of CLC in crisis situations. Simulation is one method shown to be effective in teaching communication skills.^{10 14} The advanced technique of blindfolding a resuscitation leader has been shown to reinforce use of CLC by removing visual cues, forcing the leader to develop communication skills.¹⁵ However, this has not been formally evaluated for increasing utilisation of CLC. Our aim is to determine if blindfolding is an effective strategy to teach CLC and improve CRM skills, as measured by an increase in frequency of perfect CLC use. Secondarily, it was hypothesised that blindfolding a leader would improve overall CRM performance and increase perceived task load compared with non-blindfolded leaders.

Methods

The study was performed in 2018 in an academic institution’s simulation lab. The study population included current EM or dual EM-paediatric resident physicians. Power analysis was performed prior to study onset (α=0.05) for the overall Ottawa Crisis Resource Management Global Rating Scale (GRS) scores and 53 participants were necessary based on estimated effect size of two. Participation was voluntary and written consent was obtained. The Institutional Review Board responsible for human subjects research at the university reviewed and approved this research.

Participants were block randomised; the intervention group led both resuscitation simulations blindfolded and the control group led both non-blindfolded. Online supplementary file 1 delineates participant flow through the study. Participants lead both adult and paediatric resuscitations, to avoid potential confounding effect of previous high-level experience by providing simulation scenarios of both frequent (adult) and infrequent (paediatric) clinical scenarios. Participants watched internally made videos immediately before the simulation demonstrating excellent and poor CLC. All participants were clearly instructed in the simulation prebrief that orders would only be executed if communication was in perfect CLC format; non-verbal direction would be unsuccessful. Participants were also given an orientation to the manikin and simulation lab. The resuscitation team was comprised of three embedded standardised participants for each participant (online supplementary file 2). The postsurvey included NASA Task Load Index (TLX) and qualitative questions.¹⁶ The NASA TLX is a validated tool used by participants to self-rate perceived task load with regard to mental, physical, temporal demand, level of frustration and success.¹⁷

The simulations were recorded and reviewed by two simulation fellowship-trained EM physician raters who were blind to participant identity and level of training and a third who was an internal faculty member at the institution. The raters used the Ottawa GRS for assessment. This validated tool has demonstrated effective inter-rater reliability assessing overall performance of the team leader, in addition to leadership, problem solving, situational awareness, resource utilisation and communication skills.¹⁸ The videos were reviewed by a single rater for frequency of CLC and non-closed loop commands.

Quantitative methods

Summary statistics were performed. Intraclass correlation coefficient (ICC) was calculated for inter-rater reliability. Raters underwent additional training via video review and discussions to develop a shared mental model and final ICC 1.0 or excellent inter-rater reliability. Data analysis was performed using R program for analysis of variance and regression analysis. Frequency of perfect CLC was calculated.

Qualitative methods

Participant postsurveys included qualitative questions regarding the best and most challenging parts of the simulation. The intervention group answered questions regarding degree of difficulty and leadership skills required with blindfolding. Qualitative content was analysed through an inductive qualitative analysis. Two blinded study authors independently reviewed the deidentified qualitative comments and using inductive reasoning and repeated comparisons, identified common themes.

Results

Thirty-four participants were block randomised into blindfolded or non-blindfolded subgroups. Demographic data are summarised in online supplementary file 3. When evaluating the mean Ottawa GRS scores using a linear mixed model with effects of blindfolding and simulated case, no category showed significant differences between main effects. Frequency of closed loop communication significantly increased between the blindfolded group (31.7, 95% CI 29.34 to 34.1) and non-blindfolded group (24.6, 95% CI 21.5 to 27.7) (figure 1). PGY significantly impacts all Ottawa GRS categories (table 1). When the linear model is adjusted to control for PGY, results for all groups remain insignificant. Self-rated NASA TLX scores demonstrate no significant difference in workload between groups via a Wilcoxon rank sum test (online supplementary file 4). Self-reported previous clinical adult resuscitation leader experience positively correlated with mean overall Ottawa GRS score (0.36, p<0.01), while previous clinical paediatric resuscitation leadership experience had no effect (0.07, p=0.82). Qualitative themes identified from survey comments are highlighted in online supplementary file 5.

Figure 1.

Frequency of in blindfolded versus non-blindfolded groups for adult and paediatric simulation cases. CLC, closed-loop communication.

Table 1.

Effect of PGY on mean scores of Ottawa GRS categories

Ottawa GRS category	Mean score	Effect of PGY on mean score	PGY effect SE	P value
Overall	3.848	0.531	0.122	<0.01
Leadership skills	3.772	0.568	0.127	<0.01
Problem solving	3.796	0.518	0.118	<0.01
Situational awareness	3.682	0.536	0.127	<0.01
Resource utilisation	4.249	0.460	0.116	<0.01
Communication	4.780	0.326	0.110	<0.01

GRS, Global Rating Scale; PGY, postgraduate year.

Discussion

Blindfolding the leader significantly increased frequency of CLC in the simulations. Increasing participant PGY significantly increased overall Ottawa GRS ratings, which held true for all GRS categories. The Ottawa GRS overall score was not significantly impacted by blindfolding; this held true for all GRS categories. Additional prior clinical resuscitation leadership experience positively correlated with the overall Ottawa GRS rating in adult resuscitations only.

Successful teamwork and communication is paramount to patient safety.^1–3 Communication, specifically closed-loop, is the focus of healthcare institutions nationwide to improve patient safety.¹ CLC is shown to decrease medical errors and improve task completion efficiency.^{9 19} Simulation is an effective method to teach this skill.^{10 14} Our study shows that use of CLC is significantly increased when the team leader is blindfolded, thus an inexpensive way to reinforce learnt communication skills. Without visual cues and non-verbal communication aids, leaders require CLC format to ensure specific task assignment, comprehension and timely completion. Blindfolding and positioning the leader backwards requires clear, direct verbal orders, enhancing both communication and leadership skills.

Our results demonstrate that senior residents, compared with juniors, are more effective at CRM. Advancing PGY is significantly associated with higher mean scores in all GRS categories. The PGY system is used in the USA to numerically delineate progress of a resident physician through their residency programme after medical school. A prior study found non-technical skills improved sustainably with explicit CRM training, versus with residency training alone.²⁰ Our findings of CRM skill improvement with PGY may be due to resident participation in clinical training and multiple simulations throughout their annual educational curriculum at the study institution. Simulation as a tool to improve nontechnical CRM skills is well demonstrated in the literature.^{21 22} Previous literature demonstrated that non-technical skills did not deteriorate with time.²⁰

Interestingly, more previous clinical leadership experience positively correlated with higher mean overall GRS score in adult resuscitations only. This trend excluding paediatric experience is likely related to the participant’s relative clinical inexperience. All participants rated less than 4 paediatric resuscitations as previous clinical leadership experience, while adult resuscitation experience varied in a bell-shaped curve from 0 to greater than 21.

The most notable qualitative theme identified was that scenarios provided excellent opportunities for additional resuscitation practice in a low-stakes environment (online supplementary file 5). Simulation allows residents to learn in a controlled setting and perform deliberate practice of skills without patient harm.²² Using standardised embedded participants with a single resident leader likely enhanced the learning environment safety. Blindfolded participants felt blindfolding was both the best and most challenging part of the simulation. We suspect that blindfolding challenged communication and leadership skills by removing visual stimuli, while also allowing participants to solidify algorithmic knowledge. Non-blindfolded learners commented on the challenge of CLC use. Without the blindfold forcing the loop to close, non-blindfolded learners relied on usual communication habits including undirected orders and non-verbal cues.

Limitations

This study has several limitations. Sample size, single site and resident specialty limits result generalisability. The data set is underpowered for the effect size seen in overall Ottawa GRS score variation, although posthoc analysis of power for difference in CLC was adequate. Extending the study over a longer time period with an interdisciplinary cohort would potentially provide a more meaningful analysis to the potential benefits of this atypical approach. One rater was non-blinded to participant PGY, participating as a simulation embedded participant and internal faculty member, possibly introducing bias. However, this was minimised by averaging all three ratings.

Future areas of research include evaluating communication skill retention among trainees using blindfolded simulation versus traditional training and whether simulation-based teaching of CRM skills translates into improved clinical resuscitation performance.

Conclusion

In conclusion, blindfolding the team leader significantly increases frequency of CLC. Senior residents are more effective at CRM skills than juniors. The blindfold code training exercise is an advanced technique that may improve the utilisation of closed loop communication strategies.