Summary
This multi‐centre repeated measures study was undertaken to determine how contrasting designs of cognitive aids affect team performance during simulated intra‐operative anaphylaxis crises. A total of 24 teams consisting of a consultant anaesthetist, an anaesthetic trainee and anaesthetic assistant managed three simulated intra‐operative anaphylaxis emergencies. Each team was assigned at random to a counterbalanced order of: no cognitive aid; a linear cognitive aid; and a branched cognitive aid, and scored for team functioning. Scores were significantly higher with a linear compared with either a branched version of the cognitive aid or no cognitive aid for ‘Team Overall Behavioural Performance’, difference between study groups (F‐value) 5.8, p = 0.01. Aggregate scores were higher with the linear compared with the branched aid design (p = 0.03). Cognitive aids improve co‐ordination of the team's activities and support team members to verbalise their actions. A linear design of cognitive aid improves team functioning more than a branched design.
Introduction
Cognitive aids such as posters, checklists and flow charts improve the speed and accuracy of task completion during anaesthetic crises in many circumstances 1. However, the effect of a cognitive aid on the team's functioning is less clear. A cognitive aid may improve team performance by aligning the team's understanding of the cues and tasks required in the situation with the team's expectations of how the situation will develop 2. This is often referred to as ‘team situation awareness’, and it allows team members to monitor each other's performance and speak up if a teammate requires help, or there is a safety issue 3. Conversely, a cognitive aid may distract the team from performing key tasks or may require resources that exceed what are available. In this context a cognitive aid may impair team functioning, leading to poorer outcomes 4.
Team performance may be measured by observation of task completion or by observing team processes such as communication, co‐ordination and leadership. Outcome measures may not give the full picture about how a team manages an emergency situation. Studies examining the effects of cognitive aids on team processes have either looked retrospectively at team co‐ordination, or used scores of individual team members' behaviours such as non‐technical skills scoring systems 1, 5, 6. To date, no study has prospectively examined the effect of cognitive aids on team processes.
The design of cognitive aids has also been given little attention in the medical literature 1. It is possible that when cognitive aids fail to improve performance, the design may have been a factor. Design failures may promote errors 7, 8, make the use of aids problematic 4, or discourage aid use 9. When evaluating a cognitive aid, it is important to ensure the design will prompt users to perform required tasks without producing omissions or errors 1.
Flow charts, or decision trees with multiple branches depending on the clinical situation, are common in healthcare settings; however, it is not clear if non‐linear designs are the best way to represent information for use in an emergency. Non‐linear designs may be confusing, difficult to follow, and lead to more errors 10. In addition, they may potentially require more cognitive and team resources than otherwise necessary during a crisis. An example of a complex flow chart is the American Society of Anaesthetists (ASA) difficult airway algorithm 11, which has multiple options that may be challenging to enact during a clinical crisis.
Guidelines for the management of peri‐operative severe allergic reactions have recently been developed by the Australian and New Zealand Anaesthetic Allergy Group (ANZAAG) that included two of the authors (HK and SM) 12. These guidelines, representing current best practice, have been endorsed by the Australian and New Zealand College of Anaesthetists and are recommended for use in peri‐operative settings in Australasian hospitals. In addition, these guidelines have been incorporated into the College's CPD program as one of four emergency response activities, the others being management of major haemorrhage, ‘can't intubate, can't oxygenate’, and cardiac arrest. Participation in a minimum of two of these four emergency response activities is required by fellows every three years. However, fellows can choose which two emergency response activities to complete. The detailed guidelines were designed as a set of four cards representing: diagnosis; immediate management; refractory management; and post‐event management. Two cards (immediate management and refractory management) were specifically designed for use during a crisis, and two designs were developed; linear and branched flow chart versions for both cards (Appendices 1 and 2).
The linear and branched designs represented the same information on the ‘immediate management’ and ‘refractory management’ cards. The format and wording were kept as similar as possible, but due to space and font size constraints a few changes were made. A subcommittee of the Australian and New Zealand College of Anaesthetists (ANZCA) endorsed the designs, colour scheme, wording and content. Additional information was provided in the box with the aid as suggested on the ANZAAG website: the same checklist‐style design for the two other cards (‘diagnostic’ and ‘post‐emergency management’) included in the box; hospital infusion protocols; referral forms for allergy testing centre; a letter and information booklet for patients; pathology tubes and forms for mast cell tryptase measurement 12. This additional information was only provided to the team if the cognitive aid was provided, but the participants were advised that information that is normally available would be provided on request. To date, no version of the cards has been empirically evaluated.
The primary aim of the present study was to compare the effect of two designs of cognitive aid (linear or branched) vs. no cognitive aid on aspects of team function during a simulated intra‐operative anaphylaxis emergency. The secondary aim was to determine the effect of the two cognitive aids on task completion, omissions and dangerous actions such as incorrect medication doses during the simulated emergencies. However, it was felt that educating the participants about the cognitive aid so that they would be more likely to use it took priority over this secondary measurement.
As the intent of performing this research was to determine how the design of the aids affected team and task action, only teams that used the aid in both scenarios in which the aid was provided were included in the analyses. Individuals who read from the cognitive aid without discussion with the rest of the team were also not studied, as this was not the recommended mode of use.
Methods
Ethical approval for this multicentre repeated measures trial was obtained from Monash Health, Hunter New England and The University of Queensland human research ethics committees. Participants were invited to volunteer via departmental email, and each provided written informed consent.
The participants were members of anaesthetic teams from three healthcare organisations (Monash Health and Peninsula Health in Victoria, and Hunter New England Health in New South Wales, Australia). Each study team of three consisted of a consultant anaesthetist or anaesthetic fellow, a junior anaesthetic trainee and an anaesthetic assistant (nurse or technician). The make‐up of the team reflected the actual role composition of anaesthetic teams commonly found in each organisation.
The scenarios were performed at two simulation centres using anaesthetic machines similar to those used in the participants’ clinical areas. The ‘patient’ was a Laerdal SimMan® 3G manikin (Stavanger, Norway), with the integrated monitor replacing the usual vital signs monitor. All the regular equipment, medications and intravenous fluids were available to the participants, and they were oriented to the location of the equipment and to the manikin for a period of 30 minutes before the scenarios. Also preceding the scenarios, participants were introduced to the new guidelines and the cognitive aids they would be using in the scenarios during a further 30‐min presentation.
A sign stating that no cognitive aid was available, or a box containing one of two cognitive aid designs was placed outside the operating room depending on the test condition the team was scheduled to experience. The presence or absence of the aid was not communicated to the participants before the commencement of the scenario. The two designs were either the linear version or the branched version of the immediate and refractory management cards.
Participants were asked to complete a questionnaire before the session to determine their level of experience and past exposure to the management of anaphylaxis. They were aware that all the testing scenarios were to be anaphylaxis emergencies when they were recruited. They were given the opportunity to familiarise themselves with the management of anaphylaxis before attending and during the familiarisation sessions.
Each team of three participants undertook all three scenarios. The first scenario presented (Appendix 3.1) was always a severe allergic reaction on induction of anaesthesia. The second and third scenarios (Appendix 3.2 and 3.3) were reactions to a colloid intravenous fluid with refractory hypotension, and reactions to a chlorhexidine‐impregnated central line with refractory bronchospasm. The order of the second two scenarios and of the three cognitive aid conditions across all three scenarios was counterbalanced (undertaken in a specified order and reversed order to account for any unforeseen effects). The counterbalanced study design was based on Latin squares 13. The counterbalancing resulted in 12 unique sequences of scenarios and cognitive aids (Appendix 4). The set of 12 sequences was repeated in reverse order to arrive at 24 instances of scenario and cognitive aid combinations. The order in which the 24 participating teams were assigned to one of the 24 sequences was randomised.
Audiovisual recordings were taken of all 72 scenarios. Two consultant anaesthetists (SM and HK) analysed the video data following two hours of rater reliability training on pilot data. Rater reliability was maintained by intermittent discussion, initially after every scenario and then up to every six scenarios. A third of the scenarios (24 scenarios) were observed by both raters to ensure inter‐rater reliability, with the rest being observed by a single rater. Inter‐rater reliability was assessed using intraclass correlations (ICC) for each category score. ICC values above 0.61 were considered substantial, and values above 0.80 excellent 14. Team performance was assessed with the Auckland Team Score using the mean of element scores of the three categories ‘leadership and team co‐ordination’ (LTC), ‘mutual performance monitoring’ (MPM) and ‘verbalising situational information’ (VSI), and also the value for the ‘team overall behavioural performance’ (TOBP) score with values ranging from 1 to 7 15. The Auckland Team Score was chosen as a rigorously validated method of scoring entire teams during emergency situations, and performed well in initial inter‐rater reliability testing 15.
The times from the start of the allergic reaction (as indicated by a change in the capnography trace or systolic hypotension below 100 mmHg) to key behaviours were measured. The dose and route of adrenaline and any dangerous behaviours, were also noted, to ensure avoidance of dangerous behaviours and completion of ideal critical procedures identified before the analyses. These ‘critical tasks’ numbered from seven to ten depending on the scenario. An overall technical performance score was not created because of the different nature of the scenarios and therefore different tasks required.
All statistical analyses were undertaken using SPSS version 20 (SPSS Inc., Chicago, IL, USA). Only teams that used cognitive aids in both scenarios in which the aid was available were analysed to ensure that the effect of cognitive aid use was being assessed. Repeated measures ANOVA tests with the within‐subjects factor of cognitive aid conditions and the between‐subjects factor of counterbalancing sequence were performed for the Auckland Team Scores and for the number of key actions taken. The ANOVA tests of team scores were also performed for just the teams that used the cognitive aid whenever it was provided. The latter analyses were undertaken to determine the effect on performance of using cognitive aids as they were designed, rather than the effect of merely having them available to a team. In all the ANOVA tests, Mauchley's test of sphericity was performed on the repeated measures, and a Greenhouse‐Geisser correction was used where needed. Post hoc pairwise comparisons were made using Tukey HSD tests.
Fisher–Halton exact tests were used to evaluate associations between whether teams completed key actions and the cognitive aid condition. A correction was applied to the Type I error rate across tests where multiple actions were observed to preserve α = 0.05.
Times to perform key actions were transformed into normally distributed data using logarithmic functions, and ANOVA tests were then applied with factors of cognitive aid condition and counterbalancing order.
Power calculations based on scoring of a similar pilot study of medical student teams suggested that with a repeated measures design, 24 teams would be required to observe a mean improvement of 50% for each measure (effect size F = 0.4, α = 0.05, 1 − β = 0.99).
Results
A total of 24 teams of three participants per team were recruited from three healthcare organisations (Table 1). Due to availability, teams were not equally sampled from each organisation. In six cases, the team comprised a senior trainee (an anaesthetic fellow), a junior trainee and anaesthetic nurse rather than a consultant anaesthetist, trainee and anaesthetic nurse. Over half of the participants had previously treated a case of anaphylaxis (n = 39, 54.2%) and a majority stated they would use a cognitive aid for anaphylaxis management if it were available (n = 65, 90.2%). Inter‐rater reliability measures of the Auckland Team Score were found to be substantial or excellent.
Table 1.
Participant characteristics (n = 72). Number (proportion) or median (range)
| Characteristic | Frequency | |
|---|---|---|
| Organisation | ||
| 1 | 21 (29.2%) | |
| 2 | 24 (33.3%) | |
| 3 | 27 (37.5%) | |
| Role | Median duration time in curent role; yr | |
| Consultant Anaesthetist | 18 (25.0%) | 7.0 (0.9–30.0) |
| Trainee Anaesthetist | 30 (33.3%) | 3.0 (0.2–13.0) |
| Anaesthetic nurse/assistant | 24 (58.3%) | 5.0 (1.0–23.0) |
| Have you ever treated a case of anaphylaxis? | Duration since last anaphylaxis case | |
| Yes | 33 (45.8%) | 1 yr (4 wks–35 yrs) |
| No | 39 (54.2%) | |
| If a cognitive aid/algorithm were readily available for the management of anaphylaxis do you think you would use it? | ||
| Yes | 65 (90.3%) | |
| No | 5 (6.9%) | |
The cognitive aid was either not used at all or was not read aloud to other team members in 9 of the 72 scenarios observed, and 48 scenarios for which an aid was provided. These teams were excluded from further analyses, leaving only the 15 teams that had actively used the cognitive aids in both scenarios in which the aid was available (Fig. 1). Teams were included if they used the cognitive aid and at some stage had a team member that verbalised the contents of the aid in both the scenarios when a cognitive aid was provided. No associations were found between experience of the participants and whether or not they used the cognitive aid when provided.
Figure 1.
Flow chart of recruitment, and randomisation. CA, cognitive aid.
The linear aid supported better team performance than did either the branching aid or no aid (Figs 2 and 3). There were no significant interactions between team scores across different orders of cognitive aid presentation.
Figure 2.
Analysis of the team performance categories of the 15 teams that used the cognitive aid when present. Error bars represent 95% CIs. ICCs were substantial or excellent (leadership and team co‐ordination (LTC) = 0.76, p < 0.001, mutual performance monitoring (MPM) = 0.76, p = 0.002, verbalising situational information (VSI) = 0.85, p < 0.001, team observable behavioural performance (TOBP) = 0.88, p < 0.001). *Significant differences between teams with no aid and branched and linear versions of the aid (p < 0.001). **Significant differences between teams with no cognitive aid and the linear version of the cognitive aid, and between teams with linear and branched versions of the cognitive aid (p = 0.01). CA, cognitive aid.
Figure 3.
Analysis of the aggregate team performance scores of the 15 teams that used the cognitive aid when provided. Error bars represent 95% CIs. CA, cognitive aid.
Aggregate scores of the team performance measures (the sum of LTC, MPM, VSI and TOBP with a possible range of 4–28) showed significant differences between each cognitive aid and no cognitive aid (Fig. 3). The linear aid was associated with higher aggregate scores than the branched aid (p = 0.03).
No associations were found between the type of cognitive aid and completion of critical tasks or observation of dangerous actions (Table 2). In addition, no associations were found between cognitive aid condition and the time taken to perform critical tasks.
Table 2.
Associations between cognitive aid type and method of aid use with completion of 10 critical tasks. Findings are considered significant at α < 0.05 in a Bonferroni correction that accounts for the 10 measures tested with each condition breakdown
| Critical task | Condition | Critical task performed | p value |
|---|---|---|---|
| Call for help | No cognitive aid | 13/15 | 0.34 |
| Linear aid | 15/15 | ||
| Branched aid | 14/15 | ||
| Inspired oxygen increased | No cognitive aid | 15/15 | 1.00 |
| Linear aid | 15/15 | ||
| Branched aid | 15/15 | ||
| Volatile agent reduced | No cognitive aid | 15/15 | 1.00 |
| Linear aid | 15/15 | ||
| Branched aid | 15/15 | ||
| Correct initial adrenaline dose given | No cognitive aid | 15/15 | 1.00 |
| Linear aid | 15/15 | ||
| Branched aid | 15/15 | ||
| Incorrect adrenaline doses given | No cognitive aid | 0/0 | 1.00 |
| Linear aid | 0/0 | ||
| Branched aid | 0/0 | ||
| Triggering agent identified (scenarios 2 and 3 only) | No cognitive aid | 6/7 | 0.86 |
| Linear aid | 10/11 | ||
| Branched aid | 10/12 | ||
| Adrenaline infusion started | No cognitive aid | 6/15 | 0.23 |
| Linear aid | 10/15 | ||
| Branched aid | 10/15 | ||
| Noradrenaline infusion started | No cognitive aid | 1/15 | 1.00 |
| Linear aid | 1/15 | ||
| Branched aid | 1/15 | ||
| Intravenous salbutamol given | No cognitive aid | 2/15 | 0.24 |
| Linear aid | 6/15 | ||
| Branched aid | 5/15 | ||
| Potentially dangerous action observed | No cognitive aid | 3/15 | 0.46 |
| Linear aid | 3/15 | ||
| Branched aid | 1/15 |
Discussion
This study demonstrates that the presence of a cognitive aid has positive associations with indices of the LTC (leadership and team co‐ordination), VSI (verbalising situational information) and TOBP (team overall behavioural performance) team performance measures, whereas scores of MPM (mutual performance monitoring) by team members do not seem to be affected by the presence of cognitive aids. Associations were independent of the order of presentation of the cognitive aids, suggesting that the results cannot be explained by a learning effect from undertaking the series of scenarios. Although associations between team performance scores and outcome have been shown in several studies 16, it is not yet clear what would constitute a clinically significant improvement of team process measures. Further research is needed to determine how measures of co‐ordination, leadership and communication affect the likelihood of a positive patient outcome in clinical emergencies.
Manser and colleagues observed that the presence of a cognitive aid for malignant hyperthermia management led to changes in team co‐ordination patterns and was associated with better team performance 5. Co‐ordination patterns were identified that described the type and function of information that was shared during the phases of an emergency. Manser and colleagues’ study suggested that focusing on task co‐ordination rather than recurrent assessment of the situation led to poorer performance. However, the study was a retrospective observational study in which participants were allowed to use either the Malignant Hyperthermia Association of the United States (MHAUS) cognitive aid, their own electronic devices, or no cognitive aid 17. In a later study, Burden et al. observed a decrease in communication levels when a cognitive aid was present; however, it is not clear if the reduction in overt communication indicated that team members had a shared mental model (understanding and knowledge of the situation) or that there was distraction within the team 6.
A more recent study demonstrated an improvement of individuals’ non‐technical skills and a reduction in conflict when a cognitive aid was provided and used by the clinicians present 18. Neither of the cognitive aids used in that study or in this study provided extensive directions on communication or co‐ordination of the team, making the improvements in team functioning more remarkable. One mechanism by which team function may be improved is by cognitive aids reducing the cognitive load on the clinician leading the team, allowing the leader to pay more attention to team aspects rather than remembering the priority of tasks. Cognitive aids might also help team members maintain a shared understanding of the situation and minimise the need for switching between technical and team management tasks 19.
A previous study of cognitive aids on teams suggested that the amount of communication is reduced when an aid is present 6. Our study did not measure the volume of communication, but even if the cognitive aid had reduced the volume of communication, our study still showed that leadership and communication were enhanced with an aid. Indeed, the higher scores on the VSI team performance category suggest that sharing of relevant information may in fact be increased when a cognitive aid is used. This observation is in keeping with the findings of Manser et al. that the volume of information management was higher, both in teams that were more effective and in teams that used a cognitive aid 5. The teams that performed better in Manser et al.'s observational study had a rapid and explicit task distribution, resulting in less communication after the initial stage of the crisis. The present study measured the perceived effectiveness rather than the number of communication episodes over time, and so the impact of a cognitive aid on the amount of communication was not tested.
The ability of team members to monitor each other's activities, termed ‘mutual performance monitoring’, was not significantly affected by the cognitive aid in our study. Previous research has suggested that members of teams with high levels of performance are more likely to indicate their potential errors or omissions and to respond to those challenges 20. It may be that cognitive aids do not assist with mutual performance monitoring, or that the content or design in this study did not prompt these behaviours. A different design of the cognitive aid or different scenarios may be required. For example, a different design could specifically mention the expectation of the role of each team member during the emergency and could note common errors. Against this suggestion, however, is the potential risk that a more prescriptive cognitive aid might limit flexibility and communication during the emergency, and therefore prevent adaptive co‐ordination strategies 21.
The present study also demonstrated that the specific design of the cognitive aid had an effect on team performance. Cognitive aids in the form of algorithms in anaesthetic emergencies are commonly presented as branched flow charts. Linear and branched versions of a cognitive aid are not the only method of presentation. Smartphone applications and computer‐based prompts are increasingly used 22, but a comparison of different versions has not been undertaken until now, even in paper format. By analysing results only for teams that used the cognitive aid as designed in both conditions, variance that may have been caused when teams did not use the aid or used it ineffectually is reduced. This poses a potential challenge to the common presentation of branched algorithms in clinical practice. Communication may be better with a linear aid than a branched aid because it is easier to navigate during an emergency, or easier for a reader to summarise to the team. The linear design may facilitate the process of a reader calling out items and multiple team members responding. This method of use is termed a ‘static sequential checklist with verification and confirmation’, and is thought to support team performance when there are multiple team members taking diverse roles 23. Overall, our finding warrants further investigation in a separate confirmatory study.
Previous studies into team performance have focused on individual team members rather than examining the overall team. Two previous studies were insufficiently powered to detect changes in non‐technical skills scores during simulated crises of neonatal resuscitation and systemic local anaesthetic toxicity 4, 24. However, a recent study of simulated airway management crises showed strong associations between the use of the cognitive aid and higher non‐technical skills scores of individuals 18. Some of the improvement in the team scores may be due to a reduction in cognitive load on the individuals because of less need to remember the steps and tasks required during the emergency. Reducing cognitive load may give the participants more time to consider the team aspects of the emergency rather than being a distraction to management.
The primary aim of this study was to examine the effect of the cognitive aids on team behaviours, with task completion being a secondary measure. The statistical power of this study was not sufficient to determine which of the two cognitive aids or which method of aid use improved task completion. Further research is required to determine if cognitive aid design affects task completion as well as team processes. Nonetheless, many previous studies have shown the benefit of cognitive aids on technical performance 1.
The results of this study are likely to be generalisable, at least within the Australian tertiary hospital context. Teams were sourced from three large teaching hospital networks in two Australian states, and represented a wide range of anaesthetic experience. The teams worked in combinations that are typical during their routine work.
A weakness of this study was that teams were expecting to encounter serious anaphylactic reactions in all three scenarios, and so were primed to perform well. The participants were given informed consent forms two weeks before the scenarios. On arrival at the testing location they were given a 30‐min tutorial on the anaphylaxis guidelines, and were familiarised with the cognitive aids that would be used during the testing. Prior knowledge may have been the reason why the effects of the aid on technical performance were minimal. Team members were equally primed to demonstrate good team performance behaviours, yet team performance was still improved when the cognitive aids were used. It may be that larger effects would be seen in teams that are not primed, as would be the case in routine clinical settings.
A limitation of simulation‐based studies is the learning effect of undertaking multiple scenarios. The participants received a detailed familiarisation to the simulation room, the anaesthetic machine and the manikin before testing. This was done partly to reduce the likelihood of improving by merely learning where equipment was kept and how to work the equipment, and also to make the participants comfortable in the environment. The learning effect of subsequent scenarios was controlled for by the counterbalanced, within‐subjects design and appears to have had no effect or only a small effect on the teams’ performance.
Despite extensive education about the cognitive aid and over 90% of participants claiming they would use an aid if it were available, the aid was still not used in 9 of the 48 cases (19%) that it was provided. This is similar to other simulation studies in which education about the cognitive aid was provided before testing 4, 17, 18. In the actual clinical environment, the rate of use is much lower, with reports as low as 7% of clinicians using a cognitive aid when available 9, 25. This reinforces the well‐recognised importance of education and practice with the cognitive aid before emergencies 1, 26.
Two experienced clinicians that were blinded to the presence, absence and nature of the cognitive aid in each scenario undertook the observation of data analysis in this study. One potential source of bias, despite this blinding, was that one of the observers was also present to ensure that study protocols were followed. Furthermore, blinding of the video data for the presence or absence of a cognitive aid is impossible if part of the observation is based around effective use of the cognitive aid. Blinding of the observers to design of the cognitive aid was, however, maintained.
This study used a paper‐based presentation of a cognitive aid. However, given the availability of mobile technology and a culture of instant access to information, future studies of the use of cognitive aids during emergencies will need to compare paper‐based formats with electronic checklists that may be integrated with the clinical monitor.
In conclusion, this prospective multicentre trial shows that the presence and design of a cognitive aid for use during an intra‐operative anaphylaxis improves team co‐ordination, communication, and overall performance. Linear designs of cognitive aid may more effectively improve team performance in complex medical emergencies than complex branched aids.
Acknowledgements
We thank the staff members of the Hunter New England Simulation Centre and Monash Simulation for their invaluable help in the collection and storage of data for this study. No external funding and no conflicts of interest declared.
Appendix 1.
Linear version of the cognitive aid (Immediate and Refractory)
Appendix 2.
Branched version of the cognitive aid (Immediate and Refractory)
Appendix 3.
Simulation scenarios used for testing
3.1. Induction scenario
| Major problem | Intra‐operative anaphylaxis on induction | |
|---|---|---|
| Narrative description | A 54‐year‐old woman for a right hemicolectomy for a caecal tumour develops a life‐threatening allergic response on induction. The scenario is terminated after an adrenaline infusion has commenced or three doses of adrenaline have been given. | |
| Staffing |
Simulator team A faculty member acting as a surgeon is available if called, console operator and debriefer present in control room. |
Participants 2 Anaesthetists 1 Anaesthesia nurse |
| Case briefing |
All participants You have just prepared a 54‐year‐old lady for a right hemicolectomy with the insertion of a 16G peripheral line, central line and arterial line. The surgeon has just gone to the tea room and is ready to start. You will enter the room and induce the patient. She has no previous health issues, has a normal airway (TMD 6.5 cm, good neck extension and mouth opening MP 2) She does not take any medication and has no known allergies. Her Hb is 13.2, Na 140, K 4.7 Creat 65. She has refused an epidural, but wishes to proceed with a GA and PCA for postoperative analgesia |
|
| Simulator set up manikin preparation |
3G manikin with female wig, genitals and hospital gown. Arterial, central and peripheral lines in place. Monitor connected with CVP and arterial line traces |
|
| Room set up |
Normal set up Drugs drawn up: Propofol (20 ml), Morphine (10 ml), Fentanyl (10 ml), Atracurium (5 ml red barrel) |
|
| Simulator operation |
Initially HR 80/min SR, BP 140/80, SpO2 99% After induction HR 140/min SR, BP 70/30 SpO2 94% (Assuming 100% O2). Only improves with adrenaline – no response to metaraminol or similar |
|
| Props needed | Surgical drapes and instruments | |
3.2. Colloid trigger/refractory cardiovascular scenario
| Major problem | Intra‐operative anaphylaxis to colloid (CVS) | |
|---|---|---|
| Narrative description | A 54‐year‐old woman for a right hemicolectomy for a caecal tumour develops a life‐threatening allergic response shortly after induction. The scenario is terminated after adrenaline has been given, the iv colloid infusion ceased and at least 1000 ml of fluid have been given. | |
| Staffing |
Simulator team A faculty member acting as a surgeon is available if called, console operator and debriefer present in control room |
Participants 2 Anaesthetists 1 Anaesthesia nurse |
| Case briefing |
All participants You have just induced a 54‐year‐old lady for a right hemicolectomy as per the previous scenario. As previously she has a 16G peripheral line, central line and arterial line. The surgeon been called to start. You will enter the room shortly after the patient has been induced. She has no previous health issues, has a normal airway (TMD 6.5 cm, good neck extension and mouth opening MP 2) She does not take any medication and has no known allergies. Her Hb is 13.2, Na 140, K 4.7 Creat 65. She has refused an epidural, but wishes to proceed with a GA and PCA for postoperative analgesia |
|
| Simulator set up manikin preparation |
3G manikin with female wig, genitals and hospital gown. Arterial, central and peripheral lines in place. Monitor connected with CVP and arterial line traces |
|
| Room set up |
Normal theatre set up Drugs drawn up: propofol (remainder from 1st scenario), morphine (remainder from 1st scenario), fentanyl (remainder from 1st scenario), Atracurium (remainder from 1st scenario) |
|
| Simulator operation |
Initially HR 80/min SR, BP 140/80, SpO2 99% After induction HR 140/min SR, BP 70/30 SpO2 94% (Assuming 100% O2) Only improves with adrenaline – no response to metaraminol or similar |
|
| Props needed | Surgical drapes and instruments | |
TMD, temperomandibular distance; MP, Mallampati; Hb, haemoglobin concentration; Na, sodium concentration; Creat, creatinine concentration; GA, general anaesthetic; PCA, patient‐controlled analgesia; CVP, central venous pressure; HR, heart rate; BP, blood pressure, SR, sinus rhythm.
3.3. Chlorhexidine central line trigger/refractory bronchospasm scenario
| Major problem | Intra‐operative anaphylaxis to chlorhexidine CVC with refractory bronchospasm | |
|---|---|---|
| Narrative description | A 54‐year‐old woman for a right hemicolectomy for a caecal tumour develops a life‐threatening allergic response shortly after induction. The scenario is terminated after adrenaline has been given, the central line has been removed and a salbutamol bolus 100‐200 mcg given. | |
| Staffing |
Simulator team A faculty member acting as a surgeon is available if called, console operator and debriefer present in control room |
Participants 2 Anaesthetists 1 Anaesthesia nurse |
| Case briefing |
All participants You have just induced a 54‐year‐old lady for a right hemicolectomy as per the previous scenario. As previously she has a 16G peripheral line, central line and arterial line. The surgeon been called to start. You will enter the room shortly after the patient has been induced. She has no previous health issues, has a normal airway (TMD 6.5 cm, good neck extension and mouth opening MP 2) She does not take any medication and has no known allergies. Her Hb is 13.2, Na 140, K 4.7 Creat 65. She has refused an epidural, but wishes to proceed with a GA and PCA for postoperative analgesia |
|
| Simulator set up manikin preparation |
3G manikin with female wig, genitals and hospital gown. Arterial, central and peripheral lines in place. Monitor connected with CVP and arterial line traces |
|
| Room set up |
Normal theatre set up Drugs drawn up: propofol (remainder from 1st scenario), morphine (remainder from 1st scenario), fentanyl (remainder from 1st scenario), atracurium (remainder from 1st scenario) |
|
| Simulator operation |
Initially HR 80/min SR, BP 140/80, SpO2 99% After induction HR 140/min SR, BP 70/30 SpO2 94% (Assuming 100% O2) Only improves with adrenaline – no response to metaraminol or similar |
|
| Props needed | Surgical drapes and instruments | |
Appendix 4.
Counterbalanced randomisation sequence of scenarios and cognitive aid based on latin square design
Cognitive aid (treatment factor)
None (O)
Linear (L)
Branching (B)
Scenarios (nuisance factor)
Induction/Moderately severe (I)
Colloid/Cardiovascular refractory (C)
Chlorhexidine/Respiratory refractory (R)
Each group started with the least complicated scenario first to minimise the learning effect from the more complicated scenario. The advantage of running these complex scenarios (C and R) was to investigate difficulties with both refractory problems (cardiovascular collapse and bronchospasm) and occult triggers (chlorhexidine and colloids) that may be difficult to identify and treat without the cognitive aid.
This limits the potential combinations to starting with the Induction (I) scenario:
| OLB/ICR | OBL/ICR | BOL/ICR | BLO/ICR | LOB/ICR | LBO/ICR |
| OLB/IRC | OBL/IRC | BOL/IRC | BLO/IRC | LOB/IRC | LBO/IRC |
Each treatment condition follows every other condition four times:
Counterbalancing Table 1
| Case | Order O | Order L | Order B |
|---|---|---|---|
| 1 | 1 | 2 | 3 |
| 2 | 1 | 3 | 2 |
| 3 | 2 | 3 | 1 |
| 4 | 3 | 2 | 1 |
| 5 | 2 | 1 | 3 |
| 6 | 3 | 1 | 2 |
| 7 | 1 | 2 | 3 |
| 8 | 1 | 3 | 2 |
| 9 | 2 | 3 | 1 |
| 10 | 3 | 2 | 1 |
| 11 | 2 | 1 | 3 |
| 12 | 3 | 1 | 2 |
In addition, the 12 groups were repeated in a second simulation centre. Although carry‐over effects should be balanced in the design using 12 teams, the reverse conditions were employed in the second 12 teams to balance any unanticipated effects:
| LBO/IRC | LOB/IRC | BLO/IRC | BOL/IRC | OBL/IRC | OLB/IRC |
| LBO/ICR | LOB/ICR | BLO/ICR | BOL/ICR | OBL/ICR | OLB/ICR |
Counterbalancing Table 2
| Case | Order O | Order L | Order B |
|---|---|---|---|
| 13 | 3 | 1 | 2 |
| 14 | 2 | 1 | 3 |
| 15 | 3 | 2 | 1 |
| 16 | 2 | 3 | 1 |
| 17 | 1 | 3 | 2 |
| 18 | 1 | 2 | 3 |
| 19 | 3 | 1 | 2 |
| 20 | 2 | 1 | 3 |
| 21 | 3 | 2 | 1 |
| 22 | 2 | 3 | 1 |
| 23 | 1 | 3 | 2 |
| 24 | 1 | 2 | 3 |
You can respond to this article at http://www.anaesthesiacorrespondence.com
References
- 1. Marshall SD. Use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesthesia and Analgesia 2013; 117: 1162–71. [DOI] [PubMed] [Google Scholar]
- 2. Klein G, Feltovich PJ, Bradshaw JM, Woods DD. Common ground and coordination in joint activity In: Rouse WB, Boff KR, eds. Organizational Simulation. New York: John Wiley and Sons Inc., 2005: 139–84. [Google Scholar]
- 3. Salas E, Prince C, Baker DP, Shrestha L. Situation awareness in team performance: implications for measurement and training. Human Factors 1995; 37: 123–36. [Google Scholar]
- 4. Bould MD, Hayter MA, Campbell DM, Chandra DB, Joo HS, Naik VN. Cognitive aid for neonatal resuscitation: a randomized controlled trial. British Journal of Anaesthesia 2009; 103: 570–5. [DOI] [PubMed] [Google Scholar]
- 5. Manser T, Harrison TK, Gaba DM, Howard SK. Coordination patterns related to high clinical performance in a simulated anesthetic crisis. Anesthesia and Analgesia 2009; 108: 1606–15. [DOI] [PubMed] [Google Scholar]
- 6. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MC. Does every code need a “reader?” Improvement of rare event management with a cognitive aid “reader” during a simulated emergency: a pilot study. Simulation in Healthcare 2012; 7: 1–9. [DOI] [PubMed] [Google Scholar]
- 7. Mosier KL, Palmer EA, Degani A. Electronic checklists: implications for decision making. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 1992; 36: 7–11. [Google Scholar]
- 8. Nelson K, Shilkofski N, Haggerty J, Vera K, Saliski M, Hunt E. Cognitive aids do not prompt initiation of cardiopulmonary resuscitation in simulated pediatric cardiopulmonary arrest. Simulation in Healthcare 2007; 2: 54. [DOI] [PubMed] [Google Scholar]
- 9. Mills PD, DeRosier J, Neily J, McKnight SD, Weeks WB, Bagian JP. A cognitive aid for cardiac arrest: you can't use it if you don't know about it. Joint Commisson Journal on Quality and Patient Safety 2004; 30: 488–96. [DOI] [PubMed] [Google Scholar]
- 10. Degani A, Wiener EL. Cockpit checklists: concepts, design and use. Human Factors 1993; 35: 345–59. [Google Scholar]
- 11. Apfelbaum JL, Hagberg CA, Caplan RA, et al. Practice guidelines for management of the difficult airway: an updated report by the American Society of Anesthesiologists Task Force on management of the difficult airway. Anesthesiology 2013; 118: 251–70. [DOI] [PubMed] [Google Scholar]
- 12. ANZAAG. http://www.anzaag.com/MgmtResources.aspx (accessed 26/08/2014).
- 13. Williams EJ. Experimental designs balanced for the residual effects of treatments. Australian Journal of Chemistry 1949; 2: 149–68. [Google Scholar]
- 14. Landis JR, Koch GG. An application of hierarchical kappa‐type statistics in the assessment of majority agreement among multiple observers. Biometrics 1977; 33: 363–74. [PubMed] [Google Scholar]
- 15. Weller J, Frengley R, Torrie J, et al. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams. BMJ Quality and Safety 2011; 20: 216–22. [DOI] [PubMed] [Google Scholar]
- 16. Schmutz J, Manser T. Do team processes really have an effect on clinical performance? A systematic literature review. British Journal of Anaesthesia 2013; 110: 529–44. [DOI] [PubMed] [Google Scholar]
- 17. Harrison TK, Manser T, Howard SK, Gaba DM. Use of cognitive aids in a simulated anesthetic crisis. Anesthesia and Analgesia 2006; 103: 551–6. [DOI] [PubMed] [Google Scholar]
- 18. Marshall SD, Mehra R. The effects of a displayed cognitive aid on non‐technical skills in a simulated ‘can't intubate, can't oxygenate’ crisis. Anaesthesia 2014; 69: 669–77. [DOI] [PubMed] [Google Scholar]
- 19. Marshall SD. The Effects of Cognitive Aids on Formation and Functioning of Teams in Medical Emergencies. Brisbane, Australia: School of Psychology, University of Queensland, 2015. [Google Scholar]
- 20. Cannon‐Bowers JA, Tannenbaum SI, Salas E, Volpe CE. Defining competencies and establishing team training requirements In: Guzzo RA, Salas E, eds. Team Effectiveness and Decision Making in Organizations. San Francisco, CA: Jossey‐Bass, 1995: 333–81. [Google Scholar]
- 21. Entin EE, Serfaty D. Adaptive team coordination. Human Factors 1999; 41: 312–25. [Google Scholar]
- 22. Seagull FJ, Ho D, Radcliffe J, Xiao Y, Hu P, Mackenzie CF. Just‐in‐time Training for Medical Emergencies: Computer versus Paper Checklists for a Tracheal Intubation Task. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 2007: 725–30.
- 23. Winters BD, Gurses AP, Lehmann H, Sexton JB, Rampersad CJ, Pronovost PJ. Clinical review: checklists – translating evidence into practice. Critical Care 2009; 13: 210–8. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Neal JM, Hsiung RL, Mulroy MF, Halpern BB, Dragnich AD, Slee AE. ASRA checklist improves trainee performance during a simulated episode of local anesthetic systemic toxicity. Regional Anesthesia and Pain Medicine 2012; 37: 8–15. [DOI] [PubMed] [Google Scholar]
- 25. Neily J, DeRosier JM, Mills PD, Bishop MJ, Weeks WB, Bagian JP. Awareness and use of a cognitive aid for anesthesiology. Joint Commission Journal on Quality and Patient Safety 2007; 33: 502–11. [DOI] [PubMed] [Google Scholar]
- 26. Goldhaber‐Fiebert SN, Howard SK. Implementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events? Anesthesia and Analgesia 2013; 117: 1149–61. [DOI] [PubMed] [Google Scholar]