Abstract
Background:
Breaking bad news (BBN) is an important clinical task for physicians. Unfortunately, there is no standard method to teach and assess these skills of anesthesiologists. Although anesthesiology has become a relatively safe medical specialty, complications still occur that require disclosure to patients and their families. Disclosure of bad news can be a significant source of stress for clinicians, especially for those who have low confidence in their BBN skills. Anesthesiologists’ skills in BBN can be improved with simulation-based mastery learning (SBML), an intense form of competency-based learning.
Methods:
An SBML curriculum was developed using the SPIKES (Situation, Perception, Invitation, Knowledge, Emotion, Summarize) framework for BBN and the NURSE (Naming, Understanding, Respecting, Supporting, Exploring) statements for expressing empathy. A pretest-posttest study was conducted from March 2020 to June 2022 to evaluate anesthesiologists’ performance in BBN. Participants completed a 2-hour curriculum consisting of a pretest, didactic session, deliberate practice with feedback, and a posttest. Anesthesiologists were assessed using a 16-item skills checklist.
Results:
Six anesthesiology attendings and 14 anesthesiology fellows were enrolled in the study. Three of 20 participants met the minimum passing score (MPS) at the time of their pretest. All study participants met the MPS on their first posttest (P < .001). The median participant confidence in BBN significantly increased (3 to 4, P < .001). Overall course satisfaction in the curriculum was high, with a median score of 5.
Conclusions:
Our study demonstrates that a BBN SBML curriculum for anesthesiologists significantly improved communication skills and confidence in a simulated environment. Because only 3 participants met the MPS before training, our results suggest that anesthesiologists could benefit from further education to gain effective communication skills and that SBML training may be effective to achieve this result.
Keywords: Simulation-based mastery learning, breaking bad news, anesthesiology education, clinical competence, simulation training
Introduction
Although anesthesia is considered a relatively safe specialty, the estimated incidence of major complications is 2% and that of minor complications is 30% at a large health care system.1 When adverse outcomes occur, it is often the responsibility of the anesthesiologist to disclose the bad news to the patient and their family. The prevailing strategy to teach anesthesiologists the necessary skills to deliver bad news often comes from on-the-job training.2 Despite the regularity in which clinicians have these conversations, many have not had formal training in communication skills.3 Breaking bad news (BBN) is difficult and skills can be improved with simulation-based mastery learning (SBML).4-6
SBML is an intense form of competency-based learning that requires all learners to reach a high level of simulated skills before training completion. Learners engage in deliberate practice with individualized feedback until they can meet or exceed a minimum passing score (MPS). The standard remains the same, only the time required to meet it varies among learners. Although SBML has shown to improve medical students’, residents’, and hospitalists’ BBN skills, it has not been studied in anesthesiologists or in perioperative settings.4-6 The aim of our study was to develop and test an SBML curriculum for perioperative BBN.
Materials and Methods
We performed a pre-post study to evaluate anesthesiologists’ simulated performance before and after BBN SBML training using simulated parents. The study was performed at the Ann & Robert H. Lurie Children’s Hospital of Chicago between March 2020 and June 2022. The Lurie Children’s Institutional Review Board approved this study and the requirement for written informed consent was waived (IRB 2020-3469).
All pediatric anesthesiology fellows and attending physicians were eligible for recruitment. Participants gave verbal consent before participation in the curriculum and then completed a survey on their years of clinical practice, experience with BBN, and confidence. Subsequently, participants completed a 2-hour curriculum consisting of a pretest, didactic session, and deliberate practice with coaching, using simulated parent actors. The didactic session focused on the SPIKES (Situation, Perception, Invitation, Knowledge, Emotion, Summarize)7 framework for BBN and the NURSE (Naming, Understanding, Respecting, Supporting, Exploring)8 statements for expressing empathy. Finally, all learners were required to meet or exceed an MPS on a posttest with a simulated parent before completion of training. Those who were unable to meet this standard participated in further deliberate practice until they could be retested and meet the MPS. Participants were surveyed on their course satisfaction and confidence on a Likert scale of 1 to 5 (1, strongly disagree; 5, strongly agree).
The SPIKES framework was adapted to develop a skills checklist for scoring pretests and posttests.7 The checklist was graded dichotomously as done correctly or not done/done incorrectly. Using the modified Delphi technique, an expert panel consisting of 10 board-certified physicians in anesthesiology (6), critical care medicine (2), palliative care (1), and hospital medicine (1) reached consensus on a final 16-item skills checklist. Using the modified Angoff method,9 the MPS was set at 13 of 16 (81%). In the testing scenarios, participants informed a parent that their child had died in the operating room following an allergic reaction to an antibiotic. In the deliberate practice with feedback portion, participants either informed parents that their child had a medication error or required multiple attempts for intravenous access.
Pretest and posttest scenarios were videotaped, graded by 2 raters with the checklist, and then tested for interrater reliability using the bias-adjusted kappa. Chi-square tests were used to compare the difference in proportions meeting the MPS and the percentage of learners who got each checklist item correct between pretests and posttests. Wilcoxon rank-sum tests were used to compare checklist scores and preconfidence to postconfidence. Stata version 15.0 was used to run all statistical analyses.
Results
Six pediatric anesthesiology attendings and 14 pediatric anesthesiology fellows were enrolled in the study. A summary of participants’ clinical experience is listed in Table 1. The median years of clinical anesthesia practice was 5 (interquartile range [IQR] 5-9). Most participants reported having to deliver bad news in the past, with 11 participants (55%) reporting BBN to a patient at least 3 times or more.
Table 1.
Participants’ Clinical Experience, Test Scores, Confidence, and Course Satisfaction
| Fellow (n = 14) | Attending (n = 6) | Overall (N = 20) | |
|---|---|---|---|
| Clinical experience, median years (IQR) | 5 (5-5) | 14.5 (12-31) | 5 (5-9) |
| Experience breaking bad news, n (%) | 10 (71) | 6 (100) | 16 (80) |
| Times breaking bad news, n (%) | |||
| Never | 4 (29) | 0 | 4 (20) |
| <3 | 5 (36) | 0 | 5 (25) |
| 3-8 | 5 (36) | 2 (33) | 7 (35) |
| >10 | 0 | 4 (66) | 4 (20) |
| Formal training in difficult conversations, n (%) | 8 (57) | 4 (66) | 12 (60) |
| When did you receive education? n (%) | |||
| None | 6 (43) | 2 (33) | 8 (40) |
| Medical school | 6 (43) | 0 | 6 (30) |
| Residency training | 2 (14) | 3 (50) | 5 (25) |
| Faculty | N/A | 1 (17) | 1 (5) |
| Met minimum passing score (13 of 16) at pretest, n (%) | 1 (7) | 2 (33) | 3 (15) |
| Pretest score (0-16 of 16), median (IQR) | 11 (11-12) | 12 (12-12.75) | 12 (11.5-12) |
| Posttest score (0-16 of 16), median (IQR) | 14 (13.5-15) | 14 (13.5-14.5) | 14 (14-14) |
| Precourse confidence, median (IQR) “What was your overall confidence with breaking bad news?” Likert scale 1-5 | 2 (1-3) | 3 (3-4) | 3 (2-3) |
| Postcourse confidence, median (IQR) Likert scale 1-5 | 3.5 (3-4) | 4 (4-4) | 4 (4-4) |
| Overall satisfaction, median (IQR) “What is your overall satisfaction with the course?” Likert scale 1-5 | 5 (4-5) | 5 (5-5) | 5 (5-5) |
Abbreviations: IQR, interquartile range; N/A, not applicable.
Likert scale 1-5 (1, strongly disagree; 5, strongly agree).
Only 3 of 20 participants (15%) met the MPS at pretest. There was no significant difference between fellows and attendings in the proportion who passed their pretest (P = .329) or in the pretest scores (P = .11). All study participants met the MPS on the first posttest (P < .001). One participant’s posttest video data were corrupted and could not be scored. Table 2 shows the percentage of learners who got each pretest and posttest checklist item correct. The median precourse confidence was 3 (IQR 2-3), improving to 4 (IQR 4-4), P < .001 postcourse (Table 1). The overall satisfaction in the course was high with a median score of 5 (IQR 5-5) (Table 1). The bias-adjusted kappa for interrater reliability was 0.76 indicating moderate agreement.
Table 2.
Number (Percent) of Learners Who Performed Each Pretest and Posttest Skills Checklist Item Correctlya
| Item | Pretest Correct | Posttest Correct, | P Value |
|---|---|---|---|
| n = 20 | n = 19b | ||
| n (%) | n (%) | ||
| Creates initial rapport when first walking into room | 19 (95) | 19 (100) | .323 |
| Sits down | 20 (100) | 19 (100) | —c |
| Assumes a comfortable interpersonal distance | 20 (100) | 19 (100) | — |
| Assesses family’s perception or understanding of medical situation before breaking news | 5 (25) | 13 (68) | .007 |
| Asks permission before giving the news | 7 (35) | 15 (79) | .006 |
| Gives a clear and concise “warning shot” | 11 (55) | 18 (95) | .004 |
| Pauses after delivering bad news | 13 (65) | 17 (89) | .076 |
| Delivers bad news within the first minute of the conversation | 11 (55) | 17 (89) | .019 |
| Delivers an empathic statement | 19 (95) | 19 (100) | .323 |
| Suggests a plan for the next step | 19 (95) | 19 (100) | .323 |
| Ensures family understanding | 6 (30) | 9 (47) | .275 |
| Avoids medical jargon | 11 (55) | 14 (74) | .216 |
| Gives information in small chunks | 16 (80) | 17 (89) | .439 |
| Avoids giving information while family very emotional | 16 (80) | 18 (95) | .160 |
| Avoids providing reassurances to family’s emotion | 20 (100) | 18 (95) | .311 |
| Listens attentively | 19 (95) | 19 (100) | .323 |
Minimum passing score 13 of 16 items correct.
One participant’s posttest video was lost due to data corruption.
Dashes indicate no difference between pre/post test values.
Discussion
To our knowledge, this study is novel in using SBML, an extreme form of competency-based learning, to teach BBN to anesthesiologists. Our study showed that a difficult conversation SBML curriculum significantly improved anesthesiologists’ skills in conveying bad news to patients and families. Although 80% of our participants reported having difficult conversations in the past, only 3 of 20 met the MPS at pretest, suggesting that clinical experience cannot be a proxy for competence. Sharma and colleagues6 similarly found that only 10% of hospitalists met the MPS during pretesting of simulated code status discussions. Smith and colleagues5 also found that critical care, oncology, and nephrology fellows performed worse than medical students during pretests of BBN, also suggesting that clinical experience was not sufficient to build these communication skills.
Our curriculum showed significant improvements in specific checklist items of “assesses family’s perception of medical situation before breaking news,” “asks permission before giving the news,” “gives a clear and concise ‘warning shot,” and “pauses after delivering bad news.” Although we saw significant improvement in multiple checklist items, 1 item on the skills checklist did not show much improvement on the posttest. “Ensures family understanding” only showed slight improvement, from 30% to 47%. It is possible that some skills were more amenable to acquisition through the SPIKES and NURSE framework than others. Using these skills during difficult conversations is essential for patient and family-centered care. The role of the anesthesiologist is unique compared with other medical specialties because patients typically do not choose their anesthesiologist and meet their anesthesia team the day of the procedure. Because anesthesiologists interact with their nonsedated patients briefly, there is not much time to form a strong physician-patient relationship.10 Outside of the intensive care unit, anesthesiologists rarely need to deliver devasting news such as an intraoperative death. This may mean that BBN is even more stressful for anesthesiologists when major adverse events do occur. Yet, anesthesiologists commonly have difficult conversations in less severe adverse events such as multiple IV attempts, medication errors, procedural complications, and case delays or cancellations. Having a simulated environment in which anesthesiologists can practice and hone these skills is necessary to become proficient.
Our study should be interpreted within the context of its limitations. First, our testing scenario was more emotionally challenging than our training scenarios. However, when performing an evaluation of the curriculum, all of our participants wanted to include the BBN of an intraoperative death scenario. Although this is a rare outcome, participants expressed that practicing a low-incidence, “malignant hyperthermia-like scenario” was important for their education. Simulation provides a “safe” place for participants to practice skills in a low-stakes environment. Second, our study included both pediatric anesthesia fellows as well as attending anesthesiologists. Our attending anesthesiologist cohort had significantly more years of clinical experience than our fellow cohort. Although it is possible that clinical experience could lead to improved BBN competence, we found no statistically significant difference between the fellows and attendings in terms of the proportion who passed their pretest or in baseline pretest scores. Although both groups were able to be brought to a mastery level, it is possible that our SBML curriculum would have an even greater effect on a less experienced cohort such as medical students and residents. Third, although we did find a significant difference between the pretest and posttest scores, there was no long-term follow-up to assess for skill retention. Finally, our study was limited by using a small group of volunteers with a pre-post design. However, our SBML curriculum resulted in statistically significant improvement in skills and confidence in having difficult conversations. Conveying bad news is a teachable skill and our curriculum provides a framework for anesthesiologists to effectively communicate with patients. Future work includes expanding our SBML curriculum to the larger anesthesia care team, such as certified registered nurse anesthetists, and to evaluate this curriculum’s effect on physician burnout and empathy levels.
Funding Statement
Funding: This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Footnotes
Conflicts of interest: None
References
- 1.Belcher AW, Leung S, Cohen B, et al. Incidence of complications in the post-anesthesia care unit and associated healthcare utilization in patients undergoing non-cardiac surgery requiring neuromuscular blockade 2005-2013: a single center study. J Clin Anesth. 2017;43:33–8. doi: 10.1016/j.jclinane.2017.09.005. [DOI] [PubMed] [Google Scholar]
- 2.Berger J, Blatt B, McGrath B, Greenberg L, Berrigan M. Relationship express: a pilot program to teach anesthesiology residents communication skills. J Grad Med Educ. 2010;2(4):600–3. doi: 10.4300/JGME-D-10-00012.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Messerotti A, Banchelli F, Ferrari S, et al. Investigating the association between physicians self-efficacy regarding communication skills and risk of “burnout”. Health Qual Life Outcomes. 2020;18(1):271. doi: 10.1186/s12955-020-01504-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Vermylen JH, Wood GJ, Cohen ER, et al. Development of a simulation-based mastery learning curriculum for breaking bad news. J Pain Symptom Manage. 2019;57(3):682–7. doi: 10.1016/j.jpainsymman.2018.11.012. [DOI] [PubMed] [Google Scholar]
- 5.Smith MM, Secunda KE, Cohen ER, et al. Clinical experience is not a proxy for competence: comparing fellow and medical student performance in a breaking bad news simulation-based mastery learning curriculum. Am J Hosp Palliat Care. 2023;40(4):423–30. doi: 10.1177/10499091221106176. [DOI] [PubMed] [Google Scholar]
- 6.Sharma RK, Szmuilowicz E, Ogunseitan A, et al. Evaluation of a mastery learning intervention on hospitalists’ code status discussion skills. J Pain Symptom Manage. 2017;53(6):1066–70. doi: 10.1016/j.jpainsymman.2016.12.341. [DOI] [PubMed] [Google Scholar]
- 7.Berkey FJ, Wiedemer JP, Vithalani ND. Delivering bad or life-altering news. Am Fam Physician. 2018;98(2):99–104. [PubMed] [Google Scholar]
- 8.“Responding to Emotion: Respecting”. VitalTalk. May 09, 2019. [Accessed October 25, 2022]. https://www.vitaltalk.org/guides/responding-to-emotion-respecting/
- 9.Downing SM, Tekian A, Yudkowsky R. Procedures for establishing defensible absolute passing scores on performance examinations in health professions education. Teach Learn Med. 2006;18(1):50–7. doi: 10.1207/s15328015tlm1801_11. [DOI] [PubMed] [Google Scholar]
- 10.Kopp VJ, Shafer A. Anesthesiologists and perioperative communication. Anesthesiology. 2000;93(2):548–55. doi: 10.1097/00000542-200008000-00035. [DOI] [PubMed] [Google Scholar]