“Best Case/Worst Case”: Training surgeons to use a novel communication tool for high-risk acute surgical problems

Abstract

Context

Older adults often have surgery in the months preceding death, which can initiate post-operative treatments inconsistent with end-of-life values. “Best Case/Worst Case” (BC/WC) is a communication tool designed to promote goal-concordant care during discussions about high-risk surgery.

Objective

To evaluate a structured training program designed to teach surgeons how to use BC/WC.

Methods

Twenty-five surgeons from one tertiary-care hospital completed a two-hour training session followed by individual coaching. We audio recorded surgeons using BC/WC with standardized patients and 20 hospitalized patients. Hospitalized patients and their families participated in an open-ended interview 30 to 120 days after enrollment. We used a checklist of 11 BC/WC elements to measure tool fidelity and surgeons completed the Practitioner Opinion Survey to measure acceptability of the tool. We used qualitative analysis to evaluate variability in tool content and to characterize patient and family perceptions of the tool.

Results

Surgeons completed a median of 10 out of 11 BC/WC elements with both standardized and hospitalized patients (range 5 to 11). We found moderate variability in presentation of treatment options and description of outcomes. Three months after training, 79% of surgeons reported BC/WC is better than their usual approach and 71% endorsed active use of BC/WC in clinical practice. Patients and families found that BC/WC established expectations, provided clarity and facilitated deliberation.

Conclusions and Relevance

Surgeons can learn to use BC/WC with older patients considering acute high-risk surgical interventions. Surgeons, patients, and family members endorse BC/WC as a strategy to support complex decision making.

Background

Older adults often undergo surgery in the months preceding death, which can lead to post-operative intensive care unit (ICU) admission and prolonged periods of recovery with progressive decline in functional status.^1–6 Yet, most older people prefer care focused on the relief of symptoms rather than aggressive treatments including ICU care and hospitalization near the end of life.^7–9 Despite widespread preference for symptom-focused care, the use of ICU services before death has increased over time.^1,2 Preoperative communication between surgeons and frail older patients who face a decision about high-risk surgery is a modifiable contributor to the use of treatments that are discordant with patient preferences.^10–14 Efforts to improve communication during the decision-making process could decrease unwanted, burdensome treatments near the end of life.

For patients who develop life-threatening surgical conditions, preoperative decision making is complex. Given the life-altering consequences and substantial prognostic uncertainty, the “right” decision can only be reached by exploring each individual patient’s goals and values. Efforts to improve preference-sensitive medical decisions have focused on the concept of shared decision making and the development of disease-specific decision aids.¹⁵ While decision aids can improve decision making for many medical choices,¹⁶ they are not applicable or available for in-the-moment treatment decisions for patients who face acute, life-threatening illness.

To improve complex surgical decision making for older adults, we developed a novel communication tool called “Best Case/Worst Case” (BC/WC).¹³ Building on an established conceptual model of shared decision making¹⁷ and feedback from seniors and surgeons,¹⁸ we designed the BC/WC tool for in-the-moment, acute surgical decisions. Essential tool elements include depiction of two or more treatment choices, creation of a pen-and-paper graphic aid, use of narrative to tell a story about how the patient might experience the outcomes in the best and worst case scenarios, estimation about the most likely outcome, description of how the treatment option impacts the larger context of the patient’s overall health, and providing a treatment recommendation at the conclusion of the discussion. During the conversation, the surgeon uses narrative to describe the best and worst possible outcomes of each treatment option and creates a graphic aid to illustrate the range and estimated probability of each outcome to leave with the patient and family for future deliberation (Figure 1).¹³ In focus groups, seniors and surgeons praised the tool for depicting “both sides of the story” and clearly establishing that the patient has a choice between two treatment options.¹⁸ Seniors endorsed BC/WC because it provides an opportunity for patients to consider and incorporate personal values and goals when making a complex medical decision.¹⁸

Figure 1.

Example of graphic aid component of BC/WC tool for a patient with a life-threatening surgical condition. The star represents the best case scenario, the box represents the worst case scenario, and the oval designates the most likely outcome. The location of the oval indicates whether the most likely scenario is more similar to the best case or the worst case. Adapted with permission from “Navigating High Risk Procedures with More than Just a Street Map” by ML Schwarze, JM Kehler, TC Campbell, 2013, Journal of Palliative Medicine p. 1170.

Whether surgeons can learn to use this tool for routine clinical practice is unknown. We designed a single-center pilot study to evaluate a 2-hour training program designed to teach surgeons to use BC/WC in a clinical setting. The aims of this study were to demonstrate feasibility of surgeon training, to evaluate fidelity and variability in the use of BC/WC by surgeons, and to assess the acceptability and perceptions of the BC/WC communication tool among surgeons, patients and families.

Methods

TRAINING PROGRAM DESIGN

To accommodate the needs of adult learners, we modeled our training session on Kolb’s cycle of experiential learning¹⁹ which requires learning new skills through practice. After an educational experience, learners benefit from time for reflection in order to formulate a conceptual understanding about what occurred. The learner can use this information to plan for the next experience, modifying behavior to move towards a specified goal. We also adapted Ericsson’s work in deliberate practice and individual coaching framework,²⁰ to provide ongoing challenge through an expert coach to ensure steady progression to expertise. We developed a 2-hour training session to provide experience, followed by explicit time for reflection with a coach, and a repeat experience to apply newly learned skills. Our coaches were experts in the fields of palliative care, patient-physician communication and adult education.

STUDY DESIGN AND PARTICIPANTS

We invited all attending surgeons who practice acute care general, cardiothoracic or vascular surgery at the University of Wisconsin to participate. Each enrolled surgeon completed a 2-hour training session scheduled at his/her convenience. Training included a 15-minute lecture highlighting the essential tool elements followed by demonstration with a standardized patient. After the didactic component, each surgeon practiced using the BC/WC tool with two standardized patients in two different specialty-specific simulated cases. A coach observed each case and provided real-time, one-on-one feedback to the surgeon throughout and upon completion of the first conversation. The surgeon then used BC/WC in a second case without interruption; we audio-recorded and transcribed verbatim this conversation and collected all graphic aids. Within two months of the training session, coaches met with surgeons individually to address questions and encourage utilization.

To measure surgeon’s ability to use the tool in a clinical setting, we screened hospitalized patients cared for by trained surgeons. We contacted the patient’s surgeon prior to enrollment to ensure the surgeon was offering a choice between at least two treatment options. We included patients 65 years of age and older facing a decision between an operation and a less invasive alternative (e.g. minor surgery, medical management or supportive/palliative care) who were at high risk for surgical complications. To determine high risk, we screened patients for multimorbidity and confirmed with the treating team that the patient was frail, based on a composite clinical assessment including reduced functional status, recent falls and weight loss. We included patients without decision-making capacity if a surrogate was present to participate. We excluded patients and surrogates who did not speak English or had severe hearing impairment. We audio-recorded and transcribed verbatim the decision-making conversation between the surgeon and the patient and retained a copy of the graphic aid. The University of Wisconsin Health Sciences Institutional Review Board approved this protocol.

MEASUREMENT OF BC/WC FIDELITY

To evaluate surgeons’ use and fidelity to the BC/WC tool, we scored transcripts of the final standardized patient conversations and each inpatient conversation based on 11 tool elements (Supplemental Appendix). Two reviewers (JK, LT) used an 11-point checklist to independently score each transcript and the associated graphic aid, assigning 1 point per element performed by the surgeon. Cohen’s kappa coefficient was 0.74, demonstrating substantial inter-rater agreement.²¹ To equally weigh each reviewer’s assessment, we averaged the two total scores and present this as final score for each conversation.

ANALYSIS OF CONTENT VARIABILITY

BC/WC is a framework that structures the conversation between surgeon and patient and requires the use of narrative to describe outcomes. As such, the content of the decision-making conversation is dependent upon the surgeon’s interpretation of the relevant evidence within the context of the patients’ overall health. We sought to measure the variability of clinical information presented to standardized patients when different surgeons were faced with the same hypothetical scenario. To evaluate variability between surgeons, four investigators (JK, LT, JT, MS) independently reviewed the transcripts and graphic aids for 13 BC/WC standardized patient conversations (9 general surgeons describing treatment of an anastomotic leak and 4 vascular surgeons discussing treatment of limb ischemia with gangrene). We used inductive qualitative analysis²² to identify key domains of variability within the clinical content presented by the surgeons. We then used comparative qualitative analysis²² to evaluate and describe variability between surgeons facing identical, simulated clinical scenarios.

SURGEON EVALUATION OF BC/WC

We administered a measure of decision aid acceptability to each surgeon participant at 3 and 6 months after the training session. We used a modified version of the Practitioner Opinion Survey (POS)²³ to determine if surgeons were actively using BC/WC in clinical practice and to measure surgeons’ perceptions of time required for BC/WC use, ease of use, and benefit of BC/WC compared to usual care. We used descriptive statistics to summarize survey results and the Student t test to compare responses between 3 and 6 months, and we defined a two-tailed p-value less than 0.05 as statistically significant.

PATIENT AND FAMILY EVALUATION OF BC/WC

We contacted all enrolled, hospitalized patients or family members between 30 and 120 days after the BC/WC decision-making conversation to participate in an open-ended face-to-face interview. We scheduled interviews at least 30 days after the BC/WC decision-making conversation to account for the prolonged hospitalization and recovery that older, frail patients with serious illness typically face and to respect the families of patients who died. To limit problems with recall, we conducted all interviews no more than 120 days after the BC/WC conversation. The interview guide was developed and iteratively revised by the multidisciplinary team of study investigators. The open-ended questions were designed to elicit patient or surrogate perspectives about the decision-making process, whether the treatment decision was concordant with the patient’s values, the impact of best and worst case language and the graphic aid, and the acceptability of study participation. We audio-recorded and transcribed these conversations verbatim and used inductive qualitative analysis to characterize patients’ and families’ perceptions of BC/WC.

Results

PARTICIPANT CHARACTERISTICS

Twenty-five surgeons attended a training session. Twenty-nine of 30 eligible surgeons enrolled in the study, but 4 (14%) were unable to attend a training session due to schedule conflicts (Figure 2). One surgeon left the training session early due to a clinical conflict, and did not complete the final evaluation simulated case. Seven surgeons (28%) rescheduled the training session at least once (range 1 – 4 rescheduled sessions). Surgeon and patient characteristics are described in Table 1. Surgeons were predominantly male (84 %) and had been in practice for a median of 9 years. After completing training, 19 surgeons (76%) participated in one individualized follow-up meeting with a coach. We were able to enroll at least one patient for 12 of the trained surgeons. The range of hospitalized patients enrolled per trained surgeon was 0 to 3. Sixty-five percent of patients were female. Patients had a range of acute surgical problems; bowel obstruction was the most common condition (35%). Follow-up interviews with patients or family members took place a median of 90 days after the BC/WC decision-making conversation.

Figure 2.

Enrollment and Study Completion of Surgeons and Hospitalized Patients.

Table 1.

Participant Characteristics

Surgeon Characteristics (n = 25)	n (%)
Male	21 (84)
Surgical Specialty
General surgery	14 (56)
Vascular	5 (20)
Cardiac	3 (12)
Thoracic	3 (12)
Median Years in Practice	9
Hospitalized Patient Characteristics (n = 20)	n (%)
Age
65 – 69	2 (10)
70 – 79	6 (30)
80 – 89	10 (50)
90 – 95	2 (10)
Female	13 (65)
Race
White	19 (95)
Black	1 (5)
Surgical Condition
Bowel obstruction	14(70)
Valvular heart disease	1 (5)
Peripheral Vascular Disease	3 (15)
Fracture or Other Trauma	2 (10)
Death within 30 days of Treatment Initiation	5 (25)
Follow-up Interview Participant Characteristics (n = 14)	n (%)
Interview Participant
Patient only	1 (7)
Patient and Adult Child	4 (29)
Adult child only	7 (50)
Spouse only	1 (7)
Niece only	1 (7)
Time to interview,* median (range), days	90 (49– 119)

^*Time from use of BC/WC in hospital to follow-up interview

BC/WC TOOL FIDELITY

During training, surgeons completed a median 10 out of 11 tool elements (range 5 to 11) in the second simulated case. All surgeons presented two distinct treatment options and successfully described the best and worst cases for each treatment. The BC/WC element surgeons most commonly omitted in simulation was “make a recommendation” for a specific treatment option after learning the patient’s preferences, which surgeons performed in only 55% of training conversations. When using BC/WC in clinical practice with hospitalized older adults, trained surgeons continued to achieve a median of 10 out of 11 tool elements (range 7 to 11). Surgeons presented both a best and worst case for 2 distinct treatment options in 92% of conversations with hospitalized patients, yet failed to make a clear treatment recommendation in 61% of conversations.

When presented with the same simulated clinical case, surgeons used similar narratives to describe the best and worst case outcomes for each treatment option. For the “worst case” scenario of operative intervention during simulated cases, almost every surgeon discussed the risk of intensive care unit admission and use of life support preceding death. When explaining the “best case” and “worst case” of non-surgical supportive care, one surgeon said:

The best case is he would die but he’d live a few weeks at home. We wouldn’t do any of those procedures on him. There would be no risk of that. I think we’d be able to keep him comfortable […]. And the worst case is that this infection, this, we can’t suppress it, and he dies more quickly. He dies in a matter of days. We certainly can keep him comfortable, but if he never recovers enough to be conscious to speak to, I think that might be the worst case that you would see.

Another surgeon discussing non-surgical supportive care with the same simulated patient described very similar best and worst cases:

“[…] we can make him pain free, and he may come home, and he may at least have, you know a lot more days where he can spend with his family, and have his good byes […]. the worst case scenario [is] that things may progress quickly, and he may die from this, you know, within a few days.”

However, many surgeons struggled to provide an explicit treatment recommendation, an essential element of BC/WC. These surgeons resisted coaching during this component of the simulated cases, stating:

But I usually don’t like to make my recommendation because […] I think if a physician makes a recommendation […] I feel it’s kind of manipulating because so I feel my job is to provide all the accurate information, […] then I think the decision, yes or no, needs to be made by the patient and family.

Another surgeon expressed a similar concern, “I never like to tell anyone what they should do. I can tell them what I can do, and what the outcomes from that can be, but I never want to, I want them to make this,[…] it’s their choice.” This was not a stumbling point for all surgeons. Here a different surgeon spontaneously provided a preference-sensitive recommendation in simulation:

So while I can't ultimately make the decision for you, I can certainly try to help you through it, and perhaps give you my thoughts on this […] I think the problem here, as I see it, is if her ultimate goal would be to live independent, and we really can't achieve that, then perhaps the kindest thing to do might be just to help her onto the next stage of life comfortably. And just sort of keep her comfortable, and let her die with her dignity.

CONTENT VARIABILITY

We identified two major domains of variability between surgeons using BC/WC for the same hypothetical patient: the technical strategy for treatment and the style of narrative detail. Surgeons offered a range of treatment options when faced with an identical, hypothetical case. For the same 75 year-old patient with new-onset limb ischemia and gangrene with a history of stroke who was dependent on his daughter for all activities of daily living, we identified 3 distinct surgical options and 2 distinct non-surgical options offered by 4 vascular surgeons. Surgeons offered below-knee amputation, above-knee amputation, or vascular bypass with toe amputation. Surgeons offered two different non-surgical options: either supportive medical care with antibiotics and pain medication or supportive medical care focusing on comfort alone. We observed more consistency among general surgeons caring for a hypothetical, frail 82 year-old woman with heart failure who developed a small intestine anastomotic leak. Nine general surgeons offered one of two distinct surgical options (abdominal drainage or fistula repair). All but one surgeon offered the same non-surgical option: intensive care unit admission, wound care and parenteral nutrition. One outlier offered comfort-focused care without ICU admission.

We also identified variability in the details surgeons chose to describe clinical outcomes. Some surgeons focused on specific medical and technical elements for each possible scenario; they described intensive care unit admission, mechanical ventilation, wound care, nutrition, and antibiotics. For example, one surgeon described a best case scenario with technical details by saying, “we would control the output of her intestines to that fistula with wound care, which would require a plastic bag and pieces, and it would have to be changed.” In contrast, other surgeons emphasized patient-focused outcomes including the ability to eat or drink, return home, interact with family, or control pain. One surgeon described the best case by saying, “she still probably wouldn’t be able to eat or drink. She would still probably be in a long-term care facility […] and still, probably because of all of her other medical problems and issues, probably dying from this within a couple of months.”

SURGEON EVALUATION OF BC/WC

Twenty-four (96%) of the trained surgeons completed the Practitioner Opinion Survey at 3 and 6 months. Three months after training, 96% of surgeons respondents reported that BC/WC is easy to use, 79% felt that the BC/WC tool is better than their usual approach for helping patients make decisions, and 71% reported actively using BC/WC in clinical practice outside the scope of the research study. However, only 38% of surgeons believed BC/WC saved time (Figure 3). Surgeon responses were sustained at 6 months.

Figure 3.

Practitioner Opinion Survey results at 3 and 6 months.

*indicates percent of surgeons who agreed or strongly agreed with statement

† indicates percent of surgeons who responded “yes”

There were no significant differences between surgeon responses at 3 and 6 months (p value > 0.05)

PATIENT AND FAMILY EVALUATION OF BC/WC

We identified three important functions of the BC/WC framework during patient and family follow-up interviews (Table 2). First, patients and families reported that surgeons who used BC/WC clearly defined treatment choices and encouraged comparison between the two treatments. Second, by showing the range of possible outcomes, surgeons established expectations and helped patients and families prepare for possible adverse events. Finally, patients and families valued the graphic aid as a tangible reference to facilitate future deliberation and inform family members who were not present for the discussion. Several patients and family members had saved the graphic aid months after hospitalization and reproduced the diagram at the follow up interview without prompting.

Table 2.

Patient and Family Perceptions of BC/WC

Description of BC/WC Tool	Representative Quotes
Provides Clarity about Options ◦ Defines treatment choices ◦ Allows comparison between treatment options	“I could ask specific questions if I didn’t understand, there wasn’t any vocabulary on here that was unknown to me…It just, it focused my attention on what the clear choices were.” “I just thought it was a great diagram and a great way to show people […] some things you’re faced with. And this is the best case, and this is the worst case…”
Establishes Expectations ◦ Demonstrates a range of outcomes ◦ Allows preparation for adverse events	“Allowing anyone to see a big picture of something, best case, worst case, um, no guarantees in the middle, um, helps people make important decisions. Also prepare them in case the worst case happens.” “I wasn’t surprised. I knew what the choices were, I knew why they would do this […]. I think it even says something about, you know, death. I mean that’s there. I knew that. So that was not a surprise to me.”
Graphic Aid Facilitates Deliberation ◦ Provides reference for future questions ◦ Can be used to facilitate deliberation or explanation with family and loved ones	“I think it greatly affected the way we discussed it, because, you know, it’s just spelled out. I don’t think there’s any other information […] that would have helped […]. And I looked at it again. I still have it.” “It helped to have it on paper in front of you. It helped a lot. […] This is the first time I’ve ever seen it on paper. And you forget a lot what they say, you don’t remember everything, but with this on paper, you can go back and look at it.” “I just found that to be a tremendous help. And then I could share with my siblings too so that we would get together, over at my dad’s and showed this to him, he would kinda remember, but then he wouldn’t remember, but then when he saw the diagram, and he would look at it again, it helped him.”

Discussion

Surgeons can learn to use BC/WC with high fidelity in clinical practice after completing a 2-hour structured training and individualized coaching program. In addition, surgeons found that BC/WC is easy to use and is an improvement on usual care; most trained surgeons reported adopting BC/WC into their clinical practice. After a decision-making conversation with trained surgeons, patients and their family members praised their surgeon for providing clarity about treatment options and establishing expectations. Although surgeons can readily learn and adopt the framework of BC/WC to present similar outcome narratives to patients and families, we identified modest variability in treatment choices and narrative details between individual surgeons. It is notable that surgeons reported discomfort providing a specific treatment recommendation as it conflicted with their understanding about how to support patient autonomy. This was reflected in their clinical use of the tool as less than half provided a treatment recommendation for hospitalized patients.

We have previously shown that seniors and surgeons approve of BC/WC in a hypothetical setting,¹⁸ and this study demonstrates the feasibility and acceptability of implementing BC/WC into clinical practice with a structured training program for academic surgeons. Furthermore, BC/WC is valued and embraced by surgeons, a crucial component of sustainable adoption. Because our training program included reproducible elements based on seminal theories of adult education,^19,20 we posit these methods can be applied on a larger scale to facilitate widespread, high-fidelity implementation of BC/WC.

We did observe variability between surgeons in our study, as several surgeons offered different treatment options for the same simulated clinical case. Despite the variability in treatment approach, the narratives surgeons used to describe outcomes remained similar, suggesting a general consensus from surgeons about treatment outcomes even with different surgical interventions. This variation is similar to variation currently observed in clinical practice. Surgeons often vary widely in their quantitative risk assessment²⁴ or judgement about surgical indications,²⁵ yet striving to provide precise, numerical risk-estimation may be unnecessary or even misleading to patients.²⁶ Furthermore, in high-risk clinical scenarios, precise risk predictions and high-quality evidence are often lacking or difficult to extrapolate.²⁶ Instead of reliance on risk prediction, BC/WC utilizes an alternate strategy, providing a framework for patients and families that describes composite estimates about what is likely to happen and creating boundaries around what is possible. Patients and families in our study used BC/WC to visualize and prepare for treatment outcomes, suggesting their communication needs were met by the BC/WC strategy.

The BC/WC element most frequently omitted by surgeons was “provide a treatment recommendation.” During standardized patient sessions, several surgeons explained their rationale; they believed their recommendation would violate patient autonomy. The misperception that physician recommendations compromise patient autonomy is not novel and is well described.^27,28 Yet, consistent with the goals of shared decision making, the BC/WC tool includes a physician recommendation that integrates patient preferences with the relevant therapeutic possibilities. This strategy is designed to respect and enhance, not violate, patient autonomy.²⁷ Although some surgeons easily incorporated patient preferences into treatment recommendations, our 2-hour training session was not able to overcome deep-rooted cultural notions that many attending surgeons have about autonomy and surgical decision making. We have targeted this issue as we make improvements to our training program, but we suspect this will be a difficult challenge to overcome given current practice patterns and widespread physician beliefs.²⁹

Our findings are important for physicians and researchers who aim to improve shared decision making for other high-stakes, complex medical decisions beyond surgery. BC/WC is distinct from typical decision aids that provide standardized content and numeric representation of risks^16,30 because it allows physicians to personalize the tool to accommodate the relevant clinical features of each patient’s decision. In addition, BC/WC requires only pen and paper, so it is readily available for acute clinical scenarios and face-to-face conversations between patients, families and physicians. The surgeons in this study found BC/WC easy to use, and the flexibility and accessibility of BC/WC may overcome barriers to routine decision aid use for other physicians from different specialties such as critical care medicine, palliative care and oncology who often treat patients with similar levels of complexity and acuity.

This study has several important limitations. We relied on local experts to conduct the didactic sessions and one-on-one coaching, which is resource intensive. In addition, surgeons cancelled and rescheduled multiple training and coaching sessions due to patient care and operating room conflicts, highlighting the burdens of a structured training curriculum for busy clinicians. To address these barriers, we have developed an instructional video (https://www.youtube.com/watch?v=FnS3K44sbu0) linked to a training program to improve the scalability of this intervention. Future evaluation of a revised, lower-resource implementation package is required to determine its effect. Given the challenges of conducting clinical research with frail, hospitalized older patients, were only able to enroll hospitalized patients for 12 out of the 25 trained surgeons, and so our findings may not reflect the range of surgeon practice after BC/WC training. Lastly, patient and family interviews were held between 30 and 120 days after the BC/WC intervention to facilitate collection of perspectives that incorporated longer-term patient outcomes and to respect the families of patients who died. However, the delay between the use of the BC/WC tool during a decision-making conversation and the interview may have affected the accuracy or completeness of the patients’ and families’ recall of events.

Conclusions

Academic surgeons can use the BC/WC communication tool in clinical practice with high fidelity after completing a structured training program. Surgeons, patients and family members endorse the BC/WC tool as a method to support complex decision making, but further investigation is required to determine the effect of BC/WC on clinical outcomes.