How to say “I don’t know”: development and evaluation of workshops for medical students and surgical residents on communicating uncertainty using the ADAPT framework

Abstract

Purpose

Uncertainty, or the conscious awareness of having doubts, is pervasive in medicine, from differential diagnoses and the sensitivity of diagnostic tests, to the absence of a single known recovery path. While openness about uncertainty is necessary for shared decision-making and is a pillar of patient-centered care, it is a challenge to do so while preserving patient confidence. The authors’ aim was to develop, pilot, and evaluate an uncertainty communication curriculum to prepare medical students and residents to confidently navigate such conversations.

Methods

The authors developed ADAPT, a mnemonic framework to improve student comprehension and recall of the important steps in uncertainty disclosure: assess the patient’s knowledge, disclose uncertainty directly, acknowledge patient emotions, plan next steps, and temper expectations. Using this framework, the authors developed, piloted, and evaluated an uncertainty communications course as part of an ongoing communication curriculum for second year medical students in 2020 and with surgical residents in 2021.

Results

Learner confidence in uncertainty communication skills significantly increased post-class. Resident confidence in disclosing uncertainty was significantly correlated with observer ratings of their related communication skills during simulation. Students expressed positive experiences of the class, noting particular appreciation for the outline of steps included in the ADAPT framework, and the ability to observe a demonstration prior to practice.

Conclusions

The ADAPT communication curriculum was effective at increasing learner confidence and performance in communicating uncertainty. More rigorous evaluation of the ADAPT protocol will be important in confirming its generalizability.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44186-022-00075-4.

Background

Uncertainty is defined as the conscious awareness of having doubts; it is pervasive in medicine, from differential diagnoses and the sensitivity of diagnostic tests, to the lack of a single known recovery path [1]. Given variability in patient responses to disease processes and interventions, virtually every decision in medical practice is made with a level of uncertainty [2]. However, clinicians often fail to communicate their uncertainty to patients: in one analysis of over 1000 encounters, PCPs and surgeons discussed the uncertainty surrounding their treatment plan in 1% of encounters for basic decisions, 6% for intermediate decisions, and 17% for complex decisions [3]. The hesitancy to discuss uncertainty may stem from a host of factors: the medical field culture to strive for perfection, fear of losing patient trust, or the lack of skills to have the complex conversation about uncertainty [4].

The ability to discuss uncertainty is critical to ensuring patient autonomy: shared decision-making necessitates transparency and is a pillar of patient-centered care. Moreover, discomfort expressing uncertainty within medical teams may lead to premature decision-making [5]. It is thus important to train students to accept and confidently navigate conversations surrounding uncertainty in a way that maintains, or even improves, the doctor-patient relationship, and facilitates shared decision-making [2]. Though previous studies have yielded inconsistent results on the effect of communicating uncertainty to patients [6], a 2017 systematic review concluded that studies unanimously found patient satisfaction and trust after physician disclosure of uncertainty to be dependent on how the uncertainty was communicated [7]. Such variability in patient response based on physician communication technique further indicates a need for standardized, evidence-based training on this important form of communication.

While the past decade has seen calls to increase inclusion of uncertainty topics in medical curricula, we were unable to identify established curricula in the literature regarding the communication of uncertainty. Given the growing importance of this communication topic noted in recent literature, and with suggestions from several surgical faculty—including a colorectal surgeon, acute care surgeon, and surgical residency program director, all in practice for over 10 years—we chose to introduce a module on discussing uncertainty in ongoing communication courses for medical students and surgical residents. Mnemonics and simulated patient practice have both been shown to enhance learners’ medical communication performance [8, 9]. Thus, we believed development of a mnemonic framework, coupled with simulated practice of our framework, would offer an effective method to teach communicating uncertainty. As the COVID-19 pandemic necessitated a move to online learning to comply with social distancing protocols, our aim was to design, pilot, and evaluate a framework for conversations about uncertainty in a virtual class for medical students, and to then implement a refined curriculum with surgical residents.

Methods

ADAPT framework development (Fig. 1)

Fig. 1.

Adapting to uncertainty in medicine: a framework to tell patients “I don’t know”

We developed a mnemonic framework to assist students in approaching uncertainty conversations: a 20 min focus group workshop including surgeons, physicians, social workers, and patients created a list of steps they identified as necessary for physicians to take in uncertainty conversations and yielded a list of key teaching points. Four themes derived from the focus group were similar to the four broad domains listed in Simpkin and Armstrong’s narrative review on uncertainty communication strategies [2]; our focus group identified a fifth domain of tempering patient expectations. The final set of steps were worked into a mnemonic (ADAPT) for improved recall. Adaptability is an important skill to employ in situations without a known outcome, and the letters of ADAPT stand for the key uncertainty discussion steps: assess the patient’s current knowledge, disclose uncertainty directly, acknowledge patient emotions, plan next steps with patient, and temper expectations. These steps are both easy to recall and conform to important uncertainty discussion categories denoted in the existing literature [2].

Setting and communication program

The Inova Fairfax Medical Campus includes a level I trauma center and 900-bed tertiary care referral center. The hospital’s academic affiliation with the University of Virginia (UVA) School of Medicine enables students to engage in research and other educational opportunities at the hospital during the summer following their 1st year. As COVID-related restrictions limited medical student access to the campus, we invited second-year medical students (M2s) from the UVA and George Washington University (GWU) Schools of Medicine to participate in a virtual communication course in 2020. The Human Subjects Protection Program at Inova Health System approved this research as an exempt study of standard educational practices. Given positive feedback from the course pilot in July 2020, the course—including the module on uncertainty communication—was tested with the Inova surgical residents during their ongoing quarterly communication training program in April 2021.

Communication training for M2s included a five-session simulation-based curriculum using Zoom technology (San Jose, CA). The overarching skills focus for the course was “fostering a connection” with the patient and family in telemedicine appointments using empathic communication. Each class featured a unique communication task: bad news discussions, explaining complications, managing bias, apologizing for a mistake, and communicating uncertainty. Communication tasks were presented by a different specialist each class, including geriatrics, surgery, family medicine, pediatrics, and emergency medicine. The “Communicating Uncertainty” module was the final class presented over the summer. Faculty and students found that the 1 h allotted to the first four classes did not leave enough time for reflection and discussion; thus, this final class was elongated to 1.5 h.

This 1.5 h session began with a dual demonstration: an attending surgeon demonstrated communicating uncertainty to a patient and illustrated a few common errors; students then discussed what the surgeon did well and what improvements could be made. The surgeon followed with a demonstration of an ideal disclosure of uncertainty. A 10 min didactic discussion ensued, including explanation of the ADAPT framework for disclosing uncertainty (Fig. 1). Following this didactic discussion, students practiced communicating uncertainty with actors in small groups of 4–6 learners using Zoom breakout rooms; the scenario featured a recently admitted concussion patient’s parent calling the Emergency Department for information on their son’s prognosis. The student was instructed to answer their questions using the techniques described in the didactic session.

Each room was facilitated by one faculty member and featured two simulated patients (SPs). SPs were drawn from our pool of former-patient volunteers from the Trauma Survivors Network (TSN) [10] as well as staff members trained as actors for communication simulation. Prior to class, students were provided the scenario and “cheat sheet” of medical and communication tips (Supplemental Appendix 1). In breakout practice sessions, two students worked with an SP in a simulated video-consult for 5 min, then participated in a 10 min facilitated debriefing including time for self-reflection and feedback from peers, SPs, and the faculty member present. In a second round of simulated practice, a second set of students worked with an SP on a voice call (without video). All groups then returned to the main room for a large group discussion and facilitated reflection (Supplemental Appendix 2).

The surgical residency in-person “Uncertainty” module was implemented as part of an ongoing communication curriculum, the structure of which has been fully described previously [11, 12]. Updates in educational content for the resident module included: residents were not provided with a “cheat sheet” prior to class; practice scenarios were conducted by one resident at a time; all practice scenarios were conducted as an in-person conversation; time was increased to 2 h to allow increased time for feedback of student performance and reflective discussion. All other educational elements remained the same as in the medical student pilot, including the didactic portion with demonstration, breakout rooms with actors and facilitators, and a final debriefing session.

Learner confidence and curriculum feedback

Learners rated their confidence levels on four module-specific skill objectives before and after the session using Likert-type items rated from 1, “no confidence/can’t do it,” to 5, “completely confident/can do it without a problem.” These included skills related to the ADAPT framework, including assessing understanding before sharing new information, directly communicating uncertainty, helping patients develop realistic expectations, and exploring the patient’s story when they are upset or distressed. Upon completion of the module, learners rated the session for its contribution to their skill set, graded the course from A + to F, and provided open-ended feedback. The post-session M2 reflection discussion in 2020 was also captured and transcribed for analysis.

We calculated medians and interquartile ranges (IQRs) for the pre- and post-module skill confidence assessment scores for both the medical students and residents, and further compared these data using Wilcoxon signed rank test. P values of less than 0.05 were considered statistically significant. We report M2 feedback on the curriculum using descriptive statistics and summaries of qualitative open-ended comments. All statistical analyses were conducted in R 3.3.2 [13].

Resident skill assessment

At the in-person residency module, we measured learner communication skill using the 14-item Communication Assessment Tool (CAT), a previously validated instrument developed as a patient assessment of physician interpersonal communication skills [14]. The CAT has been validated for use in simulation [11, 14, 15] and recommended for inclusion in the ACGME toolbox by the ACGME Advisory Committee on Educational Outcome Assessment [16]. Simulation observers and SPs completed the CAT following the student’s interview with the SP.

Residents’ pre-module skill confidence was compared to each CAT item to identify how pre-module confidence influenced specific behaviors during their interview, as rated using the CAT. To identify more granular changes in resident confidence, we compared pre and post-module top box scores (those self-rating as “completely confident” in the skill), using McNemar’s test to calculate P values. Of note, this analysis was inappropriate for the M2 class, as none of the M2 learners were “completely confident” before class. Generalized linear mixed models were used to assess the impact of the individual pre-module confidence on each CAT item, where the outcomes variables were the CAT item assessments (dichotomized as 5 vs 1–4). Pre-module confidence was included as a fix effect (dichotomized as 5 vs 1–4) and random effects were included to account for the intra-subject correlations among the assessments contributed by the same subject. Odds ratios for obtaining a CAT assessment equal to 5 for subjects with a pre-module confidence score equal to 5 compared with subjects with pre-module confidence score ranging from 1 to 4 were reported, along with 95% confidence intervals (CI).

Results

Learner confidence and course grade

In our M2 pilot course, 28 students attended class. As this was the final class of 5 weekly modules, only six students (21%) completed the optional pre- and post-module surveys. While student comfort and engagement appeared to increase during class over time, survey response rate decreased over the summer. Prior to the class, students’ median confidence in communicating uncertainty was 2 (where 1 was “no confidence” and 5 was “complete confidence”) with an IQR of 2–2.75. After class, student median confidence increased to 4 [IQR 3.25–4] (P = 0.053). We also assessed student confidence in other communication skills included in the ADAPT framework, including: assessing patient understanding (pre-module median confidence was 3 [IQR 3–3]; post-module median confidence was 4 [4–4]; P = 0.089), providing realistic expectations (2 [2–2] pre; 3 [3–3.75] post; P = 0.053), and exploring emotions (3 [2.25–3] pre; 3.5 [3–4] post; P = 0.037). All students rated the class an “A” or “A+” and noted that the course exposed them to new skills.

In our surgical residency module, 24 learners attended class and 15 (63%) completed the pre- and post-module confidence surveys. Not surprisingly, resident pre-module confidence was higher than that for M2 learners; resident median confidence in communicating uncertainty increased from 4 [4–4] pre-module to 5 [4–5] after (P = 0.008). Confidence in other communication skills also significantly increased, including assessing understanding (4 [4–4.5] pre to 5 [4.5–5] post; P value 0.018), developing realistic expectations (4 [3–4] to 5[4–5]; P = 0.007), and exploring emotions (4 [4–4] to 5 [4–5]; P = 0.009). Eleven residents (73%) rated the class an A or A+, three (20%) a B or C. Figure 2 displays the combined scores of both groups’ pre- and post-module confidence scores.

Fig. 2.

Combined M2, resident self-reported confidence scores in uncertainty communication skills before and after class

Resident pre-confidence vs. CAT skills assessment (Table 1)

Table 1.

Resident “Uncertainty” module results: Communication Assessment Tool vs. Pre-Class Confidence Scores

CAT Item	Pre-confidence items
	Assess p/f understanding before sharing new information			Directly communicate uncertainty to p/f			Explore p/f story when they are upset or distressed
	OR	95% CI	P	OR	95% CI	P	OR	95% CI	P
Greeted me in a way that made me feel comfortable	3.44	0.11, 103.76	0.477	1.41	0.06, 33.28	0.83	3.44	0.11, 103.7	0.477
Treated me with respect	2.18	0.06, 78.815	0.67	3.14	3.11, 3.17	< 0.001	2.18	0.06, 78.82	0.67
Showed interest in my ideas about my health	2.97	0.11, 77.66	0.513	0.73	0.03,16.59	0.846	2.97	0.11, 77.6	0.513
Understood my main health concerns	1.92	0.04, 83.1	0.734	1.13	0.03,43.81	0.947	0.43	0.01, 19.9	0.669
Paid attention to me (looked at me, listened carefully)	1.63	1.62, 1.65	< 0.001	2.26	0.08,60.48	0.628	1.63	1.62, 1.65	< 0.001
Let me talk without interruptions	NA	NA	NA	NA	NA	NA	0.17	0.00, 21.39	0.475
Gave me as much information as I wanted	NA	NA	NA	3.13	0.05,211.9	0.596	1.13	0.01, 101.6	0.957
Talked in terms I could understand	NA	NA	NA	NA	NA	NA	NA	NA	NA
Checked to be sure I understood everything	3.53	0.24, 51.05	0.355	2.22	0.17,29.55	0.546	0.93	0.05, 17.41	0.96
Encouraged me to ask questions	2.37	0.24, 23.80	0.463	1.5	0.17, 13.41	0.716	0.67	0.05, 98.69	0.756
Involved me in decisions as much as I wanted	0.7	0.038, 12.77	0.806	0.49	0.03, 8.07	0.618	2.68	0.18, 39.78	0.474
Discussed next steps, including any follow-up plans	0.72	0.037, 13.79	0.825	4.51	0.37,54.51	0.237	NA	NA	NA
Showed care and concern	NA	NA	NA	NA	NA	NA	1.1	0.03, 41.02	0.957
Spent the right amount of time with me	1.86	0.06, 54.89	0.719	NA	NA	NA	0.45	0.02, 10.87	0.622

Statistically significant P values in bold

Residents self-identifying as “completely confident” at “assessing the patient/family understanding before sharing new information” were 60% more likely to be rated “excellent” at “paying attention (looking, and listening carefully)” by their SP and observers on the CAT (CI 1.62, 1.65; P < 0.001). Those “completely confident” at “exploring the patient/family story when they are upset or distressed” were also 60% more likely to be rated “excellent” at paying attention (CI 1.62, 1.65; P < 0.001). Those “completely confident” at “directly communicating uncertainty” were three times as likely to be rated “excellent” at “treating me with respect” by their SP and observers (CI 3.11, 3.17; P < 0.001). None of the residents self-rated as “completely confident” regarding their ability to “Help the patient/family develop realistic expectations” before the class.

Student experience and lessons learned

Open-ended feedback from the post-session M2 debrief and survey provided further insight on participant experiences (additional illustrative quotes in Table 2). Overall, students expressed a positive experience:

This session was great… It is important for us to be open and honest with our patients and their families.

Table 2.

Additional medical student and medical faculty uncertainty communications class feedback from post-class discussion and written evaluations collected in July, 2020

Important teaching points emphasized by faculty	Use your open-ended questions to explore what is the most worrisome about the situation
	It is telling people that you do not know either, but that whatever happens, we will have a plan for that contingency
	What patients really notice, even if you are rushed, if you pay attention to them and the words you use when you talk to them… this was aiming to give you skills to make people feel comfortable, even if you are uncomfortable—because you will be uncomfortable for the next several years!
	I think people forgive you for uncertainty. What people do not forgive you for is poor communication
	I have to assess surgeons on 19 competencies, and of those 19, 1 involves technical skills… several of them involve communications… These skills are so critical and vital and you cannot start learning them early enough in your training
Student experiences	I really like the interviews—that definitely should stay
	I… like when we had a list right before we went into interviews with…the points we should hit
	I loved having the cheat sheets… and scenarios…, but I think it would enhance our “to go” skills if there were surprise cases
	I would not have thought to bring up patient privacy rights but it’s a really important topic to discuss when it comes to disclosure to family

Students noted that having access to the ADAPT framework was helpful in their practice:

[The framework] helped me to come up with a strategy for different situations like uncertainty.

Students emphasized that seeing a demo scenario prior to practicing with SPs was particularly helpful:

I really liked how… we had example cases… I really liked seeing… here’s what you might be inclined to do and here’s what would be ideal to do.

Students also appreciated having the opportunity to practice both with and without video:

I really appreciated that… we had one practice scenario with video and one without… phone calls will always be a part of our practice.

Our surgery faculty reiterated the importance of communicating uncertainty, and emphasized key teaching points:

I think people forgive you for uncertainty. What people don’t forgive you for is poor communication.

Discussion

Though levels of uncertainty guide virtually every medical decision, medical education has historically emphasized a black-and-white approach: classic clinical vignettes and multiple choice exams lead students to become used to the idea that there is a clear, correct answer for every problem [4]. Such paradigms encouraging the existence of certainty poorly prepare students for the reality of uncertainty they face upon engaging in patient care; this leads to student feelings of inadequacy and anxiety, and to a constant quest for unattainable perfection, which in turn leads to burnout [17].

We present a conceptual framework and class design to allow easier implementation of important uncertainty curricula in medical training programs. First, we recommend a class that is at least 1.5 h in length to ensure enough time for feedback and discussion. Second, we recommend opening the class with a demonstration of appropriate uncertainty communication from a faculty member; our students were overwhelmingly appreciative of seeing an example on which to guide their practice. We further recommend including a list of discussion steps, such as the ADAPT framework. The ADAPT framework is a literature-based mnemonic summary of the important steps in an uncertainty discussion and was helpful to our students in digesting the lesson and in retaining and implementing key points during their practice. Observed practice with SPs in small groups, followed by immediate faculty and SP feedback, is also widely accepted as an important communication course tool, allowing solidification of theoretical concepts [18]. While in-person simulation is a preferred learning environment for skills practice, communication skills can be effectively taught virtually when breakout rooms are used for small-group practice and a facilitator is assigned to each room.

Importantly, our results suggest a significant link between learner confidence prior to skills practice and observer/SP-assessment of their skills demonstration, validating the importance of assessing and addressing skills confidence. Overall, learner confidence in communicating uncertainty significantly increased after learning and practicing the ADAPT framework for both M2 and resident groups. Confidence in all other communication areas assessed also significantly increased. In addition, our results highlight the significant increase in students who reported feeling “completely confident” in the emphasized communication skills from pre- to post-class. As the ultimate goal of any curriculum is for students to leave feeling “completely confident” in implementing their newly learned skill, we believe this result further demonstrates the efficacy of our curriculum.

Our team presented an uncertainty communication module centered around the ADAPT framework to a second year of medical students; analysis of student confidence and feedback data is ongoing and will be presented in a future manuscript. We plan to incorporate ADAPT modules with tailored role play scenarios into continuing education of medical students, residents, attendings, and other healthcare workforce members; we anticipate this framework will similarly improve uncertainty communication in these groups, given that comparable curricular designs have increased communication knowledge and confidence in residents, nurses, and attending physicians alike [19, 20]. We believe ADAPT will prove to be a useful tool for educators as they present the myriad important uncertainty-related communication topics, including prognosis and end-of-life issues.

Overall, the course design presented requires limited time, materials, and faculty, and provides a feasible path to implement important uncertainty communication curricula in a virtual or in-person setting.

Supplementary Information

Below is the link to the electronic supplementary material.

Funding

None.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

Declarations

Conflict of interest

The author declares that they have no conflict of interest.

Ethical approval

The Inova Institutional Review Board reviewed the study (IRB #U20-04-3993) and deemed the project exempt on July 9, 2020.

Data

No data were accessed from outside sources.