Teaching residents how to break bad news: piloting a resident-led curriculum and feedback task force as a proof-of-concept study

Abstract

Background

Breaking bad news (BBN) is a critically important skill set for residents. Limited formal supervision and unpredictable timing of bad news delivery serve as barriers to the exchange of meaningful feedback.

Purpose of study

The goal of this educational innovation was to improve internal medicine residents’ communication skills during challenging BBN encounters. A formal BBN training programme and innovative on-demand task force were part of this two-phase project.

Study design

Internal medicine residents at a large academic medical centre participated in an interactive workshop focused on BBN. Workshop survey results served as a needs assessment for the development of a novel resident-led BBN task force. The task force was created to provide observations at the bedside and feedback after BBN encounters. Training of task force members incorporated video triggers and a feedback checklist. Inter-rater reliability was analysed prior to field testing, which provided data on real-world implementation challenges.

Results

148 residents were trained during the 2-hour communications skills workshop. Based on survey results, 73% (108 of 148) of the residents indicated enhanced confidence in BBN after participation. Field testing of the task force on a hospital ward revealed potential workflow barriers for residents requesting observations and prompted troubleshooting. Solutions were implemented based on field testing results.

Conclusions

A trainee-led BBN task force and communication skills workshop is offered as an innovative model for improving residents’ interpersonal and communication skills in BBN. We believe the model is both sustainable and reproducible. Lessons learnt are offered to aid in implementation in other settings.

Purpose of the proof-of-concept study

Effective communication is a core skill in clinical practice and postgraduate training.¹ Gaps in physicians’ communication skills are often exposed during difficult conversations.² These gaps can impact patient outcomes, with poor performance leading to patient mistrust and decreased physician satisfaction.^{3 4} The Accreditation Council for Graduate Medical Education (ACGME) underscored the importance of interpersonal and communication skills (ICS) when it included ICS among the six core competencies in 1999.^{5 6} Communication skills training for residents has been associated with decreased patient anxiety and depression⁷ and increased hope⁸ when delivering bad news. A systematic review showed that rigorous communication skills training can improve trainee-reported confidence in delivering bad news as well as observer-rated skills assessment.⁹

Direct observation of patient–physician communication, considered necessary within competency-based educational frameworks,¹⁰ remains underdeveloped during residency training¹¹ and is often difficult to implement.¹² Specific challenges include limited staff physician time, inflexible schedules, unpredictable opportunities to deliver bad news and variable abilities of staff physicians to provide meaningful feedback.^13–19 To address educational barriers and increase opportunities for residents to request direct observation, we created a two-phase programme for teaching the skills involved in breaking bad news (BBN): an interactive workshop on BBN and a resident-led ‘task force’. The goals of this innovation were to provide training on key communication skills and to create a group of dedicated clinician-educators (ie, task force) trained to observe BBN encounters and provide feedback on-demand. In this paper, we describe our educational innovation, including the BBN task force, a proof-of-concept project.

Conceptual framework

Adult learning theory provided a conceptual framework to guide development of the on-demand BBN task force. According to adult learning theory,²⁰ adult learners are thought to be self-directed and intrinsically motivated to improve performance when tasks are authentic and reflect real-world demands. The focus on formative assessment and a collegial relationship with task force observers is also rooted in competency-based education.¹¹

Methods

Timeline and participants

This prospective project was implemented from 2019 to 2020 at Cleveland Clinic in Cleveland, Ohio. Internal medicine (IM) residents (postgraduate year, or PGY, 1–3, N=148) participated in the BBN workshop. The initial task force consisted of nine volunteers including senior residents, palliative medicine fellows and palliative medicine faculty members. Any IM resident on an inpatient primary service could request a bedside observation.

Context

All IM residents have clinical experiences with death and dying due to the nature of clinical care (ie, usually high-acuity patients). In addition, IM residents at our institution have two additional teaching experiences related to this topic: Foundations of Resident Assessment, Mentorship and Emotional Intelligence (FRAME), which are monthly sessions dedicated to engaging residents in ethical topics surrounding the life of a doctor.²¹ Death and dying is part of the FRAME curriculum, and DH codirects this session. Second, during the first-year retreat, residents participate in a session dedicated to this topic as well. They discuss their experiences with a psychologist. While all residents engaged in both teaching sessions described above prior to the project’s start, no resident had standardised patient experiences focusing on communication skills during residency prior to this project.

Measurement instruments

Pre-workshop and post-workshop surveys were created with input from CYC, a social scientist with survey design experience. The pre-workshop survey measured demographics and residents’ self-assessed skills in BBN. The post-workshop survey included the same questions along with 10 items assessing the workshop (online supplemental file 1).

Description of interventions

Communication skills workshop

This 2-hour, mandatory workshop introduced basic communication skills with a focus on using a ‘warning shot’ and delivering bad news in a single sentence followed by silence and empathy. The majority of the time was spent in small group skills practice. DH taught and facilitated each of these sessions using a lesson plan developed by our team (online supplemental file 2), which integrated best practices in bad news delivery and physician communication skills.²² Residents’ self-reported skills in BBN were surveyed pre-workshop and post-workshop.

BBN task force formation and member training

The BBN task force was created in the fall of 2019 with support from the ACGME’s Back to Bedside grant.²³ Nine volunteers formed the initial task force. Task force members participated in three, 2-hour training sessions on BBN to patients and how to provide meaningful feedback. Volunteers were taught best practices from adult learning theory and principles from small group facilitation, which were applied to feedback on direct observations. During the final session, task force members participated in frame-of-reference training using video triggers featuring bad news delivery. For each video trigger, participants used a modified version of a previously developed checklist²⁴ to rate the quality of the conversation (online supplemental file 3). Between video triggers, group discussions were used to reconcile differences in ratings among participants.

Bedside observations

The following protocol was developed for bedside observations:

• One task force member is ‘on-call’ to provide bedside observations during regular business hours.

• Residents are encouraged to schedule an observation with the ‘on-call’ observer before an expected BBN encounter. Of note, during the education of residents on the process of scheduling an observation, some reluctance was expressed in terms of being observed by a coresident, but not by a palliative medicine fellow or an attending. Residents were allowed to decline observations if they were not willing to be observed by a known observer, especially if it was a resident.

• The task force member first discusses the case with the resident. Due to the on-demand style of requesting BBN observations, there is a chance of familiarity between the observer and the resident during patient encounters. However, the pre-planning of the observation includes the observer informing the participant about what to expect in terms of observation, that the observer’s purpose is to provide positive and constructive feedback, and that the observation will not be shared with the IM programme for any competency assessment, in accordance with medical education guidelines.¹²

• Prior to initiating the observation, patients are given an institutional review board (IRB)-approved information sheet by the observer (not the resident) to inform them that the observer’s purpose is to assess the resident’s communication skills with patients. The observer explains that they are not part of the patient care team and will not personally participate in the conversation. Patients are given freedom to decline. Patients were specifically not made aware that this is a BBN type of conversation, as that would impact the introductory piece of BBN for the resident; this action was IRB-approved. The observer then observes the encounter and provides verbal feedback to the resident.

• Formative assessments are performed via a modified checklist²⁴ and open-ended questions (online supplemental file 3).

Field testing

A quality assurance assessment of task force initiation (ie, field testing) for a bedside observation was carried out in February 2020, prior to COVID-19 social distancing. Field testing allows for the assessment of innovations, protocols, processes and surveys under realistic conditions,^{25 26} with adjustments made prior to formal implementation or launch. Field testing is critical in assessing innovations prior to a significant investment of time or expense.

Planned analysis

Descriptive statistics were run on demographic data. Survey results were analysed via a two-sided paired t-test. The significance level was set at 0.05. For task force training, inter-rater reliability was calculated for the first and second video using average pairwise percentage agreement and Fleiss’ kappa for nominal categories. An intraclass correlation coefficient (ICC) was calculated to assess agreement among ratings for quantitative data.

Results

Communication skills workshop

A total of 148 PGY1–3 residents participated in the communication skills workshop. Sixty-five (44%) residents were female and 108 (72.9%) completed the feedback surveys. Prior to the workshop, 21 of 108 (19.4%) residents endorsed being either ‘not skilled at all’ or marginally skilled when asked ‘How skilled are you in communicating bad news to a family member or patient about their illness?’ Post-workshop, only 5 of 108 (4.6 %) residents endorsed being marginally skilled in bad news delivery, and the remainder of the residents said they were ‘somewhat skilled’ or better. The mean score improved from 3.06 (2.25–3.88) to 3.44 (2.79–4.08) (p=0.0004). When asked ‘How helpful was this training to your overall communications skills development as a resident?’, 50.9% of the residents answered ‘very helpful’ and the average response was 4.35 (3.55–5). When asked ‘How likely is it that this training will affect your communication of bad news to patients?’, 63 of 108 (59.4%) residents endorsed ‘very likely’, with an average response of 4.43 (3.63–5) (figure 1).

Figure 1.

Pre-workshop and post-workshop survey results from the interactive BBN skills session that was attended by all categorical internal medicine residents (n=108). BBN, breaking bad news.PGY: postgraduate year

In narrative comments, residents were enthusiastic about the classroom-based simulations (Box 1). Participants reported that the training they received was applicable to their current practice of medicine and would improve future patient encounters. They also expressed willingness to undergo a refresher training in future years.

Box 1. Narrative comments from residents while assessing the BBN inclass curriculum (question 10 on the questionnaire).

• “I thought this was an excellent session. Despite being a PGY3 that has had many difficult conversations and broken lots of bad news, I still felt that my skills were improved by this course.”

• “This was an excellent and helpful course. It will change my practice for delivering bad news. Thank you!”

• “I wish I had it last year.”

• “I think a video simulation before the last exercise might have helped put everyone in the zone for acting out and getting the best experience.”

• “Really appreciate the tips and tricks, different phrasing, and practice.”

• “This framework for delivering bad news is easy to remember and helpful. Essential to every internist. Would definitely recommend it.”

• “I think there should be more refresher courses throughout the same PGY year because this topic is pretty relevant to our training.”

• “Great session. Extremely important for patient care. Hope this can lead to residents getting more chances to lead family meetings and break bad news on inpatient services.”

• “If it was not mandatory, I would likely not attend, because there is little free time, so I think we should make it mandatory.”

BBN task force recruitment and training

Seven of nine volunteers completed the 6-hour task force training and expressed familiarity and confidence in using the bedside scoring instrument. For nominal data, pairwise percentage agreement revealed that participants were not consistent in their ratings for some questions (eg, communication questions) while very consistent in others (eg, ‘Discusses plan only after patient asks to or gives permission to discuss next steps’). This was also reflected by a low Fleiss’ kappa, which indicated poor inter-rater reliability across raters on both videos (table 1). ICC, calculated to assess reliability among ratings (table 2), showed that there was poor correlation with video trigger 1 (ICC=0.12), but moderate or good correlation with video trigger 2 (ICC=0.65).

Table 1.

Comparison of rater scores on yes/no question items after observing standardised BBN videos during the task force training

Variable	Video 1		Video 2
	% agree	Kappa	% agree	Kappa
BBN
Q1	71		100
Q2	52		43
Q3	100		43
Q4	100		43
Q5	52		43
Q6	71		100
Q7	71		43
Q8	43		71
Q9	100		100
Overall	74	0.46	65	0.14
Communication
Q1	52		100
Q2	43		71
Q3	43		71
Q4	71		100
Q5	52		71
Overall	52	0.03	83	−0.09

BBN, breaking bad news.

Table 2.

Comparison of rater scores on Likert-scaled question items after observing standardised BBN videos during the task force training

	Video 1			Video 2
	Mean	SD	Range	Mean	SD	Range
Performance
Q1	0.5	0.5	0–1	0.6	0.5	0–1
Q2	1.0	1.0	0–3	0.8	0.6	0–2
Q3	0.9	0.7	0–2	0.6	0.5	0–1
Q4	1.4	1.4	0–4	0.6	0.5	0–1
Q5	1.0	0.9	0–3	1.2	0.9	0–3
Q6	1.4	1.4	0–4	0.6	0.6	0–2
Overall	1.0	1.1	0–4	0.7	0.7	0–3
ICC		0.12			0.65

BBN, breaking bad news; ICC, intraclass correlation coefficient.

Field testing

Due to COVID-19 social distancing restrictions, one bedside observation was conducted during field testing of the task force concept. The observed resident indicated that it was very easy to request a task force observation, that he/she was likely to request an observation in the future and that the task force was easy to contact. The resident was ‘somewhat comfortable’ with having an observer in the room and rated the likelihood of observation improving future BBN practice as 4 out of 5 on a Likert scale.

In discussions with residents about the task force concept at the end of the communication skills workshop and during subsequent announcements for the project, we did not encounter reluctance about being observed by palliative medicine fellows or attendings. Despite making residents aware that peer observers are trained in proper observation and feedback techniques, faculty/fellow task force observations were preferred over peers due to perceptions that feedback would be of higher quality (ie, objective and constructive). While our field testing did not provide enough data to generalise about patient comfort with the task force model, the patient involved in the first field test was comfortable with the observer in the room, as they were used to multiple providers during regular patient doctor encounters.

Subsequent task force requests

Task force paging and face-to-face observations were paused due to social distancing requirements related to the COVID-19 pandemic.

Discussion

The aim of this innovation project was to facilitate the transfer of communication skills from the classroom to real patient encounters. We specifically focused on BBN, which physicians consistently rate as an extremely important but infrequently taught skill.²⁷ This resident-led project included an interactive workshop to teach residents fundamental BBN skills and the creation of an innovative task force for direct observation of real patient encounters at the bedside. Our overall goal was for the task force to be a trainee-led, sustainable model for teaching effective BBN skills. By integrating task force observations into the clinical workflow of trainees, we hoped to promote sustainability.

The communication skills workshop proved successful as a primer for residents in BBN, but the bigger impact was increasing resident confidence in requesting observations for real patient conversations, as well as building rapport with their future observers. Similarly, the curriculum creates an opportunity to publicise the task force’s on-demand function and to get a sense of residents’ willingness to participate. Among narrative comments received from residents, a divide was sensed between wanting to participate voluntarily and feeling the observations should be mandatory (Box 1). This helped us strategise several approaches to increase residents’ willingness to participate (see the Generalisability section).

Although resource-intensive at 6 hours of total training time, the formal task force training programme was crucial to ensuring that residents were offered consistent, high-quality feedback during bedside observations. While the use of video triggers during frame-of-reference training was effective in improving inter-rater reliability, inter-rater reliability remained inconsistent during some elements of the video assessment. This parallels some studies that have explored how raters’ differing assessments of observed performances contribute to measurement error;²⁸ other studies have shown that rater training does not always improve inter-rater reliability or accuracy of assessment.^{29 30} Given that it is normal for individuals to employ different communication approaches—and that the purpose of task force observations was formative, low-stakes assessment—consistency across raters did not carry the same weight as it would have for summative, high-stakes assessments. That said, we provided open response formats to gather raters’ comments and better understand the reasoning processes which went into rater scoring. Discussions during rater training regarding verbal and non-verbal feedback to trainees will help us to improve future iterations of task force training.

Our approach to task force training is in line with recommendations from an international group of researchers.³⁰ Although they offer three differing perspectives on rater cognition to better understand variability in assessment (eg, ‘assessor as trainable’, ‘assessor as fallible’ and ‘assessor as meaningful idiosyncratic’), they all agree that faculty development is needed to enhance the observation and assessment skills of raters.³⁰ In addition, they urge immediate attention to the deficiency in real-world practice of observation-based assessments of undergraduate and postgraduate medical trainees, which currently limits further understanding within this area of research.³⁰

There have been many prior educational interventions to improve BBN ability using didactic courses,³¹ simulation,^{24 32} group discussions,³³ online assessment³⁴ and instructional videos.³⁵ However, it was our use of a rigorously trained task force of clinicians available for performing observations of real patient encounters and then providing feedback that truly made this intervention novel and potentially generalisable to other institutions and training programmes. This field test experience and the lessons learnt from it will enable our residency programme and others to offer meaningful communication skills observations once the present COVID-19 pandemic subsides.

Limitations

This educational innovation was carried out at a single institution where we had strong buy-in from the residency programme’s leadership. The results may not be generalisable to other contexts. In addition, it is possible that residents may have had prior experiences with standardised patients related to BBN training during medical school or on the clinical wards; this may have impacted survey responses during communication skills training. In addition, due to social distancing requirements during the COVID-19 pandemic, we were unable to study the effects of task force implementation longitudinally within our residency programme.

Lessons learnt

Residents’ willingness to request observations was the biggest barrier to maximising the benefit of this project and studying its impact. Residents expressed reluctance about requesting BBN observations due to perceived stress while being observed and the disruption to their daily workflow. During field testing, we identified additional challenges to full task force implementation. Based on resident feedback after the field test, we adjusted aspects of the task force model to mitigate barriers to implementation (table 3).

Table 3.

Summary of proposed project adjustments that were adopted after field testing in preparation for task force launching

Project obstacle	Adopted solution
Initial difficulty with dissemination of project purpose and protocol for requesting the task force.	Advertise the task force during academic noon conferences and town hall. Creation of e-posters that simplify the request process into three-step model.
Residents’ reluctance to be assessed and evaluated by a co-resident.	Allowed option to be observed only by fellow and/or palliative attending. Emphasising anonymity of the observation and feedback that would not be part of the programme evaluation of the resident unless they request a mini-CEX evaluation.
Sustaining number of weekly requests.	Creation of monthly ‘BBN resident of the month’ during town hall meeting to encourage others to participate. Small monitory incentives allocated from the ACGME fund for residents who participate in the observation.
COVID-19 pandemic limited the number of physicians in a room with a patient per hospital protocol.	Investigating virtual platforms that allow for proper observation of BBN by task force members. Limiting the activity to one single unit in the hospital during the pandemic.
Limited availability of a task member when requested.	Organisation of a weekly ‘call schedule’ based on observers’ academic rotation. Increasing the number of trained task members if needed.
Reluctance by inpatient rotation attending to allow residents to perform such requests.	Weekly emails to the attendings on services with residents to explain how this task force does not interfere with patient care. Open forum during quaternary faculty meetings to answer concerns that may arise.
Reluctance of the patient to participate in observation.	IRB-approved script given to patient to explain the role of the observer. Confirmation by the caring resident that the observer has no role in the patient’s care and is only available to give feedback to the resident’s communication skills.

ACGME, Accreditation Council for Graduate Medical Education; BBN, breaking bad news; IRB, institutional review board; mini-CEX, mini-Clinical Evaluation Exercise.

• Pre-implementation challenges: During the implementation process, we learnt about the need to convey our mission to programme leadership in order to garner support. We also learnt that residents truly need to appreciate the benefit of this project in order to engage, given their prioritisation of other medical opportunities that may relate better to their career interests. Communicating with resident leaders and coordinating with inpatient service attendings proved key to approaching residents and gaining maximal participation.

• Dissemination: We presented at the monthly programme town hall meetings to inform them about the ease of requesting an observation and the low-stakes nature of the observation process. Our residency programme was very supportive of this project, which helped with launching the interactive classroom curriculum and advertising the project.

• Resident encouragement: At monthly programme town hall sessions, we advertised the BBN initiative and shared positive testimonials from residents by recognising a ‘Participant of the Month’. All participants received a gift card for their first observation and were eligible for educational credit via a mini-CEX (mini-Clinical Evaluation Exercise) for ‘Conducting a Family Meeting’, which became part of their training portfolios.

• Workflow integration: We simplified the process for requesting observations. Prior to halting observations for COVID-19, we contacted staff physicians for teaching teams weekly to encourage them to request observations.

Sustainability and generalisability

Sustainability

This train-the-trainer model will create a cycle of trainee-led sustainability and continued buy-in from residency programme leadership and chief residents. Specifically, it will require yearly handoffs to more junior trainees to ensure sustainability over time. We recommend designating a faculty member to assist in the recruitment and training of new task force members. Ultimately, we believe this programme will become a sustainable element of our IM residency programme.

Generalisability

We believe that this model is applicable to other crucial communication skills and training programmes. A surgical residency, for example, could use task force observations to improve resident skills in taking surgical consent or describing an intraoperative surgical complication to a family. Our intervention was implemented at one of the largest residency programmes in the USA, and the model may be easier to implement in a programme with fewer trainees. When sharing this model with other programmes, it is important to note the narrative comments received from residents who said they felt it is better to make the curriculum and observations mandatory (Box 1). From what we experienced, residents willing to participate in either task force volunteering or on-demand observations appeared to be internally motivated to learn more about the BBN topic, or it matched their future career focus. For residents who are less willing to participate, options to enhance buy-in include making the observations a mandatory minimal requirement for a graduation milestone related to the ACGME’s ‘Interpersonal and Communication Skills’ competency, or creating a give-and-take system whereby there is a reward for participation (curriculum vitae title, mini-CEX accomplishment, ‘Participant of the Month’ award or monetary incentive).

Conclusions

Our resident-led BBN curriculum enhanced opportunities for our trainees to learn critical communication skills and to receive actionable feedback from real patient encounters. Development of the BBN task force allowed our programme to provide advanced training to task force members, preparing them to observe difficult encounters, and provide feedback in a standardised way. The results of this proof-of-concept project indicate that residents viewed the classroom curriculum favourably and that our task force members could be standardised in their approach to observation. This model, while narrowly focusing on BBN skills building with IM residents, could easily be expanded to other bedside communication skills and/or to other medical or surgical specialties.

What is already known on this subject.

• Breaking bad news (BBN) skills are critical to professional development and patient care and should be formally taught to residents.

• There are many examples of didactic skills training programmes to teach effective BBN skills, but few are led by trainees and none offers direct bedside observation and feedback of real patient encounters.

What this study adds.

• This study reports a trainee-led initiative to teach BBN skills to residents using an interactive skills workshop and direct bedside observations conducted by a task force.

• We report a proof of concept for providing standardised direct bedside observation with feedback using an innovative trainee-led task force.

Data availability statement

All data relevant to the study are included in the article or uploaded as supplementary information. A summary of all data for this project was included with our submission. We are happy to share the raw data for the didactic courses with any interested party.

Ethics approval

This project was approved by the Cleveland Clinic Institutional Review Board after expedited review.