Serious Illness Communication Tool for Palliative Care Fellows: Development, Implementation, and Lessons Learned

Abstract

Introduction: Learning expert communication skills is a core educational goal within palliative care training, yet there are few communication-based educational tools specifically designed for specialty-trained palliative care providers. Objective: To develop and implement a tool to facilitate effective learning of serious illness communication for hospice and palliative medicine fellows. Methods: A novel formative assessment tool was developed within the UCLA Palliative Care Fellowship program, and utilized throughout the academic year on a weekly basis. Focus groups were held for fellows and faculty separately at the end of the academic year in order to gain insight into the experience and effectiveness of the tool. Focus group transcripts were analyzed through thematic analysis. Results: There was a 47% participation rate in the focus groups (n = 7). Qualitative analysis demonstrated positive impact of the tool in identifying learning goals, improving quality of feedback, and in standardizing language around advanced communication skills, with most value in the first half of the fellowship year. Some aspects of the tool were found to increase feedback anxiety, including the competency scoring component, frequency of use, and utilization with individual patient encounters. Conclusion: This novel communication tool provides an important addition to serious illness communication training of specialist palliative care providers, in creating a shared mental model for naming and organizing skills, as well as generating specific high quality learning goals and feedback. Preliminary feedback from our pilot implementation phase provided important information about how to refine the tool and remove components that unnecessarily added to feedback anxiety.

Introduction

Communication plays an important role in the care of patients with life-limiting illnesses and is an integral part of hospice and palliative medicine (HPM) fellowship training. These skills can be framed as a procedure and learned through direct observation techniques with specific, actionable feedback. ¹ To teach communication skills in this procedural manner, the skills and their domains must be defined and standardized, and a common structure and progression through training must exist for delivering formative feedback that allows for consistency between teaching faculty. There are multiple serious illness conversation guides in the literature outlining structured approaches with frameworks and suggested language, typically geared toward improving primary palliative care skills throughout all specialties.^2–9 There are few resources, with the exception of the Communication Skills Evaluation or the SECURE Framework,^10,11 that aim to encompass the nuanced and complex communication skills required to provide specialty or expert palliative care.

In the authors’ experience with HPM trainees, the variety of existing resources on communication and mnemonics is not cohesive and can lead to difficulty in compiling the tools into a functional resource and shared mental model. Thus, the aim of this effort was to develop a practical standardized tool to facilitate effective learning of communication skills for the palliative care specialist in training. Drawing on the educational strategies outlined by Ambrose et al, ¹² the goal was to create a tool that fulfills multiple roles in the learning process: organization of knowledge, self-assessment of weaker skills, identification of learning goals, and the delivery of formative, targeted feedback. In this article, the development and implementation of this novel Advanced Palliative Communication Checklist (APCC) tool are described. The impact of the APCC and lessons learned are explored through focus groups with both faculty and fellows.

Methods

Tool Development

The APCC was developed as a formative assessment tool using published communication tools as guides.^2–9 A formative assessment tool is generally a low-stakes intervention intended to stimulate learning and identify strengths as well as areas that may need improvement. In contrast, a summative assessment is typically higher stakes, and is often used more formally as part of the grading process. ¹³ The APCC was created to reflect the expected skill set of a graduating HPM fellow, which incorporated both previously established guidelines as well as novel items unique to the APCC. Skills were listed in three domains: Framework, Feelings (Building Rapport and Responding to Emotion), and Facilitation. The Framework domain includes an awareness of and the ability to implement the component structure of a comprehensive serious illness discussion. The Feelings domain outlines approaches to respond to emotion and connect with the patient or surrogate verbally and non-verbally. Lastly, the Facilitation domain presents skillful ways to navigate challenging conversations as well as how to manage the multiple voices involved in a discussion. Fellows are generally expected to focus on the Framework and Feelings domains in the first half of the 1-year fellowship, and Facilitation skills in the latter half, once they already have a solid foundation in conversation frameworks and responding to emotion. Each skill was to be rated on a Likert Scale or as ‘Not Applicable’ by the faculty educator. To establish content validity, the checklist went through a series of reviews and edits by interdisciplinary palliative care specialists, including physicians, nurses, social workers, and chaplains, until the final product was agreed upon by consensus. This study underwent IRB review and was certified exempt (IRB #21-001008).

Implementation

In following the published guidance on components of a successful formative assessment tool, ¹³ our goal was to embed the APCC longitudinally in the preexisting direct observation and feedback model integrated into our normal clinical and educational workflow. The APCC was first introduced during fellowship orientation lectures on serious illness communication and reviewed as part of a communication skills workshop with standardized patients. The fellows were asked to reference the APCC in identifying skills they want to focus on for that encounter, and the group would then provide targeted feedback.

The APCC was introduced to faculty with the expectation that this checklist would be a formative assessment tool reviewed and completed weekly with each fellow on the inpatient palliative care service. Faculty and fellows were encouraged to reference the APCC in the beginning of the week to identify learning goals or skills to focus on, and at the end of the week to provide formative feedback on interactions and encounters. The APCC was also used immediately following single encounters to provide feedback. There was intentional flexibility for how the APCC could be utilized, as the goal was to gather a diverse array of experiences in using the tool. As this was a formative assessment, the checklists were handed in to the program leadership solely to monitor completion and participation.

The implementation of the APCC was modified mid-way throughout the academic year, as a result of interim feedback from the fellows and attendings. After several months, fellows highlighted the stress they felt when the APCC was brought up immediately prior to an observed patient encounter. Faculty shared that formal feedback was challenging when this tool was used for specific encounters, largely due to time constraints. In an effort to reduce stress associated with this tool and to ensure continued use, faculty were asked to stop utilizing the APCC prior to specific patient encounters. Utilizing this tool in both ways is reflected in the qualitative results.

Focus Groups

At the end of the academic year, we evaluated the experience of utilizing the checklist from the faculty and fellow perspectives. Invitations to participate in focus groups were sent, on a voluntary basis, to both fellows and faculty. Focus groups were led by non-physician faculty who were not involved in the creation of the APCC and the questions utilized were based on qualitative studies evaluating direct observation tools. ¹⁴ Focus groups had a 47% participation rate with 3 of 7 fellows and 4 of 8 inpatient faculty. Focus groups were conducted separately for fellows and faculty, with the transcripts separately collated.

The results of the focus groups were transcribed via zoom recording software and participant names were de-identified. Errors in the transcripts were independently corrected by each study lead, compared, and reviewed with focus group facilitators until a consensus was reached. Thematic analysis combining a deductive and inductive qualitative approach was utilized in coding final transcripts.^15,16 Once a consensus was reached with initial coding, themes were generated. Multiple iterations were reviewed until themes were agreed upon by study leads.

Results

Focus group results were stratified into three categories: (1) perceived purpose of the APCC and utilization, (2) experience of utilizing the APCC and (3) future utilization. Figure 1 summarizes the themes for the APCC purpose and utilization. Figure 2 summarizes themes unique to the fellow and faculty experience of the APCC, shared experiences between both groups, and suggestions for future utilization.

Figure 1.

Themes related to purpose and implementation of the APCC, generated from faculty and fellow focus groups.

Figure 2.

Themes related to experience and future utilization of the APCC, generated from faculty and fellow focus groups.

Purpose & Utilization

The focus groups highlighted core differences in the perceived purpose of the tool. While one fellow viewed the APCC as a mental tracker for naming skills and identifying learning goals throughout the year, the majority of fellows interpreted the APCC as a tool for standardizing the language used in feedback:

‘It adds a systemic way of approaching feedback in teaching, so that it could be a bit more standardized between different teachers and also different learners.’

- Fellow 3

‘It was good for getting specific feedback on what went well and what we could work on for goals of care conversations.’

- Fellow 2

The faculty described using the APCC primarily as a tool for feedback, as a way to set learning goals for the week, and as a way to present serious illness communication as a set of skills. Utilizing the APCC as a tool for feedback was also mentioned:

‘We would look at the fellow checklist and they would let me know what they were working on last week, or what they were struggling with, and we would utilize that to set the goal for the week.’

-Faculty 1

‘Earlier in the year I use it as a way of saying – this is a structure of how we think about skills and how we organize it in our head into broad categories’.

-Faculty 3

Faculty and fellows described variability in implementation of the tool that was dependent on the approach of the attending, patient volume, or other time constraints. At times the APCC was used at the beginning of the week for goal setting and at other times it was utilized after patient encounters or as a way to reflect cumulatively at the end of the week. Faculty and fellows agreed that use of the APCC was usually initiated by the attending.

Fellow’s Experience

Receiving Feedback

The experience of using this learning tool varied between the fellows and the faculty. Communication skills training in HPM fellowship inherently includes an increase in direct observation and feedback. The fellows noted that it could easily feel like ‘information overload’ when the APCC was used frequently.

‘It’s not necessarily the tool itself, it’s just the feedback overarching process.’

- Fellow 1

‘The nature of palliative care fellowship is just such a huge shift from most of everything we have done so far in training, that it just feels like everything is hyper scrutinized, which you know is for our benefit and it is how the fellowship training is structured, and then you just sort of get acclimated to that process.’

-Fellow 3

Fellows shared that the tool could feel like an obligation and could, at times, feel repetitive each week. They further noted that how they experienced feedback associated with this tool was dependent on their prior experiences with feedback and whether they received it as punitive vs constructive.

‘It seemed very fellow dependent and based on what our backgrounds are as far as feedback goes…how we have had feedback in the past and whether or not it was used as a tool to help or whether it was a punitive measure. So it’s not innately this tool, it’s just learned responses from prior feedback experiences that people are adjusting to and dealing with.’

-Fellow 1

As noted previously, the fellows shared some increased anxiety associated with the APCC, particularly when it was used for feedback in an individual patient encounter.

‘I get more anxious during the encounter if I know that we’re going to be discussing the checklist relative to that encounter…just looking at it my heart rate goes up, I think just with the knowledge that they’re paying hyper attention to everything that I say. Even though that happens anyways, it’s just more at the forefront of my mind.’

-Fellow 1

‘Once we started going through it, it was fine, but the idea of “okay, now we’re going to review it,” just always made me feel a little bit anxious. The idea of being evaluated, even though you know its neutral and helpful, can be anxiety provoking.’

-Fellow 2

Successes

Fellows noted that the tool was an effective way to reflect on the week and improved the quality of feedback while providing a concrete set of skills to work on. The consensus was that the APCC was more useful at the beginning of the year.

‘It did give me concrete things to work on and what things might be important, and some sort of framework…so I think mid-beginning it was probably a little more helpful and not as much later on.’

-Fellow 1

‘It was helpful, specially at the beginning of the year, for getting very specific feedback on what went wrong, what could go wrong for goals of care conversations.’

-Fellow 2

Fellows also shared that the tool was particularly helpful once they had more time to build a relationship of trust with an attending (often towards the end of the week) and were able to extend their established rapport into a more nuanced conversation about their communication skills while referencing the APCC.

‘It feels less under a microscope if it’s over an entire week period.’

-Fellow 1

‘It did help in my experience when it was at the end of the week and you’d had five days to get to know that person.’

-Fellow 2

‘The sense of trust in them that they want to really better you and none of this is punitive, is very critical in being able to receive feedback and carry it forward with positive takeaways…it may be less than optimal if it’s the very first time you are working with someone.’

-Fellow 3

Faculty Experience

Providing Feedback

Faculty shared that there were challenges associated with finding the right day or encounter to utilize the checklist and at times, the checklist felt burdensome. When the checklist was introduced at the start of a service week, then utilized for specific encounters and reviewed at the end of the week, there was potential for it to feel too frequent and challenging to accomplish.

‘I was definitely excited throughout the year as we were doing it weekly, and then it definitely got a bit more like something I have to do and I started to see less and less benefit for the fellow. Because I think it became too repetitive for both...I think it will be more helpful if less frequent so I can actually see the progress or change…I found it more helpful when I met the fellow again after several months vs. two weeks.’

-Faculty 4

One faculty member reflected that the repetitive nature of this tool, even if perceived as fellows’ being ‘bored’, demonstrated a difference from prior years in that this ‘boredom’ reflected the fellows’ grasp on communication skills as being ingrained and more thoroughly understood. They went on to note that the more familiar this was for fellows and faculty, the faster and easier it was to review each week and incorporate into the standard feedback.

Despite the hope that the APCC would integrate organically into feedback that was already happening as a part of standard fellowship education, most faculty acknowledged that the tool itself did change the tone of the interaction.

‘It doesn’t feel part of the standardized process and it feels like it has to be interjected…it does feel more formal.’

-Faculty 2

‘It isn’t as simple as when we do give feedback already kind of organically, and then have to pull out the goals of care checklist – it is not organic and makes it feel official.’

-Faculty 1

There were learner specific situations where faculty would not have otherwise utilized a formal teaching tool to reference because of the learner’s needs. In those scenarios the checklist could adversely impact the teaching relationship and add an unnecessary layer of stress.

‘For someone who is very much in a remedial position when it comes to communication skills…I’ve had to just kind of put the checklist aside and say, this is not how I can give you the most effective feedback right now.’

-Faculty 3

‘In particular when too much feedback is hindering their ability to progress…it can be too much because it’s very in depth going through all the different skill sets, so the ability to back off and give a person an emotional break to regain their footing is really important.’

-Faculty 2

‘For many people, I worry it creates a sense of anxiety or self judgment… For many of our learners it takes a while to separate their serious illness communication skill set from their sense of identity and self as a person.’

-Faculty 2

Successes

Overall, the faculty felt the APCC had a positive impact. Faculty found it helpful to use the APCC as a tool for establishing goals for the week. They reflected that the tool created structure and a more standardized language surrounding communication skills and feedback that was then transferred between attendings. This checklist provided an organized approach to teaching expert level communication skills with a tangible resource to reference throughout the fellowship year.

‘I feel far more effective in our ability to communicate with a fellow on what they’re working on. The fellow can then share with the next attending an item they’re working on and we’re all using the same language. This checklist helps with that transfer of learning objectives week to week significantly –that’s because fellows are much more able to clearly state a learning objective in their communication skills than prior to the checklist…I have seen fellows in prior years not always be able to describe structured ways that they’ve learned their communication skills or talk about how they would teach them. Now I am seeing that as the fellows are getting exposed to this regularly, it is becoming part of their vocabulary – that they’re able to describe the different pieces of communication that someone can work on, the different skill sets and the knowledge base that’s required to do this.’

-Faculty 2

‘...this has been a really helpful tool, but it’s also one in which, the goals that it was meant to achieve, of getting some standardization in terms of how we talked about this among faculty, to provide some baseline structure to the feedback sessions, is achieving that. But it might also be one of those transient tools that you do a lot more for a couple of years until it’s ingrained in the culture of an institution and the educational culture and then you just don't make it as prominent anymore.’

-Faculty 3

Future Utilization

The focus groups were able to cultivate a productive discussion regarding the future and how this tool can be best integrated into an HPM fellowship training program. It was suggested to create different versions of the learning tool, one for early learners and one for advanced learners. This approach may help to separate out advanced techniques from basic skills. Similarly, fellows suggested that it might be helpful to identify on the tool when in the year they would be expected to reach certain levels and to master certain skills within the checklist.

Another recommended change centered around the scoring component of the checklist. The scoring system led to the APCC directed feedback being misconstrued by fellows as summative rather than the intended formative feedback, and unnecessarily increased anxiety for learners. Fellows also noted variability in scores from attendings week to week, and that overall, the scores were harder to interpret when completed by an attending who had less of a longitudinal relationship with that fellow. In focus groups, the Likert scoring section was requested to be removed by both fellows and faculty.

‘One week they are told they are doing proficiently and the next week they are told they are not doing proficiently, so there’s a little bit of confusion on the fellows part with that which again shows the variation of week to week. It is not a linear growth. At the end of the fellowship there is a positive trajectory but there will be ups and downs.’

-Faculty 1

‘The ratings on the checklist are not super helpful for me, there is just extreme variability between attendings…One week you are performing with very high numbers and the following week with a different attending it’s quite low and it’s all over the place.’

-Fellow 3

‘It might be a challenge to see your growth when you’ve only worked with an attending month six or seven and so they don’t see any sort of development and might grade differently…whereas someone who has been with you before might actually see how you have improved on things.’

-Fellow 1

Lastly, there was a consensus between both groups that the APCC was most helpful in the first half of the year as fellows were gaining familiarity with the language of communication skills, and subsequently started to become more redundant in the latter part of the year.

‘Having learners use this primarily the first six months of the year and then shifting to the latter half to a more of a learned defined way.’

- Faculty 3

‘I think for me at the beginning it was very helpful, but towards the end or midway forward, it was least helpful.’

- Fellow 1

Discussion

The Advanced Palliative Communication Checklist is a novel tool for teaching expert-level palliative care communication skills and represents an important step in defining how to provide the most effective training in this domain. Fellows primarily identified the APCC as a vehicle for delivering targeted feedback, while faculty perceived the APCC as a tool for knowledge organization and the generation of learning goals. These three uses are integral parts of the learning process and elucidate the benefits of the APCC, which is the creation of a shared language and mental model for naming and organizing advanced communication skills. This creation of a shared language and mental model then facilitates trainees and faculty to better identify clear learning goals and provide skill-specific, targeted feedback.

The qualitative results of this study demonstrated the dual nature of feedback tools such as the APCC, in their potential to be impactful and beneficial, while also adding to the anxiety that is often associated with feedback. The emphasis on communication training in HPM fellowship inherently includes an increase in direct observation and feedback, compared to most other medical training environments. Therefore, the fellows’ adjustment to this change is already part of their learning process from the first days to weeks of their fellowship training. It is then difficult to untangle the role of a learning tool involved in feedback with the natural increase in feedback from direct observation and how a trainee perceives that experience. HPM fellows also encounter heightened emotional stress throughout their fellowship year, from the unique intensity of their conversations with patients, the experience of vicarious trauma, and the possible countertransference from their own life experiences. It is particularly important that serious illness communication tools support and challenge the learner, without imposing counterproductive levels of stress and becoming an unintended barrier to a healthy growth mindset. This feedback associated challenge also makes it particularly salient that the APCC can exist as a learning tool for naming and organizing communication skills separate from formal feedback encounters, so that it is not only associated with the feedback experience.

Based on results of this pilot study, we have adjusted the way this tool is implemented each subsequent year. Given the faculty and fellow experiences (Figure 2) and the discussion around future utilization, we decided to remove the scoring system. Without scores, the tool was optimized as a comprehensive list of skills and expectations surrounding serious illness communication for a graduating HPM fellow. Additionally, we permanently revised our implementation of the APCC, so the tool is no longer expected to be used during individual patient encounters, but rather at the beginning and end of the week to reflect on strengths and to prioritize areas for continued growth. While we do not require the fellows to use the APCC weekly, the program requires the fellows to engage in a formal review of the checklist with a faculty member of their choice bi-annually at mid- and end-of-year. They are instructed to bring the APCC to discuss with program leadership during their mid-year review, to elucidate strengths within their skill set and to identify areas of growth needed for the remainder of the year. The APCC continues to be incorporated into fellowship orientation, weekly didactics, and communication workshops in order to consistently expose fellows to the shared language and organizational structure it provides. The most updated version of the APCC is listed in Appendix 1, with the original version listed in Appendix 2 for comparison.

There were two main limitations identified in this pilot study, 1) two of the attendings that participated in the pilot and the focus groups also contributed to the development of the tool and 2) this is a single center study within one fellowship program, leading to a small sample size of fellows and faculty that participated in the focus groups. Additionally, it would be ideal to undergo expert review of the tool on a national scale outside of the study institution, to improve content validity and potentially expand its utilization to offer a standard communication approach for HPM fellows on a broader scale.

Conclusion

The Advanced Palliative Communication Checklist expands on previously established communication frameworks by providing a comprehensive resource for Hospice and Palliative Medicine fellows in advanced communication skills education. In providing a shared language and mental model for naming and organizing communication skills, it assists with both identifying learning goals throughout the year, and facilitating specific, higher quality feedback. This study also highlights some of the challenges with incorporating communication tools effectively into the process of formative feedback within an HPM fellowship. Further exploration is needed to better understand how communication education can be best integrated into existing learning and feedback structures.

ORCID iDs

Shannon Rose Bell https://orcid.org/0009-0003-5181-8742

Katelyn Dow Stepanyan https://orcid.org/0000-0002-5967-5374