Primary palliative care education in emergency medicine residency: A mixed‐methods analysis of a yearlong, multimodal intervention

Abstract

Background

Emergency medicine (EM) physicians frequently care for seriously ill patients at the end of life. Palliative care initiated in the emergency department (ED) can improve symptom management and quality of life, align treatments with patient preferences, and reduce length of hospitalization. We evaluated an educational intervention with digital tools for palliative care discussions in an urban EM residency using the reach, effectiveness, adoption, implementation, and maintenance (RE‐AIM) framework.

Methods

Our intervention, conducted from July 2020 to August 2021, included education on palliative care techniques, digital tools, and incentives for participation. We tracked goals of care conversations and palliative care consults using electronic medical record data, conducted pre‐ and posttraining surveys, and used semistructured interviews to assess resident perspectives on palliative care conversations in the ED. Outcomes included number of goals of care conversations recorded by EM residents, consults to palliative care from the ED, and resident perspectives on palliative care in EM.

Results

The results were as follows: reach—45 residents participated in the intervention; effectiveness—89 goals of care conversations were documented by 23 ED residents, and palliative care consults increased from approximately four to 10 monthly; adoption—over half the residents who participated in the intervention documented goals of care discussions using an electronic dotphrase; implementation—by the completion of the intervention, residents reported increased comfort with goals of care conversations, saw palliative care as part of their responsibility as EM physicians, and effectively documented goals of care discussions; and maintenance—at 2‐month follow up, palliative care consults from the ED remained at approximately 10 monthly, and digital tools to prompt and track palliative care discussions remained in use.

Conclusions

An integrated palliative care training for EM residents with technological assists was successful in facilitating goals of care discussions and increasing palliative care consults from the ED.

INTRODUCTION

Emergency medicine (EM) physicians frequently care for seriously ill patients with acute and chronic conditions. Many of these patients present with complex psychosocial and medical needs at the end of life, with studies showing up to 75% of older adults visit the emergency department (ED) within the last 6 months of life. ¹ , ² , ³ , ⁴ , ⁵ For these patients and their families, early initiation of palliative care services, which seek to alleviate suffering while promoting quality of life, has increasingly become a priority of both palliative and EM practitioners as it clarifies goals of care and informs treatment strategies and patient disposition. ⁶ , ⁷ When initiated in the ED, palliative care can improve symptom management and quality of life, align treatments with patient and family preferences, and reduce inpatient lengths of hospitalization. ⁸ , ⁹ , ¹⁰

Although the Institute of Medicine's (IOM) 2014 report, Dying in America: Improving Quality and Honoring Individual Preferences Near the End of Life, recommended that palliative care be considered as core training for every clinician who cares for seriously ill patients nearing the end of life, there are particular challenges related to the integration of palliative care techniques in the ED. ¹ , ² , ⁶ Many palliative care decisions are based on a thorough and accurate understanding of a patient's disease process and prognosis, which can be challenging to obtain in an ED visit. With this limited information as well as limited time and lack of access to specialized palliative care services, many EM physicians are reluctant to provide anything less than maximal medical interventions, even if this may not be aligned with a patient's wishes. ¹¹ , ¹² In response to these challenges, there has been a recent push to increase palliative care education among EM physicians, most notably through the Education in Palliative and End‐of‐Life Care for Emergency Medicine (EPEC‐EM) curriculum. ¹³ Palliative care training in most EM programs has not been formalized, despite clear interest from physicians. ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷ Furthermore, residents lack training on skills such as ethical/legal issues and withdrawing/withholding nonbeneficial interventions. ¹¹ As more education in palliative care is offered to EM residents, it is important to assess the efficacy of such interventions and continue to tailor them to the specific needs of learners. ¹² , ¹³ , ¹⁴ , ¹⁵ , ¹⁶ , ¹⁷

In an effort to improve quality of care for seriously ill patients, the Palliative Care Service at Highland Hospital in collaboration with the California Health Care Foundation (CHCF) selected the ED to pilot a primary palliative care training intervention for EM residents and physicians. The primary goal of this intervention was to implement primary palliative care skills including pain management and advanced communication skills for difficult conversations in the ED.

METHODS

We used the RE‐AIM (reach, effectiveness, adoption, implementation, and maintenance) framework to evaluate the overall impact of our ED palliative care intervention (Table 1). ¹⁸ We chose this framework because it provides a pragmatic approach to evaluate the implementation and scalability of interventions and is especially useful in assessing the implementation of complex interventions for which evaluation is not feasible through typically rigid efficacy studies. We used both quantitative and qualitative methods to assess outcomes according to this framework. For the qualitative analysis, we adhered to the Consolidated Criteria for Reporting Qualitative Research (COREQ). ¹⁹

TABLE 1.

RE‐AIM framework

Dimension	Outcomes	Data sources
Reach How well did the intervention reach the target population?	Residents participating in the ED palliative care intervention	Residency enrollment
Effectiveness How well did the intervention achieve intended outcomes?	Number of seriously ill patients identified using ICD‐10 criteria Number of residents documenting goals of care conversations Number of palliative care consults from the ED Resident comfort with primary palliative care techniques before and after trainings	Electronic medical record data Survey data
Adoption What setting characteristics impacted implementation?	Facilitators and barriers to palliative care discussions in the ED	Qualitative interviews
Implementation Were the intervention components implemented as intended?	Number of providers documenting goals of care discussions Change in number of palliative care consults from the ED Provider comfort with and knowledge of palliative care	Qualitative Interviews Survey data Hospital‐reported palliative care consults Electronic medical record data
Maintenance The extent to which the intervention becomes part of routine educational practices and maintains effectiveness.	Steps taken to continue funding palliative care training for ED providers	Hospital‐reported grant outcomes

Setting

This intervention was implemented at a county hospital residency program with the support of residency program leadership. Highland Hospital is a public hospital and level one trauma center in Oakland, California, home to a 4‐year Accreditation Council for Graduate Medical Education (ACGME)‐accredited residency in EM with 45 total residents. There is a palliative care consult service available for ED and inpatient consults during business hours.

The ED at Highland Hospital sees over 70,000 patients per year, with approximately one quarter of these patients being evaluated in the contiguous Fast Track (urgent care) system. Fast Track patients were excluded from the analysis. Approximately 21% of patients seen in the main ED are admitted to the hospital, 3.5% to critical care beds. There are also approximately 3500 trauma activations per year. These patients are primarily managed by a surgical team and were also excluded from the analysis. Thus, the total number of patients seen by EM residents during the intervention is estimated to be approximately 53,083 over 13 months.

All active residents (n = 45) were exposed to the intervention, which was led by a team of senior and junior residents with faculty mentorship from the Department of Emergency Medicine and Palliative Care Service at Highland Hospital. As this was an educational intervention, there was no formal consent required. Four residents were part of the intervention team and were excluded from the analysis.

Intervention

Our intervention is a bundled intervention that consists of educational modules, passive learning, physical tools, digital tools, and financial incentives (Table 2). Educational modules included VitalTalk, Grand Rounds, and an Introduction to Palliative Care lecture for interns. VitalTalk (Copyright © 2021 Vital Talk, vitaltalk.org) is an established program for teaching clinicians about treating patients with serious illness using simulated patients/families, role‐playing, and small‐group learning with constructive feedback to improve communication skills. The format has been adapted for EM physicians (EMTalk). Our use of VitalTalk was for educational and not clinical use. VitalTalk educators were trained VitalTalk faculty, many of them dual‐boarded in EM and palliative care. In addition to a single‐day EMTalk training held on August 26, 2020, we hosted one virtual 60‐min Grand Rounds lecture by a national expert in EM and palliative care (August 19, 2020), and hosted faculty from the Highland Hospital Palliative Care Service to hold a 20‐min small‐group workshop at the weekly EM residency conference on primary palliative care techniques with a focus on discussing code status in the ED (February 17, 2021). Furthermore, the palliative care service advised two PGY‐2 EM residents in developing a 60‐min introduction to palliative care workshop for interns (July 27, 2020), which was taught by two EM residents and one faculty in palliative care. We highlighted learning points from the Education in Palliative & End‐of‐Life Care Program Emergency Medicine (EPEC™‐EM) curriculum (a curriculum designed to teach primary palliative care to emergency physicians) with weekly slides labeled “Palliative Care Pearls,” which promoted passive learning during conference breaks. ¹³ Each “Pearl” condensed one of the EPEC‐EM lectures into a single slide that was shown during a 5‐ to 10‐min conference break. Although the intent was to show one slide per week, due to difficulties transitioning to online conferences during COVID, they were shown approximately biweekly, with a total of 19 slides shown during the year of the intervention. EPEC‐EM materials were accessed with the written permission of faculty in that program and used for educational purposes only. The intervention was designed with the understanding that not all residents would be able to participate in all educational sessions. Notably, our intervention was affected by the COVID‐19 pandemic and the transition from in‐person to virtual learning, which meant that all training modules (EMTalk, VitalTalk, small‐group workshops, and intern introduction to palliative care workshop) as well as the Grand Rounds were held online.

TABLE 2.

Intervention components

Intervention component	Description
Educational	VitalTalk training for EM physicians (EMTalk) Grand Rounds by national expert in EM and palliative care 20‐min small‐group workshop for residents 60‐min introduction to palliative care lecture for interns Palliative care pearls—weekly highlights from the EPEC‐EM curriculum
Physical tools	Badge Buddies with NURSE and UFO‐MAP mnemonics NURSE: naming, understanding, respecting, supporting, exploring UFO‐MAP: understand, fill in, outcome, map, align, plan
Digital tools	.GOC dotphrase in the electronic medical record
Financial incentives	$100 monthly prize to EM resident with the most documented goals of care discussions $500 prize to EM resident with the most documented goals of care discussions in the 12‐month academic year of the intervention

Physical tools used in our intervention were small, laminated cards (“Badge Buddies”) with guidelines on when to have palliative care conversations, a 5‐min goals of care discussion guide, and mnemonics. The mnemonics used in our badge buddies (NURSE, from VitalTalk, and UFO‐MAP) are commonly used mnemonics for delivering bad news and discussing serious illness with patients cited in primary care literature and used by our in‐house palliative care team. ¹⁹ , ²⁰ The digital tool used in the intervention was a dotphrase (.GOC) created for the electronic medical record (EMR), available in Data S1. Financial incentives were a $100 monthly gift card to the resident with the most documented goals of care conversations in that month as well as $500 prize for the resident with the most accumulated GOC documentation over the 12‐month academic year.

Implementation facilitation

Funding was provided by a $65,000 grant from CHCF to the project team, composed of two senior and two junior residents as well as two faculty sponsors, one from palliative care and one from EM. Grant recipients were also supported with monthly check‐ins with two mentors assigned by CHCF. The grant was used to reimburse our Grand Rounds speaker, fund the EMTalk session for residents, and subsidize full‐day VitalTalk trainings for two ED faculty, one senior resident, and one recent alumnus. This was an important part of our sustainability model because it will allow senior EM physicians to train junior physicians at a lower cost in the future. Lastly, the grant was used to pay for monthly financial incentives for residents as well as fund stipends for residents, medical students, and attendings who facilitated project implementation and evaluation.

Measures and outcomes

Quantitative analysis

Outcomes and data sources for the RE‐AIM analysis are summarized in Table 1. The study was approved by the Alameda Health System Institutional Review Board with a waiver of written informed consent. The analysis period was from July 20, 2020, to August 21, 2021; data were collected from posttraining surveys, EMR data, and qualitative interviews with residents. We tracked the number of seriously ill patients seen in the ED using a list of ICD‐10 codes used by the Center to Advance Palliative Care (CAPC). ²¹ We used this preliminary list as a guideline to identify patients with serious illness who may be good candidates for palliative care discussions but found it to be too broad for our population (the entire list was over 11,000 ICD codes, including codes such as “type two diabetes mellitus with hyperglycemia,” which in our patient population is very common). A team of a senior resident, EM attending, and palliative care specialist used an iterative process to refine the codes with an EMR analyst. Our final list reflected life‐threatening illnesses frequently seen in our hospital, but severe enough to warrant urgent goals of care discussions in the ED (Table 3). These diagnoses are now being used to create digital assists in the EMR, which remind physicians to review goals of care and code status with patients who are seriously ill and being seen in the ED.

TABLE 3.

ICD‐10 codes for common serious illnesses

Diagnosis	ICD‐10 code
Metastatic cancer	C79
Dementia	F03
Cardiac arrest	I46
Heart failure	I50
Stroke	I63
COPD	J44
Acute respiratory failure	J96
Alcoholic cirrhosis	K70
Renal failure	N18
Failure to thrive	R62

The number of palliative care consults from the ED was compared before and after the intervention. Goals of care conversations were not consistently tracked before the intervention but were tracked consistently during the intervention.

We collected surveys before and after individual training sessions (the introduction to palliative care workshop for PGY‐1s as well as the EMTalk training for the entire residency). Surveys asked about self‐reported comfort with primary palliative care techniques and documentation (See Data S1).

Qualitative analysis

We conducted postintervention qualitative interviews with physicians who reported attending at least three of the educational modules in the palliative care training intervention (EMTalk session, Grand Rounds, small‐group workshops, and/or PGY‐1 lecture). Residents directly involved with the intervention (n = 4) were ineligible. Participants were recruited via flyers, text, and email communication and in‐person recruitment at weekly conferences. Convenience sampling was used to select participants who met inclusion criteria and were available for interviews. Participants were compensated $10 via Amazon gift card for their participation in the interview portion of the study. Eleven participants elected to participate in interviews; two residents were ineligible for interviews due to not having attended at least three training sessions and were not interviewed. A total of nine one‐on‐one interviews were completed. Interviewees were five female and four male residents, with all classes represented (two PGY‐1, one PGY‐2, three PGY‐3, and three PGY‐4 residents). Recruitment lasted between May 2021 and July 2021 and was limited by resident eligibility and availability. Interview questions were reviewed by all members of the intervention team but were not pilot tested. Interviews were conducted in person or via teleconference in a private setting between July and August 2021 and lasted approximately 15–30 min. Interviews were structured and followed a standardized script (see Data S1). Interviews were divided roughly equally between interviewers. No repeat interviews were performed. Interviews were audio‐recorded, transcribed, and coded using a hierarchical, deductive coding system and content analysis to evaluate the aforementioned palliative care training intervention. No field notes were included in analysis. Due to time constraints on the part of both the study team and the interview subjects, transcripts were not returned to participants for comment and/or correction.

A content analysis approach was used in the analysis of the qualitative data set from semistructured interviews, as this methodology allows for the generation of meaning and insight across multiple qualitative interviews. The transcripts were reviewed by three researchers to identify emerging themes supported by multiple responses containing similar concepts. Interviews and analysis were conducted iteratively such that additional themes were identified as new interviews were conducted. Interviewers were two female medical students (RD and CN) from the University of California, San Francisco, with prior qualitative research, interviewing, and coding experience. They had no prior relationship with the residents being interviewed, and residents being interviewed only knew that RD and CN were medical students interested in palliative care and EM. They were not affiliated with the design or implementation of the intervention. CN and RD were joined by second‐year EM resident TB to code and analyze interview transcripts. All coders had prior experience in qualitative research methodology, training on qualitative interviewing, and research interests in palliative care. CN and RD were compensated $750 from the CHCF grant for their work, and TB was compensated an additional $1000 for her time. Coding was performed independently using Dedoose software (version 9.0.17), followed by multiple in‐person and virtual discussions to refine codes. To test intercoder reliability, the defined coding structure was used to code all transcripts independently, and only findings supported by multiple codes were included in results. The coding tree consisted of four major parent codes (facilitators/barriers, resources, structural, and experiences) with subcodes as described in Table 4.

TABLE 4.

Coding structure

Parent code	Subthemes	Description
Facilitators/barriers	Time	Impact of time constraints on GOC discussions
	Language	When language facilitates/impedes GOC discissions
	Family	Influence of family on GOC discussions
	COVID	Influence of COVID on GOC conversations
	Setting	Impact of physical environment/setting
	Professional purpose	GOC conversations within scope of practice or as physician meaning‐making
	Clinician comfort	Level of comfort with GOC discussions
	Continuity‐of‐care discussions	Role of ED GOC discussions in larger patient care trajectory
Resources	Script	Scripts or phrases utilized in GOC conversations
	Training	References to educational interventions
	Practice SP	Experiences with standardized patients
	Technology	Using technology to facilitate or document GOC discussions
Structural	PC collaboration	Collaboration between PC service and ED
	Documentation	Documenting GOC conversations in the ED
	Structural harm	References to racism/sexism/classism
Experiences	Goals of care	Memorable experiences with GOC conversations
	Rapport	Rapport‐building in reference to GOC conversations
	Target population	Characteristics of patients who are candidates for a GOC conversation
	Theory/Framework	Frameworks/approaches used in GOC discussions

Abbreviations: GOC, goals of care; PC, palliative care.

RESULTS

Reach

A total of 45 residents participated in some form of palliative care training. We were unable to track attendance and participation by attendings and ancillary staff. Extrapolating from yearly data, of approximately 53,083 patients seen over 13 months in the ED, we estimate 11,148 were admitted to the hospital, 1858 to critical care beds. Using EMR data, of the patients seen in the ED from July 20, 2020, to August 21, 2021, 995 met our “seriously ill” criteria and were over the age of 60.

Effectiveness

Quantitative results

Over the course of the year, 23 different EM residents documented a total of 89 goals of care conversations with seriously ill patients in the ED. Palliative care consults from the ED increased from an average of four per month (prior to July 20, 2020) to 10 per month (from July 20, 2020, to August 21, 2021). The increase in palliative care consults persisted for 2 months after the intervention, at which point we stopped tracking data. Residents continued to have an average of three GOC conversations monthly until December 2021, at which point the .GOC dotphrase was integrated into all charts automatically and required a different method of tracking. Goals of care conversations peaked at the latter half of the intervention and decreased at the end of the academic year and intervention (Figure 1).

FIGURE 1.

Documented goals of care conversations by month and intervention components

Of the 89 total patients who had goals of care conversations in the ED, 72 were admitted, seven were discharged, two were transferred, and seven expired. Thirteen of the 72 admitted patients (18%) were admitted to the intensive care unit.

Survey data

Pre‐ and post‐survey data were collected after the PGY‐1 introduction to palliative care lecture as well as the EMTalk lecture. Of 11 PGY‐1s who participated in the hour‐long training, 10 completed the preworkshop survey and nine completed the postworkshop survey.

Prior to the PGY‐1 training, only four of 10 respondents were aware that palliative care services were available in the ED. Following the training, all respondents were aware of the availability of palliative care in the ED. Furthermore, participants reported increased comfort with identifying patients who could benefit from palliative care discussions, delivering bad news to patients, leading goals of care conversations, and documenting decisions for care at the end of life (Figure 2). Survey data from before and after the residency wide EMTalk training (11 pretest responses, 14 posttest responses) also showed subjectively increased comfort with palliative care discussions (Figure 3).

FIGURE 2.

Selected results from PGY1 introduction to palliative care pre‐ and post‐training surveys

FIGURE 3.

VitalTalk pre‐ and post‐training surveys

Adoption

Forty‐five residents participated in the intervention. Over half (23 of 45) of residents used the .GOC dotphrase during the intervention, and it remained in use several months after the conclusion of the intervention and cessation of financial incentives. Qualitative interviews with residents allowed us to elicit resident perspectives of barriers and facilitators to primary palliative care and goals of care discussions in the ED.

Qualitative results

Interviews with residents revealed the following main themes: the perception of palliative care as integral to patient‐centered care, barriers/facilitators for palliative care conversations in the ED, and palliative care enhancing residents' sense of professional purpose in the ED. These topics are described with relevant themes in the following paragraphs and in Tables 5 and 6.

TABLE 5.

Barriers to palliative care conversations in the ED

Barrier	Representative quotes from deidentified interviews
Structural barriers	“The ED is an environment where many things are happening at once. It's really busy, really loud. Maybe you are not in a private room. So, there's always distractions and things that can prevent you from having [palliative care discussions].”—Interviewee 4
Time pressures	“It feels unfair to make somebody make these decisions with limited time. And the ED, on a good day, I might have 15 minutes to sit with a patient and ask them about their medical problems and do a physical exam and tell them about the plan. That's on a good day. On a bad day, you might have like three.”—Interviewee 4

TABLE 6.

Factors that facilitate palliative care conversations in the ED

Facilitator	Representative quotes from deidentified interviews
Simulated practice sessions (virtual)	“Just having had the opportunities to practice these conversations, both with patients and families and in small groups, and with standardized actors has been nice, and just makes it a little bit more comfortable and smooth.”—Interviewee 8 “I'm pleased to have practiced different ways of asking these questions in a safe and controlled environment to feel more comfortable saying them whenever the situations do arise.”—Interviewee 9
Scripts/Structured Approach to Discussions	“It was nice to have the training to get used to the types of language to use, and for myself as a provider to have a set group of questions to ask. It can always feel awkward or invasive when you are embarking on a sensitive topic, and so it can make providers feel awkward, let alone the patients.”—Interviewee 9
Documentation in the electronic medical record	“Now I feel like if I have a conversation, it's not redundant or superfluous, but it's documented in the chart in a way someone can find it. So obviously I cannot do the entire long goals of care discussion with many patients but getting a little insight into where they are and then putting that in an easy‐to‐find spot in the chart entices me to do it more often.”—Interviewee 2 “Time is always a barrier. I do not think there's much that can be done about that. It's always going to be busy in the ED, but one thing that helps is having the doc phrase that they made, because that makes it at least the documentation of it a lot easier.”—Interviewee 7
Dotphrases as triggers for goals of care discussions	“One thing I even just recently did in my documentation is I autopopulate the goals of care dotphrase into my note template, my standard note template, just as a reminder to myself.”—Interviewee 1

Palliative care as patient‐centered care

Respondents agreed that palliative care in the ED promoted a more patient‐centered approach. Some respondents likened the palliative care approach to other practices such as trauma‐informed care and harm reduction. One respondent described the shift towards use of palliative care as “EM becoming more person‐centered and less algorithmic” (Interviewee 6).

Residents underlined the utility of palliative care skills as effective tools for addressing societal and health care inequities and partnering with the patient to better align care with their goals. They underscored the positive impact that primary palliative care in the ED has on patient care, saying “I know you're busy in the ED, but this is something that you should be doing with your time. And it is something that contributes to patient care. So, I think, especially for the newer residents who are learning to practice, kind of making it a standard that this is what something that we do, is a good outcome” (Interviewee 7). Residents linked this sense of allyship with patients to job satisfaction.

Barriers to palliative care in the ED

Consistent with previous studies, residents described limited time; lack of prior knowledge of patients; and crowded, loud environments as common challenges to implementing palliative care in the ED. Communication difficulties due to language barriers, health status, or disability as well as navigating complex family structures were also common challenges that emerged as minor themes (Table 5).

Facilitators for palliative care conversations in the ED

Residents cited the use of scripts, simulated practice sessions, and digital tools as facilitators for palliative care discussions in the ED. A minority of residents interviewed cited training in medical school as a foundation of their palliative care knowledge. One resident mentioned using independent digital tools (phone application) to facilitate goals of care discussions and difficult conversations (Table 6).

Enhancing professional purpose in the ED

A frequent theme in our analysis revolved around the responsibility of EM clinicians to provide palliative care services. Residents described the ED as the sole, or primary, site of health care contact for some patients. They also recognized the acuity of the patients seeking care in the ED. In both cases, many respondents agreed that the ED was an important setting to facilitate end‐of‐life care discussions. As one respondent suggested, “It would be easy, I think, for an emergency medicine doctor to not think that was part of their job. And that would be silly … And it would be particularly silly for us at the county hospital to think that it wasn't our job because sometimes we're the only doctors for our patients” (Interviewee 6).

Respondents also noted the role of the ED as part of the continuum of care for seriously ill patients, citing goals of care conversations in the ED as an important part of a larger discussion. “The conversation is like a journey. The right thing to do is to have care physicians start wherever [patients] are” (Interviewee 6). EM residents also mentioned the importance of being “the first touch for patients to kind of prepare them for future discussions” (Interviewee 5). As reflected by quantitative data, residents described increased engagement between the ED and the palliative care service after the intervention.

Implementation

Initially, we planned to implement training in primary palliative care for residents, attendings, and ancillary staff. Although attendings and graduates were able to attend VitalTalk sessions as part of sustainability planning, it was difficult to engage nurses and physician assistants (PAs) in our intervention as they do not have protected educational time to attend sessions and were not offered continuing education credits for attendance. Furthermore, we felt it would be necessary to tailor interventions to the roles of PAs and nurses in the ED, which are slightly different from that of resident physicians. As such, the intervention was scaled back to include only 45 residents.

Maintenance

Two months after the conclusion of the intervention, the number of palliative care consults from the ED remained elevated, and the dotphrase was still being used by residents to track goals of care conversations. Building on the success of the .GOC dotphrase, after the conclusion of the intervention we collaborated with the hospital quality improvement team and EMR analysts to embed a section in the EMR prompting physicians to discuss and document advance care planning with all patients who meet seriously ill criteria (based on our ICD10 code analysis) and do not have an updated code status or POLST on file. We have not yet found funding to continue the financial $100 reward for residents using the dotphrase but have met with the hospital physician staffing group to discuss funding for additional VitalTalk trainings and resident financial incentives. However, we have plans for a yearly introduction to palliative care lecture during PGY‐1 orientation, are continuing to use the Badge Buddies, and are working with EM faculty and the palliative care service to develop annual palliative care lectures and small groups as well as a palliative care elective for emergency residents, which is now open for enrollment. Lastly, two additional hospitals within our county system have expressed interest in our intervention, and in particular, adopting the dotphrase to track goals of care conversations.

DISCUSSION

Our year‐long, multimodal intervention demonstrates that primary palliative care techniques can be taught to and implemented by EM residents in a largely virtual format, resulting in an increase in documented goals of care conversations in the ED and consults to the palliative care service for seriously ill patients. Prior research suggests that this may lead to decreased hospital costs and lengths of stay for patients, which was not evaluated in our study but may have been an unintended benefit of our intervention. ⁹ , ¹⁰

Our intervention builds on a large body of work integrating primary palliative care into EM education, underscoring the continued interest among EM clinicians in primary palliative care techniques. ¹ , ² , ⁸ , ¹¹ , ¹⁵ , ¹⁶ , ¹⁷ , ²² Our findings suggest that even virtual trainings and simulations with digital assists (i.e., dotphrases) can lead to increased resident comfort when discussing serious illnesses and leading goals of care discussions, despite common barriers such as limited time and space in the ED. Other studies have demonstrated the efficacy of asynchronous learning modules in teaching palliative care to EM residents; we offer a synchronous, multimodal approach that may be more easily integrated into standardized residency curricula using existing resources. ²²

The COVID‐19 pandemic presented significant implementation challenges. It took significantly more time and resources to change planned EMTalk courses, PGY‐1 orientation, and small‐group sessions from in‐person to online. That being said, we were able to use the pandemic as a reason to roll out our project slightly earlier, scheduling initial educational sessions at the beginning of the academic year rather than in the fall. Although this limited our preintervention data collection, with the significant mortality associated with COVID‐19 we understood that our seriously ill population would be much greater and would need goals of care discussions. Furthermore, results from our intervention demonstrate that an online curriculum can lead to effective adoption of primary palliative care techniques in the ED as well as long‐lasting institutional change. The urgency of the COVID‐19 pandemic may have increased motivation to learn about and implement primary palliative care techniques in the ED. Additionally, support from individual project champions in palliative care allowed for close collaboration between EM and palliative care, which may have played a role in the increased referrals to the palliative care service from the ED. While transitioning to in‐person education may lead to even more resident engagement, the success of our largely online and digital intervention suggests that this intervention is rapidly scalable and may be adopted across residency programs even when in‐person education and expertise is not available.

Future directions for primary palliative care education in the EM residency programs may include (1) developing a standardized list of seriously ill diagnoses that can be used to prompt clinicians to have goals of care discussions; (2) refining and sharing digital tools for tracking goals of care conversations and tracking advanced care planning documents; and (3) partnering with VitalTalk or EMTalk to develop a standardized, online curriculum to teach primary palliative care techniques to residents. Within our program, it may be useful to compare goals of care conversations before and after in‐person trainings, particularly among new classes of residents.

Limitations

There are multiple limitations to our study. All residents were exposed to the intervention, and without a comparator group that did not participate, it is not possible to directly isolate the result of our intervention on goals of care conversations and attitudes regarding palliative care among EM physicians. Unfortunately, we were not able to consistently track attendance by residents for each component of the intervention (e.g., Vital Talks, Grand Rounds). Therefore, we cannot comment on the specific effect sizes of each portion of the intervention. Our intervention and the ICD‐10 codes selected were tailored to the needs of our resident physicians and patients at our safety net hospital and may not be generalizable to other hospitals in California or the broader United States. We also received funding and incentives for residents, which limits the generalizability of our findings. Of note, goals of care conversations decreased after the cessation of the intervention and funding, but it is difficult to say what factor led to this (cessation of the intervention, graduation of a PGY‐4 class that had been trained in primary palliative care techniques, or cessation of the financial incentives). Our evaluation was also limited by a small sample size for physician interviews, due in large part to our strict inclusion criteria. Many emergency residents had participated in one or two palliative care trainings, but due to the demands of residency as well as COVID and the need for virtual training sessions, few had been able to attend the three of four palliative care trainings to qualify to participate in interviews. The small interview size likely prevented us from reaching thematic saturation and might have been inherently biased. In the future, having consistent pre‐ and posttraining surveys and more closely tracking attendance would improve our ability to understand the impact of different types of training interventions on palliative care practices in the ED. This would also improve the internal validity of our study. We did not use a previously validated survey for quantitative data, which may also introduce bias. Furthermore, tracking ongoing participation in the form of dotphrase usage and consults to palliative care for more than 2 months would give us a better indication of the lasting effects of the intervention, and it also possible that some goals of care conversations happened without using the dotphrase.

CONCLUSIONS

In conclusion, an integrated, online palliative care training for emergency medicine residents with technological assists was successful in facilitating goals of care discussions and increasing palliative care consults in an urban ED during the COVID‐19 pandemic. As we found in our interviews, EM residents are motivated to incorporate primary palliative care techniques into their practice. Online resources make primary palliative care education more accessible to EM residencies nationwide. Our data suggest that supplementing short online sessions (approximately 3 h per year) with simulated cases (one to two yearly), memory aids such as badge buddies, financial incentives for resident participation, and/or electronic assists to document goals of care conversations can significantly increase comfort with and documentation of goals of care conversations in the ED. These effects are most pronounced during the intervention. At our hospital, incorporating primary palliative care education into our residency curriculum also increased collaboration between EM residents and palliative care services for seriously ill patients in the ED, an effect that persisted beyond the intervention. We recommend that residencies seeking to incorporate palliative care education into their programs should review existing resources designed for emergency physicians, incorporate simulation or standardized patients into educational sessions, and consider adding an electronic dotphrase or standardized documentation format for goals of care conversations. Such education may equip future emergency physicians with additional tools to ensure patient‐centered care for seriously ill individuals.

AUTHOR CONTRIBUTIONS

Acquisition of funding: Linda Bulman, Justin E. Moore, Amelia M. Breyre. Study concept and design: Justin E. Moore, Erik S. Anderson, Tara D. Benesch, Linda Bulman. Acquisition of data: Tara D. Benesch, Justin E. Moore, Caroline C. Nattinger, Raizel DeWitt. Analysis and interpretation of data: Tara D. Benesch, Caroline C. Nattinger, Raizel DeWitt, Justin E. Moore. Drafting the manuscript: Tara D. Benesch, Caroline C. Nattinger, Raizel DeWitt, Elaine Dellinger. Critical revision of the manuscript: Tara D. Benesch, Caroline C. Nattinger, Raizel DeWitt, Amelia M. Breyre, Elaine Dellinger, Erik S. Anderson, Linda Bulman.

CONFLICT OF INTEREST

Linda Bulman is a trained VitalTalk instructor. The other authors declare no potential conflict of interest.

Supporting information

Data S1

Benesch TD, Moore JE, Breyre AM, et al. Primary palliative care education in emergency medicine residency: A mixed‐methods analysis of a yearlong, multimodal intervention. AEM Educ Train. 2022;6:e10823. doi: 10.1002/aet2.10823