Fidelity and feasibility of a brief emergency department intervention to empower adults with serious illness to initiate advance care planning conversations

Abstract

Context

Emergency Department (ED) visits provide opportunities to empower patients to discuss advance care planning (ACP) with their outpatient clinicians, but systematically developed, feasible interventions do not currently exist. Brief negotiated interview (BNI) interventions, which allow ED clinicians to efficiently motivate patients, have potential to meet this need.

Objectives

We developed a BNI ED intervention to empower older adults with life-limiting illness to formulate and communicate medical care goals to their primary outpatient clinicians. This study assessed the fidelity and feasibility of this intervention in a high-volume ED.

Methods

We enrolled adult patients with serious illnesses (advanced cancer, congestive heart failure, chronic obstructive pulmonary disease, chronic kidney disease on dialysis, predicted survival <12 months) in an urban, tertiary care academic medical center ED. All participants received the BNI intervention. We video recorded the encounters. Two reviewers assessed the recordings for intervention fidelity based on adherence to the BNI steps (Part I) and communication skills (Part II).

Results

We reviewed 46 video recordings. The mean total adherence score was 21.07/27 (SD 3.68) or 78.04%. The Part I mean adherence score was 12.07/15 (SD 2.07) or 80.47%. The Part II mean adherence score was 9.0/12 (SD 2.51) or 75%. The majority (75.6%) of recordings met the pre-specified threshold for high intervention fidelity.

Conclusion

ED clinicians can deliver a BNI intervention to increase ACP conversations with high fidelity. Future research is needed to study the intervention’s efficacy in a wider patient population.

Introduction

Seventy five percent of older adults with life-limiting illnesses visit the emergency department (ED) in the last six months of life.¹ Most have priorities other than to live as long as possible,² yet up to 79% of older adults in the ED have not completed advance directives.³ ED visits that do not require in-the-moment decision making (e.g. a code status conversation) represent opportunities to empower seriously ill patients to formulate and communicate their goals for future medical care through advance care planning (ACP). ACP conversations are associated with increased hospice use, fewer hospitalizations, and lower rates of in-hospital death and intensive life-sustaining treatments at the end of life.^4–7 However, we currently lack feasible, evidence-based methods to facilitate ACP in the ED.

ED clinicians are interested in engaging seriously ill adults in ACP conversations⁸ but multiple barriers prevent them from doing so. ED clinicians perceive these conversations to be time-intensive,^8,9 and they often lack training in serious illness communication.¹⁰ The hectic ED environment also makes it difficult for patients and clinicians to have sensitive and difficult discussions.

To overcome known barriers and address these ACP needs, we have developed, refined, and tested a brief negotiated interview (BNI) ED intervention to empower older adults with life-limiting illness to initiate ACP conversations with their outpatient clinicians. BNI interventions are theoretically grounded in motivational interviewing (MI)¹¹ and are designed specifically for the ED. They allow clinicians to respectfully elicit patient perspectives and provide information and resources to motivate patients to take better control of their health care decisions, which promotes improved health outcomes and experiences.¹² BNI interventions have effectively reduced future substance misuse in patients with alcohol and substance use disorders.^13–17 Only recently, though, have BNI interventions been applied to advance care planning (ACP).¹⁸

The primary goal of this pilot study was to assess whether trained ED clinicians can administer a structured BNI intervention to facilitate ACP conversations with patients’ primary outpatient clinicians with high fidelity, as measured by adherence to steps of the BNI and serious illness communication skills. A secondary goal was to assess whether this intervention is feasible to deliver in a high-volume ED in an urban, tertiary care academic medical center, as measured by the percent of eligible patients enrolled and time spent delivering the intervention.

Methods

Intervention Development and Refinement

A detailed description of the intervention’s development will be published elsewhere. In brief, we conducted a systematic development process to design the BNI intervention using rapid qualitative inquiry.¹⁹ We created a prototype of our intervention by adapting a previously described BNI intervention established for alcohol dependence²⁰ to fit the needs of patients with serious illness using literature review and an expert panel. Our expert panel consisted of palliative care physicians/researchers, a psychiatrist, and ED physicians/BNI researchers who have internationally recognized expertise in the development and implementation of interventions to improve clinician communication with seriously ill patients across medical settings, as well as the BNI method.^4,20–27

The intervention’s goal is to increase patient motivation to initiate ACP conversations with their primary outpatient clinicians. In our BNI intervention, ED clinicians ask patients if they have discussed their values and priorities for their health care with their primary outpatient clinician. Interventionists then follow a BNI framework to understand patients’ perspectives and empower them to hold such conversations in the future. The intervention aims neither to change “in the moment” decision-making nor to engage in actual ACP in the ED. Instead, it is structured to “talk about talking about it.” After each patient receives the intervention, the study PI contacts the primary outpatient clinician with a standardized email or phone call. The PI informs the clinician about the patient’s enrollment in the study and describes the study’s aims. Additionally, the study PI provides outpatient clinicians with information about continuing ACP conversations with patients and with a guide to serious illness conversation.²⁶

To refine the BNI intervention, we conducted a series of mock clinical encounters between ED clinicians and members of the hospital’s Patient Family Advisory Council (PFAC). The PFAC consists of patients, family members, and executive leaders who collaborate to improve hospital programs, policies, and patient care. PFAC members provide the patient’s perspective and regularly work with clinicians on quality improvement, patient engagement, and research. After these mock encounters, we asked PFAC members to reflect on experiences they had in the ED and what it was like to receive care there. We then elicited their feedback about the intervention’s acceptability from this perspective in cognitive interviews. We iteratively refined the intervention until we reached thematic saturation from the interviews and could no longer identify modifications to be made to the BNI intervention.

Fidelity Study

Once the intervention was refined, we conducted a pilot study in the ED with seriously ill older adults. Our institutional review board approved this study. We used convenience sampling to recruit patients in the ED from September to December 2017. We included English-speaking patients ≥65 years old with serious, life-limiting illness (metastatic cancer, oxygen-dependent chronic obstructive lung disease, chronic kidney disease on dialysis, New York Heart Association stage 3 or 4 heart failure). We also included patients ≥ 18 years old if a treating ED clinician “would not be surprised if the patient died in the next 12 months.”²⁸ We excluded patients who had a medical order for life-sustaining treatment (MOLST) in the electronic health record, were determined by the treating ED clinicians to be inappropriate for this study (e.g., in acute physical or emotional distress), or could not provide informed consent.

The PI (KO) trained two other interventionists (AR and LF), who were both physician assistants without prior experience in palliative care. The PI is an ED physician who has undergone extensive training in serious illness communication through participation in well-established and recognized courses.^29–31 He developed the training with members of the expert panel, drawing on their expertise in serious illness communication pedagogy. The Brief Negotiated Intervention-Active Referral to Treatment Institute’s alcohol abuse intervention served as the model for clinician training.²⁰ The training included three components: (1) didactic training on motivational interviewing; (2) serious illness communication training with professional actors that focuses on responding to emotion, eliciting what is most important, and respectfully introducing future care planning; and (3) bedside coaching until competency is demonstrated in >5 consecutive patient encounters. Clinicians underwent 2 hours of training, which did not include time spent receiving bedside coaching.

We video recorded all encounters. We developed a fidelity checklist based on those used to assess similar BNI interventions for alcohol use disorders in the ED.³² The checklist was adapted to fit the content of our ACP intervention by the study’s PI, a palliative care physician (REL), a clinical psychologist with expertise in motivational interviewing (MY), and a palliative care physician with expertise in clinician-patient communication research (JAT). The fidelity checklist (APPENDIX 1) is organized in two parts: I. BNI Steps; II. Communication Skills. Part I consists of six domains: Opening (e.g. a mention of the research on the importance of communicating care preferences, Rapport building (e.g. ask open-ended questions), Information & feedback (e.g. elicit current thoughts about discussing goals of care), Readiness (e.g. asking “How ready are you now?”), Summary (e.g. summarize patient’s responses), and Action (e.g. elicit specific next step from patient’s perspective). Each domain has two to three subdomains, which are scored dichotomously (yes/no). The clinician administering the intervention could receive up to 15 points for Part I.

Part II consists of six domains: Language appropriateness, Reflective listening, Use of empathic statements, Assessment of mutual understanding, Listening for cues, and Redirects when needed. We scored each domain on a three-point scale: 0 if the skill was not demonstrated at all; 1 if the skill was demonstrated occasionally or demonstrated with fair quality; and 2 if the skill was demonstrated often and with high quality. If there were no opportunities for redirection, we scored this as 2. The clinician administering the intervention could receive up to 12 points for Part II, and therefore could receive a maximum of 27 points for each interview. While there is no gold standard definition of high fidelity to a behavioral intervention, experts generally consider 70-100% adherence to key components of the intervention to meet criteria for high fidelity.^33,34 For this proof-of-concept study (stage IIa of behavioral intervention development), we proactively determined that an average fidelity score of 19 points (70%) or higher would meet criteria for high fidelity to the intervention.

Two reviewers with diverse clinical backgrounds (REL and MY) watched video recordings of the interviews and completed fidelity checklists. They established operational definitions of what constituted “adherence” for each item prior to reviewing the recordings and reviewed 6 recordings together to establish consistency of coding. Each reviewer was responsible for 23 out of 46 total recordings. The reviewers watched each recording at least twice, the first time to code Part I and the second time to code Part II. The reviewers could watch the recordings as many times as necessary to feel confident in their ratings. To assess interrater reliability (IRR), 15% of the recordings (n = 6) were assessed by both reviewers, which is consistent with published methods that double code 10-20%.^35–40 The IRR was the percent agreement between the reviewers on all scores. To calculate percent agreement, we added the number of times Reviewers 1 and 2 agreed on the same data item and divided that sum by the total number of data items.⁴¹ The reviewers resolved discrepancies in ratings by consensus. If they could not reach consensus, a third reviewer (KO) scored the recording for the items in question.

We calculated descriptive and summary statistics for the checklist as a whole, and also for each of the two parts. Further, the following Part I variables were recoded to represent a total score for each domain: Opening (3 competencies), Information & Feedback (3 competencies), Readiness (3 competencies), Summary (2 competencies), and Action (3 competencies). For example, the “Readiness” domain included three separate competencies: 1) Assess readiness; 2) Reinforce positives; and 3) Ask what would make you more ready. We added the scores for the three competencies to generate one total “Readiness” competency score.

To assess feasibility, we calculated the percent of eligible patients who consented to participate in the study. We also measured the time spent in each encounter and the number of times that the interventionist was interrupted by another ED clinician or staff member. We used the Statistical Package for Social Sciences Version 24.0 (SPSS 24.0, IBM Corp., Armonk NY) for all analyses.

Results

Sample description

We summarize the characteristics of the 46 study participants included in final analyses in Table 1. Participants were 51.1% female (n = 23) and predominantly White (82.2%). Ages ranged from 39 to 94 (M = 74.29, SD = 12.25). Metastatic cancer (51.5%) was the most common diagnosis, followed by <12-month predicted mortality (26.7%), New York Heart Association (NYHA) Class III/IV Congestive Heart Failure (CHF, 8.9%), oxygen-dependent chronic obstructive pulmonary disease (COPD, 6.7%), and chronic kidney disease (CKD) on dialysis (6.7%). The Emergency Severity Index (ESI) is a 5-level triage tool that categorizes patients into 5 groups, ranging from 1 (most urgent) to 5 (least urgent).⁴² All enrolled patients had an ESI of either 2 (Emergent, 52%) or 3 (Urgent, 48%). The majority (54%) of patients were admitted to the hospital. A minority of patients (11%) were discharged home from the ED. The remainder of the enrolled patients (35%) were admitted to the ED Observation Unit. Of these patients, 69% were ultimately discharged, 25% were admitted, and 1 patient was transferred to another hospital.

Table 1 -.

Baseline characteristics of study participants

Characteristics	N=46 (%)
Age, Mean (SD)	74.2 (12.3)
Sex, N (%)
Female	24 (52.2)
Race, N (%)
White	37 (80.4)
Black	5 (10.9)
Other	4 (8.7)
Ethnicity, N (%)
Hispanic	3 (6.5)
Life-limiting condition, N (%)
Metastatic cancer	23 (50.0)
Chronic kidney disease (CKD) on dialysis	3 (6.5)
Oxygen-dependent chronic obstructive pulmonary disease (COPD)	3 (6.5)
New York Heart Association (NYHA) III/IV Congestive Heart Failure	4 (8.7)
<12-month predicted mortalitya	13 (28.3)
Emergency Severity Indexb (ESI), N (%)
2 (Emergent)	24 (52.1)
3 (Urgent)	22 (47.8)
Post-Intervention Disposition, N (%)
Home	5 (10.9)
Inpatient	25 (54.3)
ED Observation	16 (34.8)
Home	11 (68.8)
Inpatient	4 (25)
Transfer	1 (6.3)

^aED clinician would not be “surprised if died in the next 12 months.”

^bESI is a 5-level ED triage tool that categorizes patients into 5 groups, ranging from 1 (most urgent) to 5 (least urgent).⁴²

Interrater Reliability

Reviewers double-coded six (15%) of the total recordings to assess interrater reliability (IRR). To address the Cicchetti Paradox, which produces a low Kappa coefficient despite high interrater agreement, particularly in a small sample, we assessed IRR using percentages.^43–45 Overall, IRR was 90.6%. IRR for Part I (BNI Steps) was 92.2%. IRR for Part II (Communication Skills) was 89%. The two primary reviewers reached consensus in all cases and did not require the third reviewer to intervene.

Fidelity

We summarize the results for intervention fidelity and feasibility in Table 2. Total fidelity ranged from 11 to 27 out of a total possible 27 points. The mean total score was 21.07 (SD = 3.68), or 78.04%. The total Part I (BNI Steps) score ranged from 8 to 15 out of a total 15 points. The Part I mean score was 12.07 (SD = 2.07), or 80.47%. The Part II (Communication Skills) score ranged from 0 to 12 out of a total possible 12 points. The Part II mean score was 9.0 (SD = 2.51), or 75%. Of the 46 recordings analyzed, 34 (75.6%) received a score of at least 70% (19/27 or higher), thus meeting the pre-specified fidelity threshold.

Table 2 -.

Intervention fidelity and feasibility

Fidelity % scores, mean (SD)
Part I – BNI Steps (max pts)
Opening (3 pts)	3 (0)
Rapport Building (1 pt)	0.96 (0.21)
Information & Feedback (3 pts)	2.36 (0.71)
Readiness (3 pts)	2.20 (0.87)
Summary (2 pts)	1.64 (0.65)
Action (3 pts)	1.95 (0.96)
Part I score (15 pts)	12.07 (2.07)
Part II – Communication Skills (2 pts)
Appropriate language	1.45 (.50)
Reflective listening	1.82 (.39)
Use of empathic language	1.19 (.63)
Assessing mutual understanding	1.73 (.45)
Listening for cues	1.39 (.58)
Redirects when needed	1.73 (.54)
Part II score (12 pts)	9.00 (2.51)
Overall score (27 pts)	21.07 (3.68)
Encounters with high fidelitya, n (%)	34 (75.6)
Feasibility
Average Time/encounter in mins, median (IQR)	10.5 (7.5-13.5)
Interruptions, mean (SD)	0.4 (0.7)

^aTotal score >70% (≥19/27)

Feasibility

We screened 223 subjects for participation in the study. Of these, 84 met inclusion criteria and were approached for participation. Fifty subjects consented and were enrolled. The 34 subjects who did not consent were similar in demographics to those who enrolled. Most declined to participate due to feeling weak or fatigued. Of those who enrolled, two subsequently withdrew consent. A third subject had a previously-completed MOLST on file in the EMR and was withdrawn from the study by the investigators. One subject was also enrolled but could not be included in our results because the video recording equipment malfunctioned. Therefore, 46 (55%) of eligible patients completed the intervention and were included in study analyses. The median length of the BNI intervention was 10.5 min (IQR 7.5 min-13.5 min). The number of interruptions ranged from zero to three (M = .4, SD = .7).

Associations with Intervention Fidelity

Chi-square tests revealed no association between clinician type (physician assistant or physician) and intervention fidelity (yes/no), χ² (1, n = 45) = 3.15, p = .08. Chi-square tests also revealed no association between coder background (palliative care physician/psychologist) and intervention fidelity (yes/no), χ² (1, n = 39) = .01, p = .93.

Discussion

We demonstrated that trained ED clinicians can conduct the BNI intervention to increase ACP conversations with high fidelity. Most encounters met criteria for high fidelity to the intervention. Clinicians adhered to both the BNI steps and the serious illness communication components of the intervention. The majority of eligible patients consented to participate and interventions were appropriately brief in duration.

Our study showed that it is possible to engage seriously ill patients in a discussion of the importance of ACP in the ED. We were able to approach all eligible patients and over half (55%) consented to the study and underwent the intervention. Enrolled patients were quite ill – all had an ESI of either 2 or 3 and disposition for a majority (54%) was hospital admission. The high enrollment rate suggests that seriously ill patients are willing and able to participate in a BNI ACP intervention, despite numerous barriers.^46–48 We are unable to evaluate how many full ACP conversations actually occurred for the 55% of patients who participated in the study. However, increasing the number of seriously ill patients seen in the ED who are primed, and whose clinicians are primed to conduct an ACP conversation has potential to reduce the number of these “at risk” patients who visit the ED without ACP in the future.

An ED visit can serve as a “teachable moment” for seriously ill patients. Teachable moments describe health events that motivate individuals to adopt risk-reducing behaviors.⁴⁹ Investigators have used this concept to encourage a variety of behavior changes, such as tobacco cessation and pediatric screen time reduction.^49–51 In order for an event to be a teachable moment, it should: 1. Increase perceptions of personal risk and outcome expectancies; 2. Prompt a strong affective or emotional response, and 3. Redefine self-concept or social role.⁴⁹ A visit to the ED by a seriously ill patient meets all three criteria and can be used to motivate patients to avoid similar situations in the future should they not be compatible with their goals and values. Our BNI intervention leverages the ED teachable moment and also allows for the involvement of an outpatient clinician with whom a patient has greater continuity and trust.⁴

ED clinicians worry about the time investment ACP requires and their lack of adequate skills to carry out these interventions.^10,53 In our study, the median intervention lasted 10.5 minutes which is consistent other ED BNI interventions.^32,13,55 After a brief, systematic training, participating clinicians received high scores on their adherence to the steps of the intervention and on their communication with patients. Skilled communication is critical in encounters with seriously ill patients that deal with hopes, worries, goals, and values.^2,21,56,57 Inexpert communication may leave patients confused or, worse, with increased distress.^58–61 Serious illness communication skills can be taught^{21,25,56,62,63} and these pedagogic methods should be incorporated into BNI interventions for ACP.

Beyond clinician time and training, traditional payment models can disincentivize emergency clinicians from engaging in ACP. Changes implemented in 2016 by the Center for Medicare and Medicaid Services (CMS), however, allow clinicians to submit charges for time spent facilitating ACP with their patients across settings, including in the ED.^64,65 Accountable care organizations (ACOs), value-based payment, and other risk-sharing models also encourage administrators to adopt population health management strategies at the health system level.^66,67 Increasing both access to palliative care and completion of ACP has potential to add significant value.^25,68–70 If effective and integrated system-wide, the BNI approach to increase ACP could provide cost and quality incentives for the health care system over time.

This study has several limitations and its results must be interpreted in the context of its design. First, the study was designed to assess intervention fidelity and feasibility in a single ED setting. We did not assess the efficacy of the intervention on increasing ACP conversations between patients and their primary outpatient clinicians, a study that is currently ongoing. Moreover, we do not assess the intervention’s potential harms, although neither BNI nor ACP interventions have been previously shown to produce negative effects on patients.^25,13,71 Second, the study involved only three interventionists, one of whom designed the intervention. Fidelity may change as the number and heterogeneity of clinicians performing the intervention increases. Our goal was simply to see whether it was feasible to deliver the intervention with high fidelity. However, two of the three interventionists were physician assistants without prior experience or training in palliative care or serious illness communication. This suggests that it may be possible to disseminate the intervention with a wider variety of interventionists. Third, the study was performed in a single ED at a large, urban tertiary care academic medical center with a high proportion of patients with cancer and a relatively homogenous, White population. Selection of participants by convenience sampling may have also biased our results. Future prospective testing will elucidate whether the intervention is both effective and scalable to different settings and patient populations.

Conclusions

A BNI intervention may be able to harness the ED visit by seriously ill patients as a teachable moment for ACP. Although barriers exist to primary palliative care interventions in the ED, this study demonstrated that ED clinicians can deliver a BNI intervention designed to increase ACP conversations between seriously ill patients and their outpatient clinicians with high fidelity. Trained ED clinicians followed the steps of the intervention and demonstrated appropriate communication skills. Future research will study the efficacy of our ED BNI intervention on patient and clinician satisfaction, completion of ACP conversations and documentation, and end-of-life outcomes in a seriously ill patient population.

APPENDIX 1 – Fidelity Checklist

Brief Negotiated Intervention Fidelity Checklist				(SID: ______) ASSESSOR __
Administering Clinician	: ___________	Date of Intervention	: ___________	Duration of Video:	_____m _____s
Assessor Initials	: ___________	Date of Assessment	: ___________
#Interruptions	: ___________	# Times Others Speak	: ___________

PART I - BNI STEPS	SKILL COMPETENCIES	SCORE	COMMENTS
1. Opening	Setting up the topic (not related to ED care)	Ⓨ Ⓝ
	Communicate importance of future care planning	Ⓨ Ⓝ
	Ask permission to discuss	Ⓨ Ⓝ
2. Rapport Building	Ask open-ended question	Ⓨ Ⓝ
3. Information & Feedback	Elicit current thoughts about communicating goals of care	Ⓨ Ⓝ
	Provide facts	Ⓨ Ⓝ
	Sum up and restate in patient’s own words	Ⓨ Ⓝ
4. Readiness	Assess readiness	Ⓨ Ⓝ
	Reinforce positives	Ⓨ Ⓝ
	Ask what would make you more ready?/Ask why do you say you feel ready?	Ⓨ Ⓝ
5. Summary	Summarize patient’s responses	Ⓨ Ⓝ
	Confirm patient’s responses	Ⓨ Ⓝ
6. Action	Elicit specific next step from patient’s perspective	Ⓨ Ⓝ
	Make a recommendation	Ⓨ Ⓝ
	Ask permission to communicate to the outpatient clinician	Ⓨ Ⓝ
TOTAL		/15

PART II - COMMUNICATION SKILLS	SCORE	COMMENTS
1. Language appropriate	⓪ ① ②
2. Reflective listening	⓪ ① ②
3. Use of empathic language	⓪ ① ②
4. Assessing mutual understanding	⓪ ① ②
5. Listening for cues	⓪ ① ②
6. Redirects when needed	⓪ ① ②
TOTAL	/12