Improving Residents' Code Status Discussion Skills: A Randomized Trial

Abstract

Background

Inpatient Code Status Discussions (CSDs) are commonly facilitated by resident physicians, despite inadequate training. We studied the efficacy of a CSD communication skills training intervention for internal medicine residents.

Methods

This was a prospective, randomized controlled trial of a multimodality communication skills educational intervention for postgraduate year (PGY) 1 residents. Intervention group residents completed a 2 hour teaching session with deliberate practice of communication skills, online modules, self-reflection, and a booster training session in addition to assigned clinical rotations. Control group residents completed clinical rotations alone. CSD skills of residents in both groups were assessed 2 months after the intervention using an 18 item behavioral checklist during a standardized patient encounter. Average scores for intervention and control group residents were calculated and between-group differences on the CSD skills assessment were evaluated using two-tailed independent sample t tests.

Results

Intervention group residents displayed higher overall scores on the simulated CSD (75.1% versus 53.2%, p<0.0001) than control group residents. The intervention group also displayed a greater number of key CSD communication behaviors and facilitated significantly longer conversations. The training, evaluation, and feedback sessions were rated highly.

Conclusion

A focused, multimodality curriculum can improve resident performance of simulated CSDs. Skill improvement lasted for at least 2 months after the intervention. Further studies are needed to assess skill retention and to set minimum performance standards.

Introducton

Code status or “do not resuscitate” discussions provide an important opportunity for physicians to ascertain patient preferences regarding life-sustaining therapies and goals of care, particularly in the context of serious illness. However, as these conversations involve discussion about death and dying, they can be awkward, emotional, and intimidating for both patients and health care providers. Rosen and Tesser addressed the tendency to avoid tough conversations over 40 years ago in what they termed “The MUM Effect.”¹ Therefore, arming physicians with the communication skills required to conduct effective CSDs is critical, especially given the increasing number of seriously ill patients cared for in the hospital.²

Although resident physicians routinely facilitate CSDs with patients admitted to academic medical centers,^3,4 they often lack the skills to conduct effective discussions. Residents report feeling underprepared to conduct CSDs competently,^5,6 not only because they lack adequate training, feedback, and modeling,^3,4,7 but also because of the complex nature of the conversations.⁸ Prior research has shown that these conversations often miss key elements needed for informed decision making including: 1) discussion about the patient's prognosis and goals for care; 2) the likelihood that cardiopulmonary resuscitation (CPR) will produce results consistent with the patient's goals; and 3) a recommendation about CPR and other treatments that fits the patient's values.^9,10

Previous studies have utilized short-term, intense training through workshops and seminars aimed at improving the communication skills of medical students,¹¹ residents,^12,13 and oncology fellows¹⁴ in the delivery of bad news and discussion of end-of-life (EOL) issues. However, no randomized controlled trials have evaluated a communication skills intervention to improve resident skill in performing a CSD. Further, data about retention of CSD skill are also unknown. Therefore, the current study had three aims: 1) to design and implement a multimodality CSD communication skills intervention in an internal medicine residency program; 2) to compare CSD performance measured by a standardized patient assessment between those randomized to the intervention and a control group; and 3) to assess retention of CSD skills 2 months after the intervention.

Methods

Study design and setting

This was a prospective, randomized controlled trial of a multimodality communication skills intervention designed to improve CSD skills in postgraduate year (PGY) 1 residents at a large, academic medical center. The study was conducted between July 2010 and January 2011. The Northwestern University Institutional Review Board approved the study, and all participants provided informed consent before participating.

Participants

All 38 PGY1 internal medicine residents at Northwestern University in July 2010 were eligible to participate in the study. Residents were randomly assigned to receive the educational intervention (n=19) or serve as controls (n=19) (Fig. 1). During the 6 month study period, all residents completed traditional clinical training, which included rotations in general internal medicine and subspecialty inpatient services and outpatient rotations at Northwestern Memorial Hospital and the Jesse Brown Veteran's Affairs Hospital. All residents were supervised in clinical activities by a faculty member assigned by the Department of Medicine, and participated in didactic activities 2 hours each day.

FIG. 1.

Study design.

Procedure

Intervention group residents received a multimodality communication skills intervention designed to boost CSD skills. First, residents were divided into groups of six or seven learners and taught key components of a CSD as described in the literature and by expert consensus.^15–17 Two hospice and palliative medicine faculty members (KN and ES) were preceptors. Seminars lasted 2 hours and content included: 1) defining advance care planning, 2) understanding the indications for a CSD, 3) reviewing a framework for these discussions, and 4) exploring strategies for responding to patient emotions. Faculty next demonstrated a CSD. Residents reflected on prior experiences, analyzed a model conversation, and practiced conversation elements.

After the training session, intervention group residents received self-study materials to reinforce learning concepts presented in the interactive sessions. Each intervention resident also completed three internet-based communication skills teaching modules (doc.com module #13 “Responding to Strong Emotions”; module #32 “Advance Directives”; and module #34 “Communication Near the End of Life”)¹⁸ to complement focused reading. Finally, intervention group residents kept a log where they described at least two actual CSDs they had either led or observed. This provided residents the opportunity for self-reflection¹⁹ about the process and content of CSDs encountered during clinical training. Logs were submitted anonymously to course faculty and were reviewed to explore themes and common issues.

In November 2010, intervention group residents received a 2 hour CSD skills “booster” session. Common themes from log entries were discussed, the CSD framework was reviewed, and faculty facilitators repeated the demonstration of a CSD. After the demonstration, the group debriefed about the conversation, and reviewed questions about CSD technique and approach.

Residents in the control group completed clinical rotations alone. They did not participate in the initial small group sessions, complete the self-study component, or attend the “booster” session.

Outcomes and measurements

The primary outcome measure was the difference in performance on a CSD skills examination between intervention and control group residents. Secondary outcomes included self-confidence in facilitating a CSD, and participant satisfaction with the intervention.

Demographic information (age, gender, medical school attended, United States Medical Licensing Examination [USMLE] Step 1 and 2 scores) including actual clinical experience with CSDs was collected. Residents were asked to assess their self-confidence in performing a CSD independently on a scale of 0=not confident to 100=very confident. A Likert scale was used to assess satisfaction with the communication skills curriculum.

In January 2011, each resident was asked to perform a CSD with a trained standardized patient. All residents used the same scenario of a 45-year-old man admitted to the hospital with advanced, metastatic colon cancer, and each participant was instructed to conduct the conversation as they would with an actual patient. CSDs were videotaped and one of the two faculty facilitators watched the interaction through closed-circuit video in the Northwestern University Clinical Education Center. Subjects were allowed up to 15 minutes to perform a CSD. Subsequently, the faculty member provided 10 minutes of individualized feedback.

CSD skills were assessed via an 18 item behavioral checklist that was developed using a modified Delphi technique.²⁰ The checklist was created by a panel of three hospice and palliative medicine physicians and four communication skills experts, based on previous work,^6,13 and expert recommendation regarding essential elements of patient-centered CSD communication.^7,15–17 The checklist had three sections: 1) patient-centered interviewing skills, 2) CSD skills, and 3) responding to emotion. Each checklist item was scored dichotomously (i.e., done correctly or done incorrectly), assigned one point when performed correctly, and given equal weight. Points were summed for a total score.

A panel of American Board of Internal Medicine certified clinician-educators reviewed the checklist prior to use and made consensus recommendations for revisions based on clinical experience. To assess feasibility, the checklist was pilot tested on a group of residents and a palliative medicine fellow who were not study subjects.

Data analysis

Demographic differences between intervention and control group residents were assessed using Pearson's χ² test and two-tailed independent samples t tests. Between-group differences on the CSD skills assessment were evaluated using two-tailed independent samples t tests. A second examiner (RS), who was blind to participant group assignment, scored 19 (50% of the total sample) randomly selected communication skills assessments. Inter-rater reliability was assessed using the kappa (κ) coefficient.²¹ Multiple regression analysis was used to assess the association between CSD performance and gender, USMLE Step 1 and 2 scores, CSD clinical experience, and group assignment. All analyses were performed using PASW Statistics software, version 19.0 (SPSS, Inc., Chicago, IL).

Results

All 38 PGY-1 residents at Northwestern University were eligible for the study. All 38 consented to participate in the study and completed the entire protocol. There were more male residents in the control group (n=15) than in the intervention group (n=8). In addition, control group residents reported performing more CSDs (11.7±13.2) in actual clinical care than did intervention group residents (6.2±3.7), although this difference was not statistically significant (Table 1).

Table 1.

Participant Characteristics

	Intervention group residents n=19	Control group residents n=19
Age—years	26.3±1.3	27.2±2.5
Male sex—n (%)	8 (32.0%)	15 (48.4%)
United States medical school graduate	19 (100%)	19 (100%)
USMLE Step 1	238.1±22.3	231.5±17.4
USMLE Step 2	249.1±15.6	240.0±18.5
Code status discussion, no. performed	6.2±3.7	11.7±13.2
Code status discussion self-confidence (0=not confident, 100=very confident)	65.0±4.6	60.5±13.4
Plus-minus values are means±SD

Characteristics did not differ significantly between groups.

USMLE, United States Medical Licensing Examination.

Inter-rater reliability for the CSD assessment showed substantial agreement.²¹ Across the 18 checklist items, the mean kappa coefficient (κ) was 0.70. Intervention group residents displayed significantly higher overall performance with less variation than did control group residents on the CSD checklist outcome evaluation (75.1%±8.9 versus 53.2%±16.2, p<0.001). When controlled for gender, USMLE step 1 and 2 scores, CSD clinical experience, and group assignment using regression analysis, only group assignment (intervention versus control) remained a significant predictor of communication skills assessment performance (p<0.001). There was no significant difference between groups in self-reported confidence to conduct a CSD.

Performance on each checklist item is displayed in Table 2. In the general interviewing skills section, intervention group residents made significantly more summary statements to ensure understanding of patient comments, and demonstrated significantly more partnership building and non-abandonment. In the CSD section, intervention group residents demonstrated improved performance in asking about prior experiences with EOL decision making (90% versus 53%, p=0.01); assessing patients' understanding of their condition and prognosis (100% versus 63%, p=0.003); exploring and clarifying general values and goals (84% versus 42%, p=0.007); proposing a care plan that respects the patient's values, goals, and concerns (58% versus 11%, p=0.002); and focusing on active treatments rather than on treatments that are withheld (58% versus 11%, p=0.002). In the responding to emotion category, intervention group residents explored the patient's emotional reaction more often than did control group residents (53% versus 26%), but this was not statistically significant.

Table 2.

Code Status Discussion Checklist and Performance by Group

	Intervention group (n=19)	Control group (n=19)	p-value
A. General patient-centered interviewing skills
1. Establishes rapport promptly.	100%	84%	0.07
2. Describes the purpose of the encounter during the first third of the interview.	100%	100%	1.0
3. Uses summary statements to ensure understanding of patient's statements.	95%	68%	0.04
4. Elicits additional questions and/or concerns.	68%	90%	0.11
5. Makes explicit statement of “partnership building” and/or “non-abandonment.”	79%	47%	0.04
6. Uses non-technical language.	95%	95%	1.0
B. Discussing code status
1. Asks patient about prior experiences with end-of-life decision making	90%	53%	0.01
2. Inquires about assignment of health care proxy or power of attorney for health care and identifies proxy if not already assigned.	63%	68%	0.73
3. Assesses patient's understanding of current condition.	100%	63%	<0.01
4. Explores patient's understanding of prognosis.	84%	26%	<0.001
5. Asks about patient's concerns about the future.	42%	16%	0.07
6. Explores and clarifies the patient's general values and goals.	84%	42%	<0.01
7. Discusses outcomes of attempted resuscitation (not just specific interventions).	79%	53%	0.09
8. Proposes a care plan that respects patient's goals, values, and concerns.	58%	11%	<0.01
9. Frames recommendation by focusing on "active" treatments first, rather than just on “withheld” treatments.	58%	11%	<0.01
C. Responding to emotion
1. Names, validates, or expresses understanding of the patient's emotional reaction.	79%	79%	1.0
2. Explores patient's emotional reaction(s).	53%	26%	0.10
3. Uses silence appropriately after providing information or asking a difficult question (at least 3 seconds)	26%	26%	1.0
Total Score (mean)	75.1% (±8.9%)	53.2% (±16.2%)	<0.001
Average duration of code status conversation (min:sec) (SD, min:sec)	13:56 (3:09)	10:41 (3:42)	0.01

Responses to a course evaluation questionnaire were uniformly positive (Table 3). Participants agreed that the sessions were realistic (95%), that practice had helped them prepare to perform a CSD better than clinical experience alone (89%), and that this training should be a required component of residency education (100%).

Table 3.

Intervention Group Participant Satisfaction with the Curriculum (n=19)

Statement	% Agree (4 or 5)
Practice in the code status discussion sessions boosts my skill to perform discussions with real patients.	100%
The standardized patient scenario simulates a code status discussion realistically.	95%
Practice code status discussion sessions should be a required component of residency education.	100%
Practice in the code status discussion sessions has helped prepare me to perform code status discussions better than clinical experience alone.	89%
Knowing how to perform a code status discussion is just as important as knowing how to perform any other clinical procedure.	100%

Likert Scale: 1=strongly disagree; 2=disagree; 3=neutral; 4=agree; 5=strongly agree.

Discussion

This study shows that it is feasible to incorporate a multimodality CSD skills curriculum into residency training, and that such a curriculum improves resident performance of a simulated CSD. Residents who participated in the training sessions and completed the online and self-reflection components of the curriculum outperformed their peers who completed clinical training alone. Furthermore, the improvement in resident performance appeared to last at least 2 months after training.

These findings add to what is known about resident performance of CSDs by introducing a multimodality curriculum involving self-study and evaluating short-term skill retention. Although Williams et al.²² showed improved overall performance of EOL conversations with standardized patients after an intensive workshop, several other interventions have not significantly improved overall performance despite similar amounts of teaching time.^12,13 The use of “multiple day” training programs (ranging from 24 to 105 hours)^23–26 or “booster” sessions^27,28 have been used successfully to improve the communication skills of more experienced clinicians. Although PGY-1 residents require these skills, this much time commitment would be challenging in light of new duty hour requirements imposed by the Accreditation Council of Graduate Medical Education in July 2011.²⁹ Unlike earlier interventions for residents, this program relied heavily on ongoing reinforcement over 4 months with readings, online modules, and self-reflection. In their review of communication training interventions, Rao and colleagues found that the higher the intensity of training the greater the likelihood that skills would improve and be retained.³⁰ This dose–response relationship has been well studied, as deliberate practice, repetition, and feedback are critical to skill mastery.³¹ We believe incorporating self-study into the CSD curriculum helped achieve durable results while minimizing faculty and PGY-1 resident time commitments.

Providing residents tools to address the discomfort associated with discussing difficult topics improves their clinical skills. In their examination of “The MUM Effect,” Tesser and Rosen present evidence that the communicator avoids difficult disclosure because of concerns both for self (i.e., wanting to avoid sadness when discussing bad news) and for the recipient (i.e., concern for recipient emotionality and anxiety about coping with recipient reaction).³² This has been echoed in the “breaking bad news” literature by Buckman.³³ Weiner and Cole note that successful training programs enable clinicians to adjust counterproductive beliefs about EOL communication and help communicators deal with emotional distress in the midst of interactions with dying patients.³⁴ Individual reflection and reflective small group teaching practices³⁵ guided by trained facilitators are essential for this to happen. It is possible that our group discussions, modeling, and reflection presented methods for the learners to “reframe” CSDs for themselves. We speculate that the combination of methods led to a change in attitudes relating to CSD discussions in ways that allowed the participants to cope with their own anxieties and deepen the conversations beyond their usual practice. This hypothesis warrants further study.

Two other findings are noteworthy. First, the only positive behavior that the control group performed more often than the intervention group was “eliciting additional questions.” The significance of this finding is unclear. One possible hypothesis is that residents in the control group posed this question as a last resort, not knowing what else to say. Given the fact that the intervention group had longer discussions, and given the fact that this technique is often used at the conclusion of a conversation, it is also possible that more of the residents in the intervention group ran out of time before getting the opportunity to ask this question. Second, no difference was seen between groups in the “responding to emotion” section of the checklist. This suggests there may be two types of skills that need to be acquired and mastered within a CSD. The first skill set – “discussing code status” behaviors – may be more familiar, lending their adoption to lecture-based learning, reading, and reinforcement. The second skill set – “responding to emotion” behaviors – may be more affective, difficult to acquire, and less likely modeled in medical school and residency. Empathic responses may require more dedicated skills practice to be acquired, maintained, and used in clinical practice.

Our data do not reveal a between-group difference in resident self-confidence in leading a CSD. This is despite the fact that intervention group residents acquired significantly more communication skill, measured objectively, than did their control group peers. This finding reinforces previous findings that neither self-confidence nor clinical experience is a proxy for skill.³⁶ Dedicated educational experiences with deliberate practice are needed to boost resident competency in communication skills, just as they are needed to master physical examination techniques,³⁷ bedside procedures,³⁸ and intensive care unit patient management.³⁹ Furthermore, the relatively low self-confidence ratings even in trained residents (compared to self-confidence in performing other procedures) suggests that communication training may be more complex than other procedures in medicine, and that improving self-confidence requires even more practice and mastery.

This study has several limitations. First, it is a single institution study with a small sample size. Because our baseline assessment of participants was limited to a few items, it is possible that unknown factors – medical school education, exposure to different experiences or teachers during internship, or other unmeasured demographic variables – impacted resident performance. Second, the primary raters were not blinded to training status of residents, making it possible that bias was introduced in assessing the CSDs. This was accounted for, however, by use of a third rater who was blind to group assignment. Third, although inter-rater reliability was high, it was not perfect. This may be because of the subjectivity involved in communication skills ratings compared with evaluation of procedural or technical tasks. Fourth, residents' CSD skills were evaluated using a single standardized patient case. A broader scope of patient cases is desirable for medical learner evaluation. Finally, our evaluation of skill retention is limited by the relatively short interval (2 months) between training and evaluation. We are currently conducting a follow-up study to assess retention of these skills after 1 year.

Conclusion

In conclusion, this study introduces a model that begins to address known deficiencies in CSDs in academic hospitals.⁴⁰ Our intervention required a limited amount of faculty teaching time, an important consideration in postgraduate medical education given the limits on resident time and faculty resources. Future studies are required to refine this model and set a minimum standard of proficiency. Finally, we must ultimately determine if competency in CSD skills translates into better outcomes for patients and their families.

Acknowledgments

We thank the Northwestern internal medicine residents for their dedication to patient care and education. We acknowledge Dr. Douglas E. Vaughan, Chair of Medicine, for his support and encouragement of this work. Dr. McGaghie's contribution was supported in part by the Jacob R. Suker, M.D., professorship in medical education and by grant UL RR 025741 from the National Center for Research Resources, National Institutes of Health (NIH). The NIH had no role in the preparation, review, or approval of the manuscript.

Author Disclosure Statement

No competing financial interests exist.