Effects of Education on Interns' Verbal and Electronic Handoff Documentation Skills

Abstract

Background

Improving handoff communications is a National Patient Safety Goal. Interns and residents are rarely taught how to safely handoff their patients. Our objective was to determine whether teaching safe handoff principles would improve handoff quality.

Methods

Our study was conducted on the inpatient services at 2 teaching hospitals. In this single-institution, randomized controlled trial, internal medicine interns (N  =  44) and residents (N  =  24) participated in a 45-minute educational session on safe handoff communication skills. Residents received additional education on effective feedback practices and were asked to provide each intern with structured feedback. Quality of interns' electronic and verbal handoffs was measured by using a Handoff Evaluation Tool created by the authors. The frequency of handoff communication failures was also assessed through semistructured phone interviews of postcall interns.

Results

Interns who received handoff education demonstrated superior verbal handoff skills than control interns (P < .001), while no difference was seen in electronic handoff skills. Communication failures related to code status (P < .001) and overnight tasks (P < .050) were less frequent in the intervention group.

Conclusions

Interns' electronic handoff documentation skills did not improve with the intervention. This may reflect greater difficulty in changing physicians' electronic documentation habits.

Editor's Note: The online version of this article contains the survey instrument ^{(78.5KB, doc)} used in this study. Teaching materials are available upon request to the authors.

What was known

Communication failures during handoffs are well described; lack of standardization and inadequate training have been implicated as contributing factors.

What is new

Internal medicine interns and residents participated in a 45-minute educational session on safe handoff communication skills, and residents were taught how to provide interns with structured feedback on handoff quality.

Limitations

Single-site study, with a single assessment of the outcome and the potential for instrument reactivity and observation effects.

Bottom line

We found that a brief handoff education session improved interns' verbal handoff skills and could be easily incorporated into residency programs' curricula without a need for added resources.

Introduction

Communication errors can cause preventable health care adverse events. Some of these errors occur when physicians handoff their patients' care.¹ Resident duty hour limitations have fragmented inpatient care and increased handoff frequency.² Communication failures during handoffs are well described, with lack of standardization and inadequate training in handoff skills having been implicated as contributing factors.^3,4 The Institute of Medicine⁵ has responded by recommending resident training in handoff communications. Therefore, developing ways to teach and improve resident handoff quality is a goal for educators.

Although various strategies to improve handoff quality have been described,⁶ relatively few studies have tested whether teaching residents handoff skills improves handoff quality.⁷ Our primary aim was to determine the effectiveness of handoff education on the quality of interns' electronic and verbal handoffs by comparing the handoff quality of interns who received education to those who did not. Secondary outcomes included communication failures reported by handoff receivers, environmental handoff behaviors, and changes in interns' knowledge and attitudes regarding handoffs.

Methods

Participants and Settings

Interns and residents rotating on an inpatient general or subspecialty medicine service in January 2009 at 2 University of Pennsylvania teaching hospitals were asked to participate. Services consisted of 1 attending physician, 1 resident, 2 interns, and medical students. Interns and residents were randomly assigned to services in June 2008. Subject randomization was by service so that interns giving and receiving handoffs were in the same study group. Illness severity scores were obtained and averaged for all patients on each service, 1 year before the study, by using case mix index and all patient-refined diagnosis-related groups. Services were assigned to the intervention (handoff education) or control group to equalize patient disease severity between groups. Twenty-four interns and 12 residents on 3 services received the intervention; 20 interns and 12 residents on 3 other services were controls.

At our institution, residents are responsible for the electronic handoff document from July to December. This study was conducted in January when interns assume this role without formal training. Residents and interns are responsible for verbal patient handoffs to the corresponding on-call intern and resident the entire year. Handoff documents are created in an electronic sign-out system that is integrated within our computerized order entry system (Eclipsys Corporation, Malvern, PA).

The University of Pennsylvania Institutional Review Board approved the study. All participants provided written informed consent.

Intervention

The study occurred over a 3-week period (figure). Previously, interns and residents received no formal handoff training. Control interns and residents did not receive handoff training. During study week 1, intervention interns and residents received a 1-hour educational session on handoffs led by 2 faculty members. The first part (45 minutes), for interns and residents, was on safe handoff principles. The second part (15 minutes), for residents only, reviewed strategies for giving feedback about handoffs. These sessions were grounded in handoff principles promoted by the Joint Commission,⁸ practiced in high reliability industries,⁹ and endorsed by clinical consensus groups.¹⁰ Elements included case studies describing handoff errors, an overview of reasons to improve handoffs, essential elements of verbal and electronic handoffs, and examples of our institution's electronic handoff tool highlighting best practices. Video clips and print examples of excellent, mediocre, and poor verbal and electronic handoffs were shared to generate discussion. During resident feedback training, a handoff video was shown and a resident practiced giving feedback to the faculty, followed by a group debriefing. Residents received a 12-item reference card listing safe handoff elements. Residents were asked to observe and provide their interns with feedback on 2 handoffs within the following 2 weeks. Teaching materials are available upon request.

FIGURE.

Study Design

During weeks 2 and 3, research assistants audiotaped verbal handoffs and collected copies of corresponding electronic handoff documents for 2 handoffs per intern. Handoff communication failures were assessed by the study investigators (blinded to study group assignment) who interviewed the handoff receiver by phone the postcall morning with a questionnaire designed by the investigators.

Handoff Evaluation Tool

We developed a handoff evaluation tool (HET) with content validity based on best handoff practices^8,10 (provided as online supplemental material). The questionnaire included open-ended and direct yes/no questions that probed for issues directly related to the previous night's handoffs. Questions focused on overall quality, areas to improve, errors or near-miss events, anticipatory guidance, and absent or incomplete handoff information. Parts 1 and 2 assessed overall handoff quality; part 3 assessed handoff quality at the individual patient level. Part 1, completed by a research assistant when the handoffs were audiotaped, included 4 items about the handoff environment. Parts 2 and 3, completed by 1 of 3 study investigators blinded to study group assignment, were scored by listening to the audio-recorded handoff while reviewing the electronic handoff. These investigators developed coding rules and reviewed 7 HETs (interrater agreement, 94%). Each grader then scored one-third of the total HETs. Graders did not teach the curriculum and were unaware of group assignments.

Assessment

All interns completed a preintervention survey, and intervention interns also completed a postintervention survey self-assessing their handoff quality and interest in developing handoff skills. One question assessed knowledge and asked for 3 important handoff elements. The postintervention survey had 2 additional items assessing the utility of the handoff lecture and resident feedback.

The HET was scored as follows: skills performed appropriately (“yes”) received 1 point; skills performed mostly correct (“yes partially”) received 0.5 points; skills performed mostly incorrectly (“no”) received 0 points. Each intern received an overall HET score (all measured skills), a composite electronic score (electronic skills only), a composite verbal score (verbal skills only), and a score for each individual skill. Handoff environment skills were scored individually. An analysis of covariance looked at handoff quality while controlling for patient length of stay and illness severity. A t test measured the difference in the verbal handoff length (minutes) between intervention and control groups. Communication failures reported during postcall interviews were calculated numerically, and χ² analysis determined differences between groups. The survey knowledge question was scored 0 to 3 depending on how many handoff elements were identified. Paired t tests were used to analyze differences in the pre-post intervention surveys.

Results

All interns (intervention  =  24; control  =  20) agreed to participate. All intervention interns attended the didactic session. All intervention and 14 control interns (70%) completed at least 1 audio-recorded handoff. Overall, 48 handoff sessions were evaluated for intervention interns and 24 for control interns (318 and 152 patients, respectively). Forty-four of 45 interviews (97.8%) with postcall intervention interns and 35 of 39 interviews (89.7%) with postcall control interns were completed.

The table summarizes differences in handoff quality. The composite verbal handoff quality (79.6% versus 69.1%; P < .001) and the overall handoff quality (verbal and electronic) (82.6% versus 75.6%; P  =  .007) were significantly higher in the intervention than the control group. There were no differences in the composite electronic handoff score or environmental handoff skills. For individual verbal handoff skills, intervention interns performed better communicating relevant baseline physical examination findings (60.7% versus 41.6%, P  =  .04) and reviewing important medications (77.4% versus 58.7%, P  =  .01) (table). Differences in overall and composite scores were not significant after controlling for patient length of stay or illness severity. There were no differences in verbal handoff length between groups.

TABLE.

Comparison of Handoff Scores Between Intervention and Control Groups: Percentage Positive Rating Scores, Effect Sizes^a

Compared with the control group, intervention postcall interns reported significantly fewer communication failures related to patient code status (4% versus 22.3%) (χ²  =  17.4, P ≤ .001) and overnight tasks (2.4% versus 8.8%) (χ²  =  4.51, P  =  .03). All intervention interns completed the presurvey and postsurvey. Interns reported greater satisfaction with their verbal handoff postintervention (x̄  =  4.13) compared to preintervention (x̄  =  3.87), (t  =  −2.794, P ≤ .01). Knowledge of handoff elements significantly improved after the curriculum (2.23 versus 2.60, t  =  2.36, P  =  .02).

Discussion

Few tools to assess handoff quality exist.¹¹ The HET was developed with best handoff practices from the literature supporting the content validity of its assessments. With the exception of 1 study, handoff education studies have measured subjective curriculum effectiveness (ie, trainee comfort; self-assessed skills).^11–13 Using a randomized design, our pilot suggests that a handoff skills educational intervention can improve the quality of intern's verbal handoff skills and reduce the frequency of certain communication failures. Improved intervention group performance begins to provide support for consequence validity of HET assessments.

The ultimate goal of handoff education is to eliminate adverse events during transitions of care. Since such events are relatively infrequent over short time periods, some studies use handoff communication failures as an outcome measurement.^14,15 We found more communication lapses related to lack of clarity of cross-cover tasks and patient code status among control interns, suggesting that handoff educational interventions may reduce some handoff communication failures that contribute to preventable adverse events.

Electronic handoff quality and environmental handoff behaviors did not improve with our educational intervention. We do not believe that this was due to an ineffective curriculum but rather a result of the shared ownership of the electronic handoff between interns and residents and the effects of culture. Although interns are expected to assume responsibility of the electronic handoff in January, immediately after the initiation of our study, we could not control for residents continuing to manage the document. We believe that interns may not have felt comfortable changing the electronic document coauthored by their resident owing to team hierarchy. Attitudes and behaviors related to previously learned handoff practices are deeply ingrained within residency programs. This may explain why, despite our educational intervention, interns and residents failed to change their electronic handoff behaviors and the setting and style of their handoffs. Improvements in electronic handoff skills have been reported by Gakhar and Spencer.⁷ However, their study used the presence or absence of handoff components as an outcome variable, while ours assessed the accuracy and usefulness of the electronic handoff information. Additionally, their study implemented an electronic handoff tool as part of the intervention, and electronic handoff improvements may have been due to system changes, not the educational intervention. Improving electronic handoff quality may prove harder than verbal quality owing to the challenge of assessing electronic medical record documentation created by multiple authors. However, since physicians rely on handoff documents for clinical decision making, more research is needed to find ways to improve the electronic handoffs.

Our study has several limitations. Our outcome was assessed soon after our intervention and only a single measurement was performed. Our study was conducted in the middle rather than the beginning of the academic year, potentially reducing the intervention's impact. The pretest questionnaire may have cued the intervention group to the learning objectives and enhanced learning. We did not assess sustainability of handoff improvements. Fewer control group interns completed the study because they did not wait for the research assistant to audio-record their verbal handoff. The control group may have been less aware of the study's procedures, resulting in decreased participation. Interns in the intervention arm knew they were being audio recorded, which could have improved their performance. This Hawthorne effect would be expected in both groups. Our assessment of handoff communication failures through morning phone interviews may have been subject to recall bias. We do not know if improved handoff measurements were related to intern education, resident feedback, or both.

Conclusion

We found that a brief handoff educational session improved interns' verbal handoff skills when compared with no handoff teaching. Electronic handoff documentation skills did not improve with the intervention, which may reflect greater difficulty in changing physicians' electronic documentation habits. The HET was developed as an objective measure of verbal and electronic handoff quality and may prove useful with additional study.

Our intervention is simple enough for other programs to incorporate into their curriculum without additional resources. We have sustained our curriculum for 3 years by including it in our intern orientation. With minimal training, several residency program faculty members now teach the curriculum and facilitate small group handoff simulation sessions immediately after the teaching session.