Participation and Experience of Third-Year Medical Students in Handoffs: Time to Sign Out?

ABSTRACT

BACKGROUND

Although interns are expected to be competent in handoff communication, it is currently unclear what level of exposure, participation, and comfort medical students have with handoffs prior to graduation.

OBJECTIVE

The aim of this study is to characterize passive and active involvement of third-year medical students in the major components of the handoff process.

DESIGN

An anonymous voluntary retrospective cross-sectional survey administered in 2010.

PARTICIPANTS

Rising fourth-year students at two large urban private medical schools.

MAIN MEASURES

Participation and confidence in active and passive behaviors related to written signout and verbal handoffs during participants’ third-year clerkships.

KEY RESULTS

Seventy percent of students (n = 204) responded. As third-year medical students, they reported frequent participation in handoffs, such as updating a written signout for a previously admitted patient (58 %). Students who reported frequent participation (at least weekly) in handoff tasks were more likely to report being confident in that task (e.g., giving verbal handoff 62 % vs. 19 %, p < 0.001). Students at one site that did not have a handoff policy for medical students reported greater participation, more confidence, and less desire for training. Nearly all students believed they had witnessed an error in written signout (98 %) and almost two-thirds witnessed an error due to verbal handoffs (64 %).

CONCLUSIONS

During their third year, many medical students are participating in handoffs, although reported rates differ across training environments. Medical schools should consider the appropriate level of competence for medical student participation in handoffs, and implement corresponding curricula and assessment tools to ensure that medical students are able to effectively conduct handoffs.

Electronic supplementary material

The online version of this article (doi:10.1007/s11606-012-2297-9) contains supplementary material, which is available to authorized users.

INTRODUCTION

There is an unprecedented focus on handoffs among medical trainees. In addition to recent residency duty hour restrictions that have increased the frequency of handoffs, the Accreditation Council of Graduate Medical Education now requires that all residents be “competent in handoff communications.”¹ While many residency programs are implementing handoff education and evaluation programs, it is unclear whether and how to prepare medical students to be competent in handoff communications. In a large national survey in 2010, the skills that internal medicine program directors most highly desired for their incoming interns were being able to provide a prioritized and organized written signout (89 %) and execute an organized verbal handoff (84 %).² This highlights the importance of understanding medical student exposure and participation in handoffs. While most student curricular efforts around handoffs focus on the 4th year sub-internship, third-year medical students may observe and participate in handoffs during their required clerkships.^3–6 Because no empirical data exist concerning third-year medical student handoff experiences, this study, at two large US medical schools, aims to characterize involvement of third-year medical students in the major components of handoffs, namely the written signout and verbal handoff, during core clerkships including the required internal medicine clerkship.

METHODS

We created an anonymous survey (See Online appendix) to characterize third-year medical student involvement in handoffs. The survey was reviewed for clarity and content validity by two faculty (one at an outside and one at a participating institution), one educator in charge of the survey center at one of the institutions, and two fourth year students at one of the participating institutions. The survey included domains that referred to both (1) “written signout," the written document to transfer information, ” and (2) “verbal handoff,” the dialogue that takes place at the time of shift change.^7,8 The survey queried medical students about both passive observations of written signouts and active involvement (assisting in creating written signout, updating written signout) in both written signout and verbal handoffs (see Appendix). We focused on active behaviors, such as creating or updating the signout, in part because students anecdotally reported using written signouts to follow the patients on the service without participating in the creation of the signout. To characterize the frequency with which these behaviors occurred, a scale was designed that reflected the anecdotal experience of two of our authors (ME, EB). Specifically, frequency of task was rated using a 5-point scale with the following verbal anchors: (5) Always (nearly daily); (4) Often (at least weekly); (3) Sometimes (roughly once a month); (2) Rarely (less than five times a year); (1) Never. Students reported the required core clerkships (internal medicine, surgery, pediatrics, ob/gyn, family medicine, neurology, psychiatry) on which they had the most exposure to the handoff tasks. Students were asked about their confidence in these skills using a Likert-type scale ranging from 1 (Not at all confident) to 5 (Very confident), if they had received any training, what type of training they received, and if they desired more training. While not a primary aim, we also asked students whether they witnessed errors in the handoff process.

The voluntary, non-incentivized survey was administered to students who had just completed their third year at two large urban medical schools. At one school (A), students were surveyed on paper at a sub-internship orientation. At the other school (B), students voluntarily completed an electronic survey (SurveyMonkey™, Palo Alto, CA) during their required third-year clinical skills assessment at the end of their third year. While both schools lacked robust training in handoffs in the third-year of medical school, there were site differences in the role of third-years and the mechanism of the handoff process. At school A, appropriate use of signouts was discussed briefly (5 min) at the orientation for third-year students, and students received a memo stating they should defer creation and update of written signouts to their residents for patient safety reasons, and had “read-only” access to signouts until fourth year, when they would also have “write access.” School B had a thirty-minute case-based handoffs small group session during April of the third year, within 4 weeks prior to the students’ completion of the survey. While there was no policy in place regarding the involvement of third or fourth year medical students in the handoff process, at some clinical sites, students could create and update written signouts. At both schools, more extensive handoff training occurred significantly after completion of this survey, as part of either sub-internship clerkship or capstone experiences.

Data were merged into a Microsoft Excel spreadsheet with a numeric code denoting school. After examining item responses, responses regarding frequency of participation and observation of error were grouped into three categories by defining the two extremes of participation. “Frequent” participation in a handoff task was defined as a response of 4 (Often, once a week) or 5 (Always, nearly daily). Likewise, “infrequent” participation was defined as a response of 2 (Rarely, less than five times a year) and 1 (Never). Self-reported “confidence” in a handoff task was defined as either: (4) Confident or (5) Very confident. Descriptive statistics were used to summarize the frequency of medical student involvement in active and passive behaviors related to written and verbal signouts and the frequency of witnessed errors. Unadjusted analyses were performed using two sample tests of proportion. In addition, site-adjusted logistic regression was used to assess whether those students who reported frequent participation in a specific handoff task also had greater self-reported confidence in that same handoff task. Similar analyses were performed examining the relationship between: (1) frequent participation in a handoff task and desire for training; and (2) the self-reported confidence in a handoff task and the desire for training. This study received institutional review board (IRB) exemption at both institutions.

RESULTS

Participation in Signouts and Handoffs

Seventy percent (145/206) of medical students at both schools responded, with a higher response rate at the school that used a paper survey at a required orientation (80 % vs 65 %, p = 0.03). Student participation in the handoff process was highest on the medicine clerkships (81 %, Fig. 1).

Figure 1.

Participation in handoffs by third-year medical students by clerkship.

Student participation in passive behaviors related to written signout and verbal handoffs was high. For example, 92 % of students reported using written signout to follow patients on the team at least weekly, and 77 % of students observed the creation of written signouts at least weekly (Table 1). Likewise, 83 % of medical students reported observing verbal handoffs at least weekly. Active participation in the written signout by students was less common. Nearly 60 % of medical students reported updating the written signout for a previously admitted patient at least once a week. Fewer medical students (39 %) reported assisting in creation of the written signout, giving (26 %) verbal handoffs, or receiving (21 %) verbal handoffs.

Table 1.

Participation in Handoff Task (%)

Task	Infrequent; < 5 times/year	Sometimes; ∼ 2 times/month	Frequent; At least weekly
Written Signout
Observed creation	13 (9)	21 (14)	101 (77)
Used to follow patients	5 (3)	7 (5)	133 (92)
Assisted in the creation for a newly admitted patient	40 (28)	48 (33)	57 (39)
Assisted in updating for patients remaining in the hospital	18 (12)	43 (30)	84 (58)
Verbal Handoff
Observed	7 (5)	18 (12)	120 (83)
Gave verbal handoff*	75 (52)	32 (22)	37 (26)
Received verbal handoff	91 (63)	23 (16)	31 (21)

^*For “gave verbal handoff”, n = 144 due to one missing response

Confidence in Performing Signouts and Handoffs

In site-adjusted logistic regression models, students who reported frequent participation in an active handoff behavior reported greater confidence in that behavior (Fig. 2). For example, students who reported frequent participation in creating a written signout for a new patient were more likely to report feeling confident in that task, compared to those with infrequent participation (70 % vs. 24 %, p < 0.0001) (Fig. 2).

Figure 2.

Association between participation in handoffs by third-year medical students and self-reported confidence.

Training in Performing Signouts and Handoffs

Despite frequent student participation in handoffs, few students reported receiving training on preparing written signouts (30 %) or conducting verbal handoffs (23 %). Those that did report training typically qualified their answers with explanations, such as “on the job training” or “learned from a resident”. There was no association between receipt of training and reported confidence in handoff behaviors.

School Differences

Compared to school A, at school B (without a policy regarding student participation in handoffs) students reported greater participation in active handoff behaviors, such as creating a signout for a new patient (49 % vs. 27 %, p = 0.004), updating a written signout on a previously admitted patient (62 % vs. 53 %, p = 0.03), and giving verbal handoff (43 % vs. 3 %, p < 0.001). At school B, students also reported greater confidence in two handoff tasks: (1) creating a written signout for a new patient (52 % vs. 30 %, p = 0.007); and (2) giving a verbal handoff (40 % vs. 19 %, p = 0.007). Lastly, students at school B expressed less desire for training for all active handoff behaviors: creating a written signout (41 % vs. 75 %, p < 0.001), updating a written signout (35 % vs. 64 %, p = 0.001), giving a verbal handoff (49 % vs. 78 %, p < 0.001), and receiving a verbal handoff (40 % vs. 63 %, p = 0.008).

Students’ Perceptions of Errors in Signouts and Handoffs

Almost all students reported witnessing an error in the written signout (98 %), and almost two-thirds of students reported witnessing an error due to poor verbal handoff (64 %) at least once during their third year. Compared to verbal handoffs, errors in written signouts were more frequently observed (at least once a week) (30 % vs. 5 %, p < 0.0001).

DISCUSSION

While third-year medical students have not traditionally been the focus of handoff training, they report frequent participation in active handoffs behaviors such as updating a written signout. Frequent participation in handoff behaviors was associated with students reporting confidence in that behavior. However, few students reported formal training. Observed site differences in participation, confidence, and desire for training likely related to differences in medical school policy and practice. Lastly, all students believed they had witnessed a handoff error, highlighting the need for future study of handoff errors students are reporting.

This study underscores the need to define the appropriate level of participation and corresponding assessment and training necessary for medical students. Program directors desire residents to be competent in handoffs prior to residency training.² Using a milestones-based approach such as the “Entrustable professional activities (EPA)” currently emphasized in residency training is one potential useful framework for designing a medical school curriculum.^9,10 It may be reasonable to suggest that medical students, at the end of third-year, should be competent at certain handoff tasks, such as how to create a written signout. In contrast, the ability to verbally deliver a handoff requires higher order synthesis skills to formulate clinical judgment and to anticipate future events. Therefore, providing and receiving verbal handoffs is a skill unlikely to be mastered in the third year, and may be more appropriate for fourth year medical student and resident education. As an “EPA”, achieving competency would require incremental implementation of handoff education and assessment over the third and fourth years of medical school, having students first recognize the importance and principles of safe and effective handoffs, followed by practicing written signouts before performing supervised oral handoffs. One mechanism to achieve widespread adoption of training in medical schools and ensure competence of medical school graduates in core handoff skills would be to incorporate this skill into the United States Medical Licensing Examination (USMLE) Clinical Skills Examination, although the optimal means of integration of this training into medical school curricula is unknown.

Student participation in handoffs could reflect students’ intrinsic desire to learn these skills or their recognition of their importance in ensuring patient safety and continuity of care. Medical students may also face unclear expectations or an extrinsic expectation to perform these skills by residents and attendings. Because of this, it is critical that clerkship directors and medical school leaders set expectations and create curricula and assessment tools to train third and fourth year students in handoffs. This is especially true for internal medicine clerkship, which had the highest rate of participation in handoffs reported by students. Such curricula could borrow from the various methods that have been described in previous literature, such as a “handoff selective” for medical students,¹³ simulation,^4,14 or experiential learning.¹⁵ Medical schools should also develop and disseminate policies that explicitly outline the appropriate level of participation in handoffs for medical students. However, as evidenced by school A’s results, policy alone will not be effective without a supportive institutional culture that carries the same message to residents and faculty supervisors. Local policies need to take into account institutional practice to avoid discordant messages between policy and practice.

Although handoff errors were nearly universally reported by students, it is unclear how accurate error identification is among untrained students. Because students report errors in written signout, more robust systems and training focused on error identification are warranted. One prior study at one of our sites confirmed a high rate of medication errors in written signouts, which does correlate with students’ reports of written signout errors.¹¹ “On-the-job” learning may be insufficient for students and may promote error-prone practices, reinforcing the need for formal curricula and assessment of these skills. This is especially important given the potential for a false sense of confidence associated with on-the-job participation by students. Since the survey did not ask any additional information regarding the specifics of the errors, the nature of the errors and if any harm to patients resulted is unclear. Efforts to train medical students as safety champions to identify handoff errors, as well as to provide more detailed characterization of such errors, may be helpful.¹²

There are several limitations to this study. First, this study was performed at two large urban private medical schools, limiting generalizability of these findings. Because school differences were observed, local institutional culture, practice and policy are likely important mediators of student exposure to the handoff process. Medical schools should investigate the actual roles of their own third-year medical students during the handoff process. The survey was administered differently at both sites, which may have contributed to some of the differences noted. This study is retrospective, relying on student self-report. While the Association of American Medical Colleges uses this method routinely in large-scale surveys of medical students, our results are subject to recall bias regarding the volume of handoffs and errors witnessed, especially for rotations in the distant past. We also have limited information by rotation. Because we did not link self-reported confidence to actual practice, we are limited in our ability to draw conclusions of what ‘confidence’ in signout ability means. Students may be reluctant to report weaknesses in their skills, resulting in an overestimation of their confidence. In contrast, students may be unwilling to report an error made by their team, resulting in under-reporting of errors. As discussed, students’ definition of an ‘error’ may vary widely in magnitude and scope, highlighting the need for future work in this area.

CONCLUSION

In an era of increasing focus on patient handoffs, this study shows that third-year medical students at two different schools actively participate in handoffs, despite little or no training. To prepare future physicians for internship, in which they are expected to be ready to perform handoffs, medical schools should consider appropriate levels of competence of handoff skills for medical students and the curricula and assessment tools needed to ensure this level of competence.