Development and Implementation of an Oral Sign-out Skills Curriculum

Abstract

Introduction

Imperfect sign-out of patient information between providers has been shown to contribute to medical error, but there are no standardized curricula to teach sign-out skills. At our institution, we identified several deficiencies in skills and a lack of any existing training.

Aim

To develop a sign-out curriculum for medical house staff. Setting: Internal medicine residency program.

Program description

We developed a 1-h curriculum and implemented it in August of 2006 at three hospital sites. Teaching strategies included facilitated discussion, modeling, and observed individual practice with feedback. We emphasized interactive communication, a structured sign-out format summarized by an easy-to-remember mnemonic (“SIGNOUT”), consistent inclusion of key content items such as anticipatory guidance, and use of concrete language.

Program evaluation

We received 34 evaluations. The mean score for the course was 4.44 ± 0.61 on a 1–5 scale. Perceived usefulness of the structured oral communication format was 4.46 ± 0.78. Participants rated their comfort with providing oral sign-out significantly higher after the session than before (3.27 ± 1.0 before vs. 3.94 ± 0.90 after; p < .001).

Discussion

We developed an oral sign-out curriculum that was brief, structured, and well received by participants. Further study is necessary to determine the long-term impact of the curriculum.

INTRODUCTION

Communication failures figure in 25–67% of adverse events.^1–5 Transfers of care in hospitalized patients are particularly vulnerable to communication failures during the sign-out of clinical information between providers.^6–11 Consequently, the Joint Commission on Accreditation of Healthcare Organizations now mandates a standardized approach to hand-offs of patient care.¹² In addition, the Accreditation Council on Graduate Medical Education requires that residents be proficient in communication skills that “result in effective information exchange and teaming with patients, their patients’ families, and professional associates.”¹³

At present, however, residency sign-out systems are not standardized, and skills training is rare.^14,15 Furthermore, there are no publicly available or published curricula for improving physicians’ sign-out skills, and there are no universally accepted standards for hand-offs of patient care among physicians, though some groups have recently issued recommendations for best practices.^16–18

To address these gaps, we developed a brief, structured sign-out curriculum for Internal Medicine house staff. In this paper we describe the curriculum development, instructional strategies, and initial implementation experience and present our plans for further expansion and evaluation of the curriculum.

PROGRAM DEVELOPMENT AND DESCRIPTION

Needs Assessment

We followed a six-step model in developing our curriculum.¹⁹ First, we conducted an in-depth local needs assessment. We informally surveyed house staff and discovered that none had had any training in sign-out skills during medical school. We also examined our internal practices and found that there were no institutional standards as to how and when hand-offs were to take place, no requirements as to who participated in the hand-off, and no consistent method of conveying information orally or in written form. In addition, although ward attendings were urged to evaluate residents’ sign-out skills, they were given no formal guidance or evaluative criteria.

To better understand existing practices, we audiotaped sign-outs and interviewed post-call and night float house staff about the previous days’ sign-outs. From these direct observations, we concluded that the quality of written sign-out was dependent largely on the technological support available to residents, whereas the quality of verbal sign-outs was more variable and appeared driven primarily by the skill set of individual participants. We therefore decided to focus our curriculum on oral sign-out skills and to direct efforts to improve written sign-out at systems issues, such as developing standardized templates.

Literature Review

First, we attempted to identify extant sign-out curricula by searching MEDLINE and numerous curricular web sites.^20–27 While we identified an extensive literature on physician–patient communication, we were unable to locate any curriculum specifically for teaching physician–physician communication skills. We therefore determined it would be necessary to develop our own.

Next we conducted an additional literature search, seeking to identify explicit processes that improve hand-offs. We reviewed effectiveness studies and recommendations from recognized authorities.^{16–18,28–36} We also included articles discussing hand-offs in other high-reliability fields.^37–41 Several studies suggested that a quiet setting, structured format, and opportunity for interaction represent key elements of successful sign-outs.^{17,18,28,42,43} From our direct observations we also noted the importance of concrete language, description of the clinical condition of the patients, anticipatory guidance, and clear plans and rationales for all assigned tasks. These elements became highlights of our curriculum.

Curriculum Description

Although we considered incorporating the curriculum into the intern orientation before the start of residency, we rejected this idea because orientation was already very full and we felt the curriculum would be most useful to interns who had already had some experience with sign-out. We decided on a noon conference format, with identical sessions to take place in the summer at all three of the hospitals through which our interns rotate. This allowed us to reach interns early enough for the workshop to be useful but well enough into internship for them to have had some practical experience with sign-out and cross coverage. We also designed a pocket card, placed a structured oral sign-out format on the residency web site, and added a section on sign-out to the “Intern Survival Guide” each intern receives on entering the program.

The goals of the curriculum were to sensitize interns to the concept of sign-out as an important contributor to patient safety and to improve their skills in providing sign-out. To achieve these goals we developed several specific curricular objectives, covering cognitive, affective, and psychomotor domains (Table 1). Based on these objectives, we determined that a combination of instructional strategies would be necessary and, therefore, structured the workshop in two parts.

Table 1.

Curriculum Objectives

	Objective
Cognitive (knowledge)	By the end of the curriculum, ≥80% of the interns will be able to list three purposes of the oral sign-out and three purposes of the written sign-out.
Affective (attitude)	By the end of the curriculum, ≥80% of the interns will rate sign-out as an important component of patient care (4 or 5 on a 5 point Likert scale).
	By the end of the curriculum, ≥80% of the interns will describe increased comfort with providing sign-out (≥1 point improvement on a 5 point Likert scale).
Psychomotor (skill or competence)	By the end of the curriculum, each intern will have presented a simulated sign-out on a real patient that follows the SIGNOUT format, particularly including clinical condition, and plans and rationales for each instruction or anticipatory event.
	By the end of the curriculum, each intern will have received a simulated sign-out during which he/she was attentive, took notes, and interacted with the provider. These interactions may include a request for clarification, request for more information, explicit acceptance of responsibility for the patient, or read-back of an assigned task.
Psychomotor (behavior or performance)	By the end of the intern year, each intern will have been observed by a supervising resident at least once to have presented sign-out in a quiet setting, using concrete language, and following the SIGNOUT format for ≥80% of the patients on the list.

First, we designed a 30-min interactive large-group discussion facilitated by a study investigator (L.I.H.). We covered five topics: the importance of sign-out and consequences of poor sign-out; the ideal content of written and oral sign-outs; the differences between oral and written sign-out aims and techniques; the SBAR (situation, background, assessment, recommendation) format for communication about emergent situations;⁴⁴ and a new structured form of oral sign-out.

Our insistence on a formal approach to oral sign-out represents the most novel part of the curriculum. There is no uniform standard for sign-out format akin to that of the history and physical, making it difficult for interns to consistently include all desirable content. We developed a standardized format modeled on SBAR that includes all the essential elements of an oral sign-out in an easy-to-remember order: SIGNOUT (Table 2). The facilitator demonstrated the use of this format by modeling a sign-out for the large group.

Table 2.

SIGNOUT Format for Oral Communication

Mnemonic		Sample sign-out
S	Sick or DNR? (highlight sick or unstable patients, identify DNR/DNI patients)	OK, this is our sickest patient, and he’s full code.
I	Identifying data (name, age, gender, diagnosis)	Mr. Jones is a 77-year-old gentleman with a right middle lobe pneumonia.
G	General hospital course	He came in a week ago hypoxic and hypotensive but improved rapidly with IV levofloxacin.
N	New events of day	Today he developed a temp of 39.5°C and his white count went from 8 to 14. We got a portable chest x-ray, which was improved from admission, took out his Foley and sent blood and urine cultures. U/A was negative but his IV site looked a little red so we started vancomycin.
O	Overall health status/clinical condition	Right now he is satting 98% on 2 l nasal cannula and is afebrile.
U	Upcoming possibilities with plan, rationale	If he becomes persistently febrile or starts to drops his pressures start normal saline at 125 cc/h and have a low threshold for calling the ICU to take a look at him because of concern for sepsis.
T	Tasks to complete overnight with plan, rationale	I’d like you to look in on him around midnight and make sure his vitals and exam are unchanged. I don’t expect any blood culture results back tonight so there is no need to follow those up.
?	Any questions?	Any questions?

In the second half of the workshop senior residents were dismissed to provide a low-key atmosphere for the remaining interns and students. Remaining participants were divided into small groups, with each facilitated by a study investigator (L.I.H.) or a chief resident. Each participant took it in turn to sign out the first patient on his own patient list to the person on his right while the remainder of the group listened. Immediately following, the small group offered feedback for both the provider and receiver of the sign-out, including discussion of structure, clarity, information that should have been included or omitted, and additional questions that could have been asked. The next intern then signed out, until each intern had had the opportunity both to present and receive an oral sign-out. Learners thus each had the opportunity for real-time practice, evaluation, and feedback as an integral part of the seminar. The small group part of the session lasted about 20 min.

Evaluation

At the conclusion of the workshop, participants were asked to complete an evaluation form, which assessed satisfaction with the course, attitude towards sign-out importance, and retrospective pre–post comfort with providing sign-out. The simultaneous self-assessment of pre and post skills at the conclusion of the workshop avoids response-shift bias, in which participants’ internal standard changes as the result of the educational intervention itself.⁴⁵ This method has been shown to be more consistent with independent skills assessments than prospective pre to post self-evaluation.⁴⁶ Pre–post comfort levels were compared with a paired t test using SAS 9.1.2 (SAS Institute, Cary, NC, USA).

The Human Investigation Committee at Yale-New Haven Hospital and the Institutional Review Board at the West Haven VA Hospital reviewed and approved the project and granted an exemption of signed informed consent to maintain anonymity of participants.

PROGRAM IMPLEMENTATION AND ASSESSMENT

We conducted an hour-long sign-out session in August at each of the three hospitals through which interns rotate. Thirty-four evaluations were completed, 14 by interns, 14 by students, and 6 by others. We did not take attendance, but we received evaluations from 92% of interns who could have been at the conference at two sites. At the third site, attendance was very poor and we received only two intern evaluations (15% of eligible). Because evaluation forms were distributed at the conclusion of the workshop, we received very few evaluations from residents, who left partway through the session.

The overall mean score given to the course on a 1-to-5 scale, where 5 is best, was 4.44 ± 0.61. Mean score for the large group session was 4.35 ± 0.73; for the small group session it was 4.55 ± 0.71. The course was rated similarly by interns and students (4.50 ± 0.52 vs. 4.33 ± 0.72; p = 0.49). All participants rated sign-out as important or very important to patient care (mean score 4.88 ± 0.33) and the SIGNOUT mnemonic as likely to be useful or very useful (mean score 4.46 ± 0.78). The SBAR format rated a lower mean score of 4.18 ± 0.83. Perceived comfort at providing sign-out increased significantly (3.27 ± 1.0 before vs. 3.94 ± 0.90 after; p < 0.001). Interns as a group reported higher pre and post scores than students, but their improvement in subjective comfort with sign-out was still significant (3.57 ± 0.65 vs. 4.21 ± 0.58; p = 0.002).

During the small group session, interns tended to ramble rather than keep to a concise, structured format, but they recognized the deficiency. While they made efforts to provide anticipatory guidance, they did not always recognize when it was needed. In addition, despite repeated feedback after each subsequent sign-out, it was challenging to get interns to read back instructions, as has been recommended by JCAHO. This is a completely novel form of communication for most house staff, and their discomfort was evident.

DISCUSSION

We developed a brief, formal curriculum to teach sign-out skills to interns that incorporated a variety of instructional strategies, including discussion, modeling, individual practice, evaluation, and feedback. Overall satisfaction with the course was high and participants reported improved comfort with sign-out skills at the conclusion of the workshop.

The most common suggestion for improving the curriculum was to offer it even earlier in the academic year. One alternative would be to present the curriculum at the June intern orientation. However, this would make real-time practice with actual patients impossible. In addition, interns who have not yet done any night-time cross-coverage might have less appreciation for the types of problems commonly encountered overnight that can be avoided with good sign-out. In fact, we found that interns had difficulty with the anticipatory guidance part of sign-out, although they had already had some experience on the wards.

We have several plans for further expansion and evaluation of the curriculum. We are now developing a sign-out evaluation tool so that residents can provide evaluation and feedback for their interns throughout the year. This tool will include areas of deficiency noted during the small group sessions, such as structured format, anticipatory guidance, and read-back. In addition, we plan to repeat our direct observation of house staff sign-outs to determine the curriculum’s effect on actual sign-out practices. Because of the inherent steep learning curve of the intern year, we cannot compare practices before and after the curriculum in the intervention group itself. Rather, we will observe practices in the intervention group in the spring and compare them to those of a previous cohort of interns who did not receive any sign-out education or evaluation. Finally, based on feedback from medical student participants, we are exploring the possibility of developing a similar curriculum for the medical school.

In the next iteration, we will also address two challenges that we encountered during the first implementation. First, several participants were on outpatient or elective rotations and were disappointed not to have the opportunity to practice giving sign-out. We have developed written sign-outs for use by participants who do not have their own patients so that all attendees can participate in the small group session. Second, despite presenting the curriculum at each of the hospitals through which our house staff rotate, we only reached approximately a quarter of all interns because those who were post-call or on vacation or elective and intensive care rotations were not available, and the new ward interns had difficulty making time to attend educational conferences. Next time, therefore, we will offer the workshop at each participating hospital during two different rotations.

In summary, we developed a formal sign-out skills curriculum incorporating a variety of teaching techniques; a novel structured sign-out format; and real-time practice, evaluation, and feedback for all participants. We intend this to be one step in improving residents’ communication competence and reducing medical errors from poor sign-out.