Sign-out snapshot: cross-sectional evaluation of written sign-outs among specialties

Abstract

Background

Sign-out is the process (written, verbal, or both) by which one clinical team transmits information about patients to another team. Poor quality sign-outs are associated with adverse events and delayed treatment. How different specialties approach written sign-outs is unknown.

Objective

To compare written sign-out practices across specialties and to determine consistency of content, format, and timeliness.

Methods

The authors evaluated all non-Intensive Care Unit written sign-outs from five inpatient specialties on January 18, 2012, at Yale-New Haven Hospital, focusing on content elements, format style, and whether the sign-outs had been updated within 24 hours. In our institution, all specialties used a single standardized sign-out template, which was built into the electronic medical record.

Results

The final cohort included 457 sign-outs: 313 medicine, 64 general surgery, 36 pediatrics, 30 obstetrics, and 14 gynecology. Though nearly all sign-outs (96%) had been updated within 24 hours, they frequently lacked key information. Hospital course prevalence ranged from 57% (gynecology) to 100% (pediatrics) (p<0.001). Clinical condition prevalence ranged from 34% (surgery) to 72% (pediatrics) (p=0.005).

Conclusion

Specialties have varied sign-out practices, and thus structured templates alone do not guarantee inclusion of critical content. Sign-outs across specialties often lacked complex clinical information such clinical condition, anticipatory guidance, and overnight tasks.

INTRODUCTION

Handoffs occur during a change in responsibility for a patient from one care provider or team to another.¹ Patient handoffs have become increasingly central as duty restrictions have led to increased transfers of care.² Clear and concise interactions between physicians during handoffs are important for ensuring patient safety, as poor communication often contributes to medical errors.³ Sign-out is the process (either written, verbal, or both) by which information is transmitted about patients from one medical team to another.¹ Failure to communicate relevant information effectively during a sign-out can lead to mistakes or delays in medical decision-making that ultimately threaten patient safety.⁴ Poor quality sign-outs have been associated with adverse events, near miss events, delayed treatment and diagnosis, and inefficient use of time.⁵

Despite the importance of sign-outs for patient care and safety, there appears to be much variation and little standardization of these sign-outs, even within a specialty at a single institution.⁶ Previously, we found that only 38% of written sign-outs among our internal medicine residents contained information on the patient’s current clinical condition (including “symptoms, vital signs, physical exam, laboratory/procedure results, or clinical stability”), and only 30% of written sign-outs included information from all three categories of current clinical condition, hospital course, and tasks to complete.⁷

Though researchers have examined written sign-out approaches within some specialties,^6-11 we are not aware of any studies that have compared how sign-outs vary across specialties. Standardization of sign-out across specialties may offer the opportunity to improve sign-out quality, strengthen communication between care providers, and reduce the potential for error.¹² Many articles have advocated for standardization as an improvement strategy for sign-outs.^{4, 6, 13-16} Furthermore, it is likely that there is a core set of sign-out information needs that spans specialties. Though it may be important to allow structured flexibility, since specialties likely approach sign-out in unique ways, it is important to determine the degree to which specialties agree on the fundamentals.

Since 2007, this institution has had a centralized handoff committee composed of clinicians from multiple departments across the hospital. This committee developed a single standardized sign-out template built into the electronic medical record and used by all specialties. The committee also developed handoff curricula for trainees, with common core elements across the institution.¹⁷ This study was designed to determine whether the committee’s efforts resulted in the presence of a common core set of sign-out standards across specialties, or whether specialty-specific cultures overwhelmed the attempt at institutional standardization. Therefore, we conducted a cross-sectional analysis of sign-outs from five inpatient specialties at our institution, assessing content, format and whether the sign-outs had been updated within 24 hours.

METHODS

Setting

Yale-New Haven Hospital (YNHH) is a 966-bed academic medical center located in New Haven, Connecticut. At the time of our study, YNHH used an electronic medical record system called Sunrise Clinical Manager [Allscripts, Chicago, IL]. Sign-outs were embedded within this electronic medical record.¹⁷ We studied sign-outs from medicine, general surgery, pediatrics, obstetrics, and gynecology. We further subdivided medicine into “hospitalist” and “house staff,” as these teams function independently at YNHH.

Sign-out processes at YNHH differ among specialties, but usually contain both verbal and written components. Hospitalists are the exception, as they do not conduct verbal sign-out. Hospitalist teams are comprised of attending physicians, physician’s assistants (PAs), and nurse practitioners (NPs/APRNs). Hospitalist sign-outs are expected to be written initially by the admitting attending. PAs or NPs/APRNs are expected to update the sign-outs each day. Hospitalists are discouraged from writing “nothing to do” if there are no overnight tasks. Internal medicine house staff teams are comprised of attending physicians, residents, interns, and medical students. Residents are expected to write the sign-out initially, but interns or residents can update the sign-out. Verbal sign-out is expected to take place between interns, with residents supervising. On general surgery teams, interns and PAs generally write and update sign-outs. Sign-out usually occurs between an intern and a PA. In pediatrics, interns are expected to write and update sign-outs, though senior residents can update them as well. Interns are expected to lead the verbal sign-out process and residents are expected to supervise. It is expected that sign-outs are updated daily. For both obstetrics and gynecology, interns and residents write and update sign-outs. Residents are expected to present verbal sign-out for gynecology, while both interns and residents are expected to present verbal sign-out for obstetrics, supervised by attendings.

The written sign-out note is embedded within the electronic medical record (EMR) and includes a common structured template for all specialties. Some specialties have additional specialty-specific fields.¹⁷ Some fields (name, location, gender, age, medical record number, visit reason, allergies, weight, admission date, conservator status, care coordination issues, and likelihood of being discharged the following day) are automatically inserted from the EMR, while others (primary medical doctor, history, and “to do” list) are free text, requiring data entry. Pediatric sign-outs automatically also include diet, while surgery sign-outs automatically include selected laboratory results. A patient’s medications are optional to include when printing the sign-outs.

Any clinician with access to the EMR (including medical students, PAs, APRNs, house staff, fellows, and attending physicians) can review and/or modify sign-outs. Once created, a sign-out note remains open for continued editing and revision until the patient is discharged. Consequently, each patient can have only one sign-out note per hospitalization from any given specialty. However, different specialties can create separate sign-out notes for the same patient. In preparation for verbal sign-out, teams often print a sign-out “report” which includes all sign-out notes for their patients.

Study cohort

We obtained all non-Intensive Care Unit (ICU) written sign-outs from five inpatient services at Yale-New Haven Hospital on the evening of Wednesday, January 18, 2012. This date was in the middle of the resident work year, the rotation block, and the week. We chose to study five inpatient specialties- internal medicine, general surgery, pediatrics, obstetrics, and gynecology- that we felt would best represent a diverse group of general clinical services.

Though we included patients in step-down units, we excluded all patients in ICUs because ICUs often have sign-out systems quite distinct from non-ICU services. We excluded well baby nursery patients because pediatric teams generally do not produce written sign-out notes for these patients.

We included in our study any sign-out note created by the same specialty as the service to which the patient was assigned. We did not count sign-out notes written by consulting specialties.

Measures

We assessed sign-out based on content and format, and whether the sign-outs had been updated within the last 24 hours. We defined outcome measures based on existing evaluation tools in the literature, and expert opinion.^{4, 5, 7, 9, 10, 18, 19} We included whether the sign-out had been updated because shift-to-shift sign-out by definition is intended to be focused primarily on short-term upcoming events and because these are often related to the most recent events of the day. Opportunity for sign-out related adverse events is higher when old, irrelevant information is not removed and when new, important information is not added.⁴

For content, we determined the presence or absence of several content items: the primary medical doctor (PMD); diagnosis and/or presenting symptoms; general hospital course (defined as a description of any event occurring prior to sign-out collection date, including medication changes, treatments, and operations); new events (defined as a description of any event occurring on the sign-out collection date, including medication changes, treatments, and operations); clinical condition (defined as either objective data such as vital signs, physical exam findings, laboratory or procedure results; a description of the patient’s stability or trajectory [eg. “improved” or “resolved”]; or a description of the patient’s symptoms); and description of current mental status.

We assessed whether the sign-out contained anticipatory guidance: statements predicting overnight events. We defined these statements as if/then statements or statements qualifying a PRN order. We also examined whether the sign-out contained tasks (assignments for the overnight team), but we excluded the phrase “nothing to do.” We evaluated the text of all anticipatory guidance statements and overnight tasks to calculate frequencies of containing a plan for the predicted event or task, a rationale for that plan, both a plan and rationale, or neither.⁵ For example: “If patient requires transfusion [anticipation], please call blood bank [plan], patient has multiple antibodies [rationale].” To test the reliability of coding, we randomly selected a set of 50 records for reabstraction by a different investigator (L.I.H.), of which 46 had sign-outs available. The kappa score for agreement on presence of anticipatory guidance was 0.87 and for presence of tasks was 0.92.

To examine format, we described the major variants of sign-out layout (ie. whether the sign-out contained only prose; whether it listed events by issue; or whether it listed by either date or a combination of issue and date). We also noted the date when the sign-out had last been updated.

We created a composite score based on four elements that we felt defined a comprehensive sign-out, based on the literature, ^{4, 5, 7, 9, 10, 18} and that would be applicable across multiple different specialties: updated in the last 24 hours, included diagnosis/presenting symptom, included general hospital course, and included clinical condition. We did not include elements such as primary medical doctor, medications or laboratory results because our sign-out system automatically incorporates these from the electronic medical record. Sign-outs received one point for each of the four elements that they contained. We considered a sign-out with a composite score of four to be a comprehensive sign-out.

We looked at several variables to see if they were associated with comprehensiveness of sign-out: specialty; training of the last person who updated the sign-out (such as medical student, APRN, resident, or attending), which we used as a proxy for who generally updated/wrote the sign-outs; and days since admission.

The Human Investigation Committee at Yale Medical School approved this study and granted waivers for the Health Insurance Portability and Accountability Act and for patient consent.

Analysis

We assessed the frequency of content elements, format, and being updated within 24 hours, and tested differences among specialties with chi-square or Fisher’s exact tests as appropriate. Using a logistic regression model, we analyzed whether a composite score of four was associated with specialty, level of training of last updater, and days since admission. We used SAS 9.2 (SAS Institute, Cary, NC) for quantitative analyses. We used p<0.05 as the level of significance. All tests were two-tailed.

RESULTS

Study sample

On the study date there were 654 patients admitted to eligible specialties: 392 medicine, 88 surgery (including trauma, surgical oncology, transplant, endocrine, vascular, emergency general surgery, and gastrointestinal), 119 pediatrics, 42 obstetrics, and 13 gynecology. After excluding all ICU patients and well baby nursery patients, and re-categorizing patients who had been labeled with the wrong specialty, there were 489 eligible patients: 330 medicine (comprised of 165 hospitalist patients and 165 house staff patients), 68 surgery, 36 pediatrics, 41 obstetrics, and 14 gynecology. Of these eligible patients, 457 (93%) had sign-out notes (Table 1).

Table 1.

Characteristics of study sample

	Hospitalists	House Staff	Surgery	Pediatrics	OB	Gynecology
Total patients	165	165	68	36	41	14
Total sign-outs	165	148	64	36	30	14
% Total patients with sign-out	100	90	94	100	73	100
Total pts admitted >24 hrs prior to data collection	106	100	33	22	14	5
Total sign-outs for pts admitted >24 hrs	106	100	32	22	14	5
% Pts admitted >24 hrs with sign-out	100	100	97	100	100	100
Sign-out process	Written	Written and verbal	Written and verbal	Written and verbal	Written and verbal	Written and verbal
Main facilitator(s) of sign-out process	PAs/APRNs	Interns and residents	Interns and PAs	Interns	Interns and residents	Interns and residents
Days since admission: Median, days (25th-75th percentiles)	3.0 (1.0-9.0)	5.0 (1.0 - 12.0)	1.5 (1.0 -6.5)	3.0 (1.0- 12.5)	1.0 (0.0 -3.0)	1.0 (0.0 -8.0)

OB: Obstetrics

Content

Content elements are shown in Table 2. The frequencies of including plan and rationale in anticipatory guidance statements and tasks are shown in Figure 1 for medicine only, which comprised the great majority of anticipatory guidance statements and tasks.

Table 2.

Content elements of written sign-outs, by specialty

	Hospitalist	House Staff	Surgery	Pediatrics	OB	Gynecology	P value
Updated within 24 hours of data collection	164 (99.4)	143 (96.6)	59 (92.2)	34 (94.4)	29 (96.7)	10 (71.4)	<0.001
Days since last update, mean (SD)	0.08 (0.3)	0.30 (1.2)	0.58 (1.3)	0.25 (0.6)	0.17 (0.5)	2.21 (3.8)	<0.001
Primary medical doctor	126 (76.4)	103 (69.6)	10 (15.6)	12 (33.3)	2 (6.7)	0 (0.0)	<0.001
Diagnosis	160 (97.0)	142 (96.0)	62 (96.9)	29 (80.6)	29 (96.7)	14 (100.0)	<0.001
Hospital course	149 (90.3)	136 (91.9)	54 (84.4)	36 (100.0)	29 (96.7)	8 (57.1)	<0.001
New events	80 (48.5)	103 (69.6)	47 (73.4)	25 (69.4)	19 (63.3)	7 (50.0)	<0.001
Clinical condition	90 (54.6)	88 (59.5)	22 (34.4)	26 (72.2)	18 (60.0)	8 (57.1)	0.005
Mental status	5 (3.0)	11 (7.4)	1 (1.6)	1 (2.8)	0 (0.0)	0 (0.0)	<0.001
Any anticipatory guidance	11 (6.7)	58 (39.2)	4 (6.3)	14 (38.9)	1 (3.3)	0 (0.0)	<0.001
Any task (excludes “nothing to do”)	28 (17.0)	44 (29.7)	14 (21.9)	2 (5.6)	0 (0.0)	2 (14.3)	<0.001

OB: Obstetrics

Figure 1. Proportion of medicine sign-outs including anticipatory guidance and tasks.

We identified several themes for anticipatory guidance statements in the complete dataset. Most anticipatory guidance statements discussed one of the following: what to do if the patient developed a fever; medication recommendations for insomnia, agitation, or pain; plan for responding to a change in vital signs; parameters for avoiding nephrotoxic medications in patients with acute kidney injury; and transfusion parameters.

Tasks tended to focus on following up recommendations from consult teams; following up lab results; monitoring for worsening respiratory status and for appropriate overnight fluid intake and output; and ensuring the patient had the appropriate overnight procedure or imaging study. Types of tasks were not appreciably different between specialties except that surgical tasks were most often to conduct a post-operative check on the patient. When we included “nothing to do” as a task, we found that they comprised 45 (51%) of medicine house staff tasks; however, no other specialty used that phrase. Thus, listing “nothing to do” as the task seems to be a pattern specific to medical house staff sign-outs at our institution.

Format

Format varied among specialties. Pediatrics and obstetrics both tended to format their sign-outs by issue (89% and 97%, respectively). Surgery was divided between prose only (50%) and other (date or issue/date combination, 47%). Medicine hospitalists used predominantly prose format (96%), while medicine house staff were more evenly divided among the three format groups. Gynecology contained mostly prose formats (79%).

Training of last updater

Training of last updater was used as a proxy to represent the person who most often contributes to the sign-out in each specialty (Figure 2). We found variations both among specialties and even within specialties; for instance, house staff and surgery were fairly evenly divided among the three training groups.

Figure 2. Training of last updater, by specialty.

Updated

Nearly all sign-outs across specialties (96%) had been updated within 24 hours.

Composite score analysis

While 412 (90%) sign-outs across all specialties had at least three elements of our composite score, less than half (49%) of all sign-outs contained all four elements. In addition, no sign-out contained zero elements. Surgery had the lowest percentage of sign-outs with a composite score of four (Table 3). Compared to surgery, all specialties (with the exception of gynecology) were significantly more likely to have a composite score of four. We found that students and PGY1 trainees were significantly more likely than either other group (RNs/APRNs/PAs, and PGY2+trainees/fellows/attendings) to have a composite score of four. Days since admission was not associated with a composite score of four.

Table 3.

Multivariate analysis of achieving composite score of 4

		Odds Ratio	Confidence Interval	P value
Service				0.019
	Surgery (Reference)
	Hospitalist	3.06	1.56 - 6.02
	House Staff	2.86	1.50 - 5.49
	Pediatrics	3.12	1.24 - 7.85
	Obstetrics	3.40	1.30 - 8.91
	Gynecology	2.40	0.68 - 8.39
Training				0.014
	PGY2+, Fellow, Attending (Reference)
	Nurse, APRN, PA	0.85	0.49 – 1.48
	Student, PGY1	1.93	1.05 – 3.55
Days since admission		1	0.99-1.00	0.36

DISCUSSION

In this cross-sectional study, we explored the written sign-out practices of several inpatient specialties. Because YNHH uses a standardized electronic sign-out template across all services with built-in prompts, we predicted that there would be a resemblance among sign-outs in content elements, format, and being updated. We found that basic information such as diagnosis and hospital course was included at high rates by nearly all specialties. In addition, most sign-outs in all specialties had been updated within 24 hours. Conversely, we found that complex elements, such as clinical condition, anticipatory guidance, and tasks, were included at relatively low rates in all specialties despite training and templating. Nonetheless, despite similar overall areas of strength and deficiency, specialties differed significantly from each other in rate of content inclusion. Specialties were also not uniformly superior or inadequate at developing high quality sign-outs. For example, pediatric sign-outs were among the best of all specialties at including information about clinical condition and anticipatory guidance, but were the least likely to include the diagnosis and tasks to complete.

Sign-outs across specialties performed well on many elements, including being updated. It is also important to note that much important content was automatically included (such as demographics, medications, key laboratory results); at other institutions transcribing such content can be very time-consuming and error-prone. However, we found that the sign-outs were weakest about including information that may be considered most important to the sign-out. Sign-outs lacking content elements that require more clinical assessment (such as clinical condition, overnight tasks, and anticipatory guidance) are associated with an increased risk of medical errors.⁵ Complex variables thus are important to include in written sign-outs because they help create a full picture of the patient for the covering team.

Interestingly, our multivariable analysis found that junior trainees (students and interns) were more likely than senior clinicians to include all elements of our composite score. We were surprised by this finding because our previous work showed that senior residents are more likely than interns to generate complex content such as clinical condition or plans for anticipatory guidance during verbal sign-outs.⁷ It is possible that junior clinicians rely more heavily on written sign-out materials and are consequently more motivated to be comprehensive and thoughtful in constructing them. This finding is worthy of further investigation.

Format varied widely among specialties, and sometimes even within specialties. We do not know whether certain formats are more effective than others, but it may be worth attempting some standardization of format, at least within specialties. In the same way that clinicians have adopted a uniform method of presenting a patient’s history and physical on rounds so that all team members know what to anticipate during the presentation, standardization of sign-out format may provide a more efficient and expected way of presenting data to the overnight covering team. For example, we noted that the obstetrics service used a highly structured and consistent format for the free-text section of the sign-out that was entirely obstetrics-specific. In fact, it was not even used by the same house staff when writing gynecology sign-outs. In effect, this specialty has adopted a “structured flexibility” approach in which elements of the sign-out have been standardized in appropriate areas and at appropriate levels without unnecessarily rigid requirements that a standardization that works in one setting be applied to a different setting.²⁰ In a similar fashion, the overall sign-out template used at the institution was designed with a structured flexibility approach in mind, in which major content and design elements were standardized (overall print report appearance, automated import of key fields, standardized major topic areas) but written format and specific content items were left open for different specialties to address in their own ways.

Our results for medicine house staff were fairly comparable to those found in similar studies. Horwitz et al. analyzed the sign-out process for medicine house staff at this institution, prior to the implementation of a standardized sign-out template and prior to a sign-out curriculum that is now taught to medicine interns when they begin residency.²¹ We found that 81% of written sign-outs included hospital course (defined as any event occurring during hospitalization, including new events); 39% of sign-outs included clinical condition (defined as in our study); and 99% mentioned whether there were tasks to complete.⁷ In our present study, we found that 92% of medicine house staff sign-outs included hospital course; 70% of the sign-outs included new events; 59% included clinical condition; and 60% mentioned tasks (30%, excluding “nothing to do”). Bump et al. studied the sign-out process for medicine interns and found that 99% of written sign-outs included general hospital course (similar to our findings); 42% of sign-outs included anticipatory guidance (also similar to our findings); and 90% included tasks.⁹ Thus, the biggest difference between our study and the two prior studies was a higher prevalence of clinical condition information but a lower frequency of tasks. These comparisons suggest that medicine house staff training may need to focus more on including tasks in computerized written sign-outs, or at least making it clear when there are no tasks to do.

Our results demonstrated that there were certain differences among specialties, but often consistency within specialties, which suggests that there are specialty-specific cultures regarding written sign-outs. Yet we further found that even within a single specialty, practices were distinct in different settings. For instance, obstetrics and gynecology are considered separate services but are staffed by the same resident pool. However, there were very different practices between obstetrics and gynecology regarding update frequency (97% versus 71%, respectively), which likely had a trickle down effect on variables such as hospital course and new events. Medicine provides another example; hospitalists and house staff varied on several elements such as format, anticipatory guidance, and tasks. The medicine hospitalist and house staff teams function independently at YNHH. Their divergent approaches to the written sign-out may be influenced partly by the fact that hospitalists do not do verbal sign-out, and also by the distinct types of training that house staff and PAs/APRNs (whom we found to be the main group updating hospitalist sign-outs) receive.

This study has several limitations. First, as a single-site study at an academic institution it may not be widely generalizable. Our study is cross-sectional and may have been influenced by individual practitioners, especially within smaller fields such as obstetrics and gynecology. Our study focused on written sign-out only; thus, some of the components missing in these sign-outs may have been discussed in person during verbal sign-out. Nonetheless, we believe written sign-outs should always contain at the very least the elements in our composite score, as verbal information is not as durable. There are no official standards for how to write a sign-out; however, our assessments were based on numerous reports in the literature and have high face validity. Finally, we did not assess outcomes associated with quality of sign-outs, or accuracy of the content, and therefore cannot draw conclusions about ideal content or format.

In conclusion, we conducted a cross-sectional study at an institution with uniform EMR-based written sign-outs and determined that specialties differed in their approach to the written sign-out even in the presence of standardized templates. We found that there were certain specialty-specific sign-out practices, but that specialty practice varied in different settings. Most sign-outs across all specialties were updated within 24 hours and included basic information such as diagnosis and hospital course. Sign-outs less frequently included complex data such as clinical condition, anticipatory guidance, and overnight tasks, despite prompts included within the template. Thus, there is a trend across all specialties to report basic patient data and to update sign-outs, but not necessarily to address challenging aspects of a patient’s clinical picture or to predict potential overnight events. Given the variability we encountered, we concluded that structured templates alone do not guarantee inclusion of critical content. We support institutional standardization of the sign-out template but acknowledge that structured flexibility is also necessary. We recommend teaching residents and hospitalists across specialties how to write sign-outs in a way that highlights basic sign-out principles, but that addresses specialty-specific cultures. We suggest assessing the effect of these interventions on effectiveness of sign-outs and on patient outcomes. Future research may focus on comparing sign-outs over time to understand whether there is improvement as more experience is gained.