Analysis of a Computerized Sign-out Tool: Identification of Unanticipated Uses and Contradictory Content

Abstract

A computerized tool designed to facilitate physician sign-out has been in use at Vanderbilt University Hospital and Children’s Hospital for close to a decade. The authors produced descriptive statistics of sign-out tool use by hospital unit, user’s professional role, and time of day. Results showed anticipated use by resident physicians and nurse practitioners to generate and print notes, as well as unanticipated use by nurses, case managers, and medical receptionists/care partners to print providers’ notes. Additionally, the authors categorized the content of sign-out notes having possible discrepancies and identified contradictory content. Findings have implications for workflow and redesign of the sign-out tool.

Introduction

The use of standardized sign-out procedures has been identified as a means to reduce communication errors during the transfer of inpatient care responsibilities between physicians. With resident work hour reforms in place, more patient handoffs take place than in the past, opening opportunities for information to “fall through the cracks.” According to Solet et al., a combination of oral and written communication provides physicians the best way to perform patient handoff¹. The use of computerized systems can augment this process. Errors at patient handoff can contribute to discontinuity of care and adverse events, but the use of computerized systems to facilitate handoff has been associated with reductions of these measures^2,3. Implementing computerized sign-out in hospitals has been described^4,5,6 but researchers have yet to perform full scale evaluations of how these systems are used in practice and can be improved.

At Vanderbilt University Hospital and Children’s Hospital physicians have used a computerized tool to facilitate patient handoff for close to a decade. This paper presents descriptive usage statistics plus a method to classify user content. These findings are intended to guide evaluation and subsequent redesign of the sign-out tool.

Background

Users access the sign-out tool through WizOrder, Vanderbilt’s care provider order entry system. Based on either a user-specified list of patients or a hospital unit’s census, the user can view, modify, and print “sign-out notes,” which consist of demographics, current information from orders (condition, diagnosis, medications, allergies, etc.), a “case summary” section, and a “to check” section. Demographics and current information are automatically generated from hospital databases, and users contribute text to “case summary” and “to check” sections. The intent of the “case summary” is a narrative of a patient’s current hospitalization, while the “to check” is a list of issues that demand immediate attention. Text in both of these fields is saved to a text file on a file server so that users can retrieve and update the information during the patient’s stay. A provider leaving the hospital generally prints sign-out notes at the end of his or her shift to use as a basis for conference with an incoming provider who will take on patient responsibilities. The printout is physically handed over to the incoming provider who can use it to quickly retrieve and record information for the patients in his or her care.

Methods

To examine sign-out tool usage, the authors configured the sign-out tool to record sign-out notes in a database each time a user saved his or her work. Each record in the database contained a “case summary,” “to check,” medical record number, case number, hospital unit, bed assignment, and timestamp for a patient when the user saved it, as well as the provider’s user ID and an indication of whether the provider printed the record. The database collected records from September 1, 2006 through December 31, 2006. During this time period, the hospital experienced 22,718 inpatient/observation admissions.

Descriptive Statistics

The authors examined sign-out tool usage in terms of generation and printing of sign-out notes according to hospital unit, provider role, and time of day. A sign-out note is defined as a database record for a patient saved by a provider at a specific time containing both a “case summary” and “to check” section. “Case summary” and “to check” sections may be null or non-null. Each sign-out note is identified by its patient's medical record number and timestamp of creation. A sign-out note is unique if either the “case summary” or “to check” is non-null, and the text of either the “case summary” or to check differs from the preceding record of a patient's hospitalization or is the first record of a patient's hospitalization containing a non-null “case summary” and/or “to check.” A sign-out note is printed if its database record indicates a user printed it. Users in the sign-out database were cross-referenced with a staff authorization database to determine their professional role. Users included attending physicians, resident physicians, medical students, nurse practitioners, nurses, case managers, medical receptionists/care partners, and others, which included social workers, respiratory therapists, dietitians, and unspecified ancillaries.

Content Analysis

Prior examination of “to check” notes revealed a tendency for users to state there was “nothing to do” or “nothing to check” (often using abbreviations like “NTD” or “NTC”) but also include additional care instructions. The authors randomly selected 500 unique sign-out notes generated by general medicine residents with “to check” sections containing “NTD,” “NTC,” and/or “nothing” followed by additional text to identify potential contradictions. A physician reviewer analyzed the content of “to check” notes and categorized them. The authors calculated the occurrence of the following categories: indication of code status, plan of care information (e.g. “patient is going to a procedure”, “NPO”), demographic information, behavioral (e.g. “patient combative”), pertinent medical history (e.g. “allergic to NSAIDS”), conditional statements (e.g. “if short of breath give 20 mg lasix”), transfer/discharge information, and contradictions where the cross-cover team was given explicit tasks despite the label “nothing to do.”

Results

Descriptive Statistics

Users created 232,963 total sign-out notes for 13,519 unique patients, or 59.51% of patients admitted to the hospital during the study period. Of the total notes, 22% (52,560) were unique. Users printed 108,660 notes, or 46% of the total notes.

Table 1 shows usage statistics for the units with the highest and lowest ratios of number of unique patients with sign-out notes to number of unique patients with orders. A ratio close to 1 meant sign-out was a frequent part of the unit's workflow. General medicine, post-partum labor and delivery, pediatric medicine, trauma, and neonatal intensive care generated the most total notes, while labor and delivery, neonatal intensive care, pediatric medicine, general medicine, and cardiac stepdown generated the most unique notes. General medicine, trauma, pediatric medicine, post-partum labor and delivery, and medical intensive care printed the most notes.

Table 1.

Highest and lowest sign-out tool usage estimates for units sorted by proportion of unique sign-out cases to unique order cases.

	Unique sign-out cases/unique order cases	Total printed/unique sign-out cases	Unique notes/unique sign-out cases	Total unique notes	Total printed notes	Unique order cases	Unique sign-out cases
Neonatal ICU	100.00%	12.01	9.28	6042	7817	637	651
Trauma ICU	99.87%	13.32	1.56	1230	10484	788	787
Pediatric medicine	99.35%	4.02	3.14	4287	5498	1375	1366
Neuro ICU	97.47%	9.64	2.17	1004	4464	475	463
Renal	96.93%	9.00	1.82	974	4824	553	536
Pediatric cardiology	96.64%	9.56	4.47	1411	3021	327	316
Adult medicine	96.12%	8.71	2.81	4180	12931	1545	1485
Neurology	95.74%	7.14	1.24	1140	6575	962	921
Medical ICU	95.54%	14.46	1.27	705	8052	583	557
Pediatric critical care	90.03%	6.86	4.72	3068	4456	722	650
Pediatric surgery	45.09%	2.61	1.84	398	563	479	216
Labor & delivery	40.62%	1.05	1.80	1042	607	1428	580
Cardiac ICU	39.82%	2.01	1.11	294	531	663	264
Cardiac cath lab	35.15%	1.12	1.23	714	648	1653	581
Orthopaedics	33.93%	6.06	0.87	295	649	1266	214
Women's surgery	31.02%	1.31	2.75	876	417	1025	318
Surgical ICU	30.60%	3.58	1.46	180	440	402	123
Burn	29.52%	4.23	1.61	50	131	105	31
Surgical stepdown	25.89%	5.47	3.40	667	1072	757	196
General surgery	16.49%	2.64	1.39	239	454	1043	172

Table 2 shows the number of unique sign-out notes generated and number of sign-out notes printed by role. Residents generated the most unique notes, while nurses printed the most total notes.

Table 2.

Unique notes generated and total notes printed by professional role

Professional role	Generated	Printed
Resident	37012	39594
Nurse practitioner	5917	6411
Medical student	3987	3563
Nurse	3127	40117
Case manager	1618	5491
Medical receptionist/care partner	372	10560
Attending	327	528
Other	200	2396

Figure 1 illustrates that printing levels reached peaks between 05:00 and 08:00 as well as 16:00 and 18:00. Unique note generation was highest between 11:00 and 13:00. Figure 2 shows that the majority of printing by nurses and unique note generation by residents occurred at times similar to those of peak printing and unique note generation in Figure 1.

Figure 1.

Notes printed and unique note generated by time of day

Figure 2.

Unique notes generated by residents and notes printed by nurses

Content Analysis

Table 3 shows the frequency of each content category. There were a total of 38 notes that required activity from the cross-covering provider. Sixteen of these required following up on a laboratory, cardiology, or radiology test result, and 10 on a consult. 10 required an order, and 6 required a physical check of the patient (e.g., post-catherization hematoma check). The most common type of additional information was conditional advisories, usually advising the covering provider how to treat common conditions faced by patients. Other common conditional statements included warnings not to give pain medications and conditions that were ignorable. Pertinent medical information frequently included information such as recent radiology tests and their results, ongoing cardiology evaluations, and lab results.

Table 3.

Content categories in “to check” fields

Category	N (%)
Conditional advisories	274 (55%)
Pertinent medical history	163 (33%)
Transfer/discharge instructions	90 (18%)
Plan of care information	70 (14%)
Code status	68 (14%)
Contradictory Medical Tasks	38 (7.6%)
Behavioral information	18 (3.6%)
Demographic information	13 (2.6%)

Discussion

Descriptive Statistics

Units with high levels of sign-out activity represented a variety of specializations, while surgery units generally had lower levels of sign-out activity. Although we did not calculate average length of stay (LOS) per unit, we posit that longer unit LOS correlates with high levels of sign-out activity. During a patient’s stay on a unit the number of unique notes generated varied from 9.28 (neonatal intensive care) to 1.6 (cardiac intensive care), and total notes printed varied from 14.46 (medical intensive care) to 1.31 (women's surgery). The ranges exceeded our expectations.

The sign-out tool was designed for the task of “physician sign-out,” so identifying a large proportion of non-provider users was unexpected. In particular, nurses, case managers, and medical receptionists/care partners were found to have high levels of printing and low levels of unique note generation. Of non-providers, nurses were responsible for printing the most notes on the majority of units. However, notable exceptions to this trend included medical receptionists in medical intensive care printing over 66% of notes and medical receptionists in neurology printing more than 34% of notes. Although non-providers were responsible for low levels of unique sign-out note generation in most units, case managers authored 64% of unique notes in trauma. Informal analysis shows that trauma sign-out notes are used exclusively for patient disposition, and that case managers facilitate this process. Additional observations include medical student sign-out activity being limited mostly to obstetrics and gynecology units.

Resident physicians and nurse practitioners authored 93% of notes they printed. In contrast, nurses authored only 8% of the notes they printed, despite printing more notes than residents overall. Proportions of unique note generation and printing were similar to nursing for other non-providers. Since resident physicians and nurse practitioners instantiate medical decisions for a patient's care team, they are responsible for authoring the bulk of patient care directives, including sign-out notes. The difference in contributions and printing for providers and non-providers suggests that providers serve as “producers” and non-providers as “consumers” of sign-out content. Providers might not be aware that their sign-out notes are being used for more than the organization of patient information for cross-cover or night float physicians. Awareness of consumption by non-providers may influence the content of the notes generated.

The peak times for printing shown in Figure 1 coincide with the times of morning nursing shift change and afternoon provider sign-out. Users appear to print sign-out documents for the new shift. Figure 2 shows that the peak time of providers generating unique notes was from 11:00 to 1:00. New unique note generation by residents reduces substantially after 17:00. Since providers update their notes predominantly in afternoon, provider content may no longer be current.

Content Analysis

Effective physician handoff is important to prevent medical errors.² We found 38 notes (7.6%), despite containing the instruction that there was “nothing to do” for the patient, that contained specific tasks for the on call team to complete. While we do not know of any adverse events resulting from these, several, such as following up cardiac enzymes or electrocardiograms, could lead to adverse outcomes. Some of these instructions were buried within “to check” sections of nearly 80 words. These errors of omission likely result from note reuse. Future system development should design to prevent these errors, perhaps by specifically notating “nothing to do” as a section that is not copied from prior experience.

The most common findings in the “to check” section were of pertinent medical history and conditional advisories. Including these in the “to check” section highlights their use to the cross-covering physician. Further enhancements, such as automatically including potential discharge information or recent procedures from a CPOE system, may also add value to a sign out tool.

Conclusion

The interaction of unit, user’s professional role, time of day, and note content define usage of the sign-out tool. Unanticipated uses and contradictory content shown in this paper can be used to guide design decisions of a new sign-out tool.