Signout: A Collaborative Document with Implications for the Future of Clinical Information Systems

Abstract

Signout is an unofficial clinical document used traditionally to facilitate patient handoff. Qualitative studies have suggested its importance in clinical care. We used a novel technique to quantify the use of signout by analyzing clinical information system logfiles. Viewing and editing events were collected for 1,677 unique patients admitted to our internal medicine service. We found the average patient’s signout on a given day is viewed frequently (>6x) and edited frequently (>2x) with multiple unique viewers (>3) and editors (>1). We also found that signouts are used throughout a 24-hour period, not just at the time of handoff. Finally, we showed that they are viewed months and even years after their creation. Signout is therefore a highly utilized, collaborative, clinical document used for more than patient handoff. Our findings also suggest that clinical information systems may benefit from the introduction of collaborative tools such as subscription, versioning, and author-attribution utilities.

Introduction

Healthcare in the hospital setting is highly collaborative¹, and team interactions and frequent shift-changes introduce challenges in communication, continuity and data transfer.² This can amount to something like a dangerous game of “operator”, in which messages can be arbitrarily permutated at each transmission point, and can lead to uncertainty for clinical decision-makers.³ In fact, it has been shown that adverse medical events are associated with periods of cross-coverage.⁴ The Joint Commission has recognized these potential hazards by including a standardized approach to patient handoff in the current set of National Patient Safety Goals.⁵

One solution to the problem of coordinating patient care among multiple providers is a widely implemented clinical document known as ‘signout’. There are several types of information typically found in a signout document including patient demographic information, a brief summary of the patient’s clinical situation, contact information, medications, and a to-do or task list.^{6, 7} Evidence suggests that implementing computerized signouts may contribute to lower adverse event rates.⁷ However, signouts are not standardized and often are not considered part of the official medical record.⁸ Medical students and residents typically are not trained to write or use a signout.⁹ Yet it is an important document given its purpose: to ensure pertinent information is clearly transmitted between caretakers to provide continuity of care and prevent medical errors. Furthermore, studies have suggested that signout is often used beyond patient handoff, such as during pre-rounding or rounding activities and to assist with daily task management.⁶

Most studies of signout have evaluated its content and relationship to clinical workflow in a qualitative manner through focus groups and interviews. Our study aimed to confirm the sentiment that signout is an important and highly utilized clinical document through the objective analysis of coded data. We attempted to do so in a novel and efficient manner by studying it through the analysis of clinical information system (CIS) logfiles. We further hypothesized that by better characterizing the use of signout we would be able to determine what enhancements could potentially render this document even more useful for the purpose of increasing efficiency and reducing errors in collaborative care.

Methods

Overview:

This is a retrospective, descriptive study of signout usage. We examined log files of system use for a 6-month time frame by a teaching internal medicine service at a busy, urban academic medical center. Primary data collection utilized analysis of log files stored in our clinical data warehouse created by our web-based Clinical Information System (WebCIS).¹⁰ WebCIS is used at our institution for patient data lookup and sometimes for authoring clinical notes. Of particular relevance to this study, signout notes are authored and viewed using WebCIS.

We chose to limit the study to data associated with patients admitted to the internal medicine service because of prior knowledge that housestaff on this service regularly use WebCIS to author and access signout. The time frame was chosen to help identify the appropriate patients, as it corresponds with six, approximately 1-month long, rotation blocks for the interns on the medicine service. All data were collected and analyzed in accordance with our Institutional Review Board guidelines and approval.

CIS Log Mining:

WebCIS creates logfiles for system auditing purposes that include a one-line entry for essentially every click that users make when logged into the system. A line in the logfile contains a timestamp, the user-id of the person making the click, an indication of the context in which that click was made, a description of the data to which that click led, and the medical record number of the patient of interest. Colleagues at our institution have previously described a CIS Mining framework.¹¹ It is based on established web-mining techniques and applies four phases of analysis to log files generated by WebCIS, including: data collection, preprocessing, pattern discovery, and pattern analysis. We applied this CIS Mining framework to WebCIS logfiles to study signout.

Data Collection:

Our base dataset was a complete set of logfiles spanning a 169-day period (6/20/2006-12/6/2006), which contained information on all user activity via WebCIS.

Preprocessing:

Preprocessing includes data cleaning and transformation. We pruned the original large data set by selecting only those lines in the log files that pertained to patients admitted to the medicine teaching service. This was achieved in several steps. First, we analyzed log file entries and found all patients on the primary list of each intern when rotating through the medicine service. A primary list is a feature in WebCIS that catalogs the patients of each individual caretaker. After the complete patient list was generated, all log entries in the original data set that did not contain a reference to one the qualifying cases were deleted. Finally, individual files were saved, one for each unique patient, containing all log file entries associated with that patient. This served not only to make the data set more computationally manageable, but also eliminated noise from the log files, such as entries created by administrative processes and error logs.

Pattern Discovery:

The patterns of interest were the frequency with which signout was viewed and edited, and how many users were doing the viewing and editing. The derivation of these metrics is summarized in Table 1. It should be noted that frequencies and user numbers were calculated on a 24-hour basis.

Table 1.

Pattern discovery grid showing methods to determine signout view and edit frequency, and number of users viewing and editing. All values were calculated on a 24-hour basis.

	Frequency	Users
View	average # views per reader per patient signout	average # viewers per patient signout
Edit	average # edits per editor per patient signout	average # editors per patient signout

Pattern Analysis:

In this last step in CIS mining we explored visualization techniques to demonstrate overall trends in the dataset. Histogram displays were constructed to visualize temporal patterns of daily (intra-hospitalization) and retrospective (inter-hospitalization) signout viewing.

Results

We successfully collected and extracted log files documenting WebCIS access of clinical data on 1677 unique patients admitted to the general internal medicine service. The total size of the log files of this data set was 13,539,958 unique entries (lines).

Viewing and Editing:

The frequency of reading and editing, and the number of users reading and editing signouts are summarized below in Table 2.

Table 2.

Results from pattern discovery grid. Please refer to Table 1 for units of reported values.

	Frequency	Users
View	2.56	3.60
Edit	1.62	1.40

Temporal Visualization:

We constructed histograms to visualize temporal viewing of signout documents. Figure 1 shows the number of views of all signout documents in our data set by all users distributed over the 24-hours in a day. Figure 2 shows a histogram of the differences in time between when a signout was authored and when it was viewed for all patients, and for one representative individual patient.

Figure 1.

A histogram showing the count of all signout document viewing events as distributed throughout the 24-hours in a day.

Figure 2.

A histogram showing the elapsed time between when signout documents were written and when they were viewed for (a) all patients and for (b) one representative individual patient.

Discussion

We used a novel, efficient, and objective technique to study the clinical document known as signout and its associated workflow patterns. Results from our analysis of CIS log files confirm our stated hypothesis that signouts are used frequently and collaboratively on a busy teaching internal medicine service. Signouts are used at times other than during handoff in the hospital setting, they are used in between admissions, and they are often referred to several years after they were initially authored.

Viewing Frequency and Users Viewing:

As presented in Table 2, signout for a given patient was viewed more than twice daily for the average user, and there were on average more than three viewers per day. We can multiply these numbers to see that for a given patient, there were on average more than six daily viewings of signout. An individual intern or resident typically carries 8-12 or 16–24 patients respectively, and multiplying those numbers by at least two views daily per patient demonstrates that housestaff spend a significant amount of time viewing, and re-viewing patient signout documents.

The histogram displayed in Figure 1 further characterizes the viewing of patient signouts, and has important implications. Van Eaton et al have shown through focus groups with surgical residents that signout is used for daily workflow such as pre-rounding and rounding activities, in addition to using it to facilitate patient handoff.⁶ Figure 1 serves as more objective evidence to support this hypothesis. Certainly there are peaks in signout viewing activity that correspond to patient handoff times (7–8am and 5–8pm). However, it is also clear that this document type is viewed throughout the 24-hour period, especially adjacent-to and between handoff times.

Editing Frequency and Users Editing:

Signout documents were typically updated by a unique user more than once daily, and the fact that the average number of editors was greater than one per day suggests that there are multiple authors/editors that collaborate on this document type. A possible explanation is that covering or more senior physicians are periodically helping the interns with the daily task of updating the signout. Multiplying the average number of edits/user by the average number of editors/day indicates that a particular patient’s signout is edited through WebCIS more than twice daily.

Relationship Between Editing and Viewing:

Recent work by Hripcsak et al studied computer access to clinical data during the evaluation of patients presenting to the emergency department through our WebCIS system.¹² It was noted that resident signout notes were among the most frequently used “old data” sources. To further explore this finding, we created our histograms shown in Figure 2 that visualize the elapsed time between when a signout note was authored or edited, and when it was viewed. The initial spike of viewing seen in the first couple of days since a note was written was expected, as a note recently written would likely be viewed frequently. What is remarkable, however, is that when all of these events were plotted together it became clear that signout notes were viewed significantly beyond the time from which they were written. Figure 2a shows viewing activity of old signout documents well beyond what would typically be the timeframe for a hospital admission.

Figure 2b is a histogram showing this parameter for a single patient. There were many views of this patient’s signout notes within the first two days after each was written. More views were captured several days after each signout was written. These views likely signify a follow up visit post-hospitalization, when old signouts were viewed to learn about a previous hospitalization (some physicians have access to WebCIS in the outpatient setting at our institution). Signouts written about this patient were also viewed about one month after they were originally written, likely representing data gathering efforts at the beginning of a subsequent hospital admission. It is probable that most patients who are admitted to the hospital will demonstrate similar secondary spikes of retrospective viewing activity seen as seen in Figure 2b. The summation of these retrospective, or inter-hospitalization views is likely the cause of what appears to be a baseline level of activity seen in Figure 2a that stretches out to almost eight years.

Study Limitations:

One important point to consider when interpreting these results is a common practice of our housestaff regarding their use of signout (which is likely to be found at other institutions as well). WebCIS provides convenient access for viewing and editing signout, but there is much viewing and editing that occurs off-line. Physicians typically print their signouts and keep them in a pocket. This paper printout is constantly referred to during the day, and small updates are made by writing on the printed sheet of paper. However this does not detract from our findings of high frequency of editing and viewing – if anything it suggests we are underestimating the actual use of signout.

As mentioned previously, signouts can be viewed in WebCIS by physicians in the outpatient setting. We chose not to distinguish between signout views that were made from within the hospital and those that were made in the outpatient setting. This allowed us to have a better picture of how signouts were used beyond an initial hospitalization, demonstrated by Figure 2. However, we realize that this affects the measurements of what we are assuming to be intra-hospitalization reading frequency presented in Table 2 and Figure 1. When a patient is seen in the outpatient setting, it is possible that a physician will view his or her signout just once. These occurrences may have caused an artificially low result for our reported average number of daily views and users viewing. Notably, recognition of this actually further strengthens our conclusion that signout is a collaborative document with many viewings and viewers.

This study included only patients admitted to the general internal medicine service at our large, academic medical center, which may limit the generalizability of our findings. Furthermore, we are fortunate to have access to signouts through our web-based CIS which is not available at all institutions, some of which may record signouts entirely on paper.

Implications for the Future of Signout:

We have shown that signout is a widely used and collaborative document that is viewed by multiple members of the caretaker team daily and frequently updated. We also showed that signout is used for more than handoff, as demonstrated by its use throughout the entire day. Finally, we demonstrated that signouts written during past hospitalizations are later used as reference material, even years afterwards. Given these and previous findings, and the increasing recognition that patient handoff plays a key role in the quality of hospital care, it is clear that this document deserves attention, not only from the medical community but also from those of us working in the field of biomedical informatics. Perhaps it is time for signout to be integrated into the medical curriculum, and for it to become an official part of the medical record.

Implications for the Future of Clinical Information Systems:

Signout documents likely will, and perhaps should, spend a significant amount of time in the physical world as paper in the pockets of physicians. However there is a role for clinical information systems to play in managing and disseminating signout, and CIS designers should consider adding this functionality to their of clinical documentation tools. Beyond signout integration, our study suggests that new CIS functionalities may facilitate collaborative care for patients in the hospital setting.

There are collaborative tools currently used in other arenas such as on the Web and within industry that are growing in popularity and have proven quite useful in enhancing both work and play.¹³ Several of these tools map well to the pattern discovery grid developed in this study, as they can be useful in each of the situations represented therein (Table 3). The web itself, and popular “personal broadcasting” utilities such as blogs, have demonstrated great utility in getting information dispersed to many viewers. High frequency viewing is facilitated by RSS/Atom feeds and other subscription services, which help keep groups of people alerted to any changes to web content and blogs. High frequency editing is enhanced by versioning and rollback tools that enable the safeguarding of rapidly changing content such that of wiki’s or company intranet knowledgebases. Difference engines allow users to more specifically determine what content has been changed. When more than one user can make such changes, concurrency control ensures that no one accidentally ruins someone else’s work. Commenting and attribution mechanisms allow individuals to maintain a distinct identity and authorship recognition among their peers.

Table 3.

Theoretical framework for relationship of collaborative web tools to the study findings.

	Frequency	Users
View	RSS, Subscription/Alerting Services	Web / Blog-like Distribution
Edit	Versioning & Rollback, Difference Engine	Concurrency Control, Commenting, Attribution

We believe these collaborative tools may be useful if implemented into clinical information systems. Our analysis of signout and its use has demonstrated the high frequency of reading and editing and the multiple users who do so with this document in the hospital setting. Table 3 represents our proposal of how such tools could be effectively aligned with the results of our workflow analyses.

Conclusion

We used a CIS log mining technique to characterize an important clinical document known as signout. Signout was shown to be a collaborative document that is used for much more than patient handoff. This area of clinical documentation deserves more attention, standardization and training efforts. Our findings also suggest that clinical information systems could benefit from collaborative tools such as subscription, versioning, and author-attribution utilities. This opens up an area for future research, which the authors of this paper intend to pursue.