In Search of Social Translucence: An Audit Log Analysis of Handoff Documentation Views and Updates

Abstract

Communication and information sharing are critical parts of teamwork in the hospital; however, achieving open and fluid communication can be challenging. Finding specific patient information within documentation can be difficult. Recent studies on handoff documentation tools show that resident handoff notes are increasingly used as an alternative information source by non-physician clinicians. Previous findings also show that residents have become aware of this unintended use. This study investigated the alignment of resident note updating patterns and team note viewing patterns based on usage log data of handoff notes. Qualitative interviews with clinicians were used to triangulate findings based on the log analysis. The study found that notes that were frequently updated were viewed significantly more frequently than notes updated less often (p < 2.2 × 10⁻¹⁶). Almost 44% of all notes had aligned frequency of views and updates. The considerable percentage (56%) of mismatched note utilization suggests an opportunity for improvement.

Introduction

Care teams require communication and information sharing to coordinate and provide care for patients in the hospital¹⁻³. However, poor communication is frequently associated with errors and sentinel events⁴. Studies on improving patient outcomes and care processes after the implementation of electronic health records (EHRs) have shown both positive and negative results⁵⁻⁸. Critical patient information is often difficult to obtain quickly or is simply missing in the EHR^9,10. Moreover, team member information needs differ according to clinician training and specialization¹⁰. Clinical documentation, a common form of team communication, often contains out of date information or is simply not read^11,12. These and other problems pose a challenge to adapting EHR based tools to facilitate team-based communication and information sharing.

There is, however, emerging evidence that new electronic tools for facilitating handoffs, handoff documentation tools, are increasingly used as a source of patient information^10,13,14. Schuster et al. found that despite handoff notes being written by and for residents, other services, such as ancillary services, also consulted these notes for information¹⁰. Furthermore, Vawdrey et al. demonstrated that despite its original intent to facilitate transitions of care for resident physicians, non-physician services, such as nursing and pharmacy, make up more than half of all views of handoff notes¹³.

In a follow-up study to Vawdrey et al, residents, the primary authors of handoff notes, acknowledged team members reading their notes¹⁴. Some even acknowledged a change in the frequency of updates to their handoff notes based on how others in the team incorporated handoff notes into their daily workflow¹⁴. However, many questions still remain about using handoff documentation as a way to disseminate information to other care team members. One such question is to what degree are residents’ expectations and practices in regards to updating handoff notes aligned with other team members? In this study, we examine the patterns of updates and views for electronic handoff notes (handoff notes from this point forward) at a large tertiary care hospital. Specifically, this study investigates the degree with which users are aligned in their viewing and updating practices and focuses on the following question: Are more frequently updated notes also viewed more frequently?

Materials and Methods

Handoff Tool

Handoff Tool is a custom-designed module included in the commercial EHR system (Sunrise Eclypsis). At the time the study data was collected, Handoff Tool included 9 free text boxes: Active Issues, Consult Notes, Contact Information, Coverage To Do List, Discharge Planning, Hospital Course, Notes and Comments, Patient Summary, and Primary To Do List. The tool also provided the functionality to produce a printable report that included structured data, such as labs and medication. Furthermore, users could choose to print a cover sheet that summarized selected patients. At the time of the study, only physicians, physician assistants, and nurse practitioners could make edits to the handoff document, but all members of the care team could access and view the document. In contract to traditional notes, such as the daily progress notes, which started blank each shift, handoff notes were updated from the last save iteration of the note. The tool is described in further detail in¹³.

Dataset

To examine updating and viewing practices related to handoff notes, we collected the document-updating and document-viewing event logs for all of October 2013 at Columbia University Medical Center campuses of NewYork-Presbyterian Hospital, a large urban, academic medical center. The document-editing event log contained the date and time of each edit event, along with which free-text box was edited, the patient medical record number, the patient bed location, the clinician user ID, and the care provider role. The document-viewing event log contained the date, time, and duration of each handoff document access. Additional information in the log included the patient medical record number, the patient bed location, the clinician user ID, and the care provider role. The dataset was then divided into individual notes for each patient per. Each unique handoff document was identified by the patient medical record, and each shift was defined as being either from 7am to 7pm (day shift) or 7pm to 7am the next day (night shift). For each document, the number of updates and views were tabulated. An update was defined as an event when the handoff document was edited. A view was defined as instance when a clinician, other than the primary author of the note, opened and closed the note. The primary author for a note was defined as the clinician who made the most edits to the handoff note during a particular shift. Furthermore, the frequency of views for each clinician role for each note during each shift was also calculated. To correlate the frequency of views and updates, each document was assigned an activity level score, which reflects the relative frequency of the views and updates for each document (see Table 1).

Table 1.

Activity Level Coding Schema

Updates/Views	None (0 Updates)	Minimal (1–2 Updates)	Frequent (3 or More Updates)
None (0 views)	A (matched case)	D (more updates)	E (more updates)
Minimal (1–2 views)	G (more views)	B (matched case)	F (more updates)
Frequent (3 or more views)	H (more views)	I (more views)	C (matched case)

Quantitative Analysis

To identify temporal patterns of viewing, the hour of each view was extracted and plotted. To identify the relationship between updates and views, a linear regression was fitted on the frequency of views and updates for each patient’s note per shift. Then each note was stratified into one of three categories based on the frequency of updates according to the schema found in Table 1. An ANOVA (and Kruskal-Wallis) test was then performed to identify if there was a difference in frequency of note views between the three levels of note updates.

The statistical packages pandas, numpy, and scipy for Python were used to analyze the data, and ggplot2 for R was used to visualize the data.

Qualitative Analysis

To triangulate the finding from the quantitative analysis, interviews were conducted, transcribed, and analyzed for common themes. In total, 3 interviews were conducted. Two residents from the adult general medicine department, and one nurse practitioner from a pediatric hematology/oncology unit were interviewed. All clinicians were consented prior to interview. This study was approved by the Columbia University Medical Center Institutional Review Board (CUMC IRB).

Results

Description of Handoff Note Views

In total, 70,042 handoff notes were included in the analysis. Handoff notes were viewed an average of 2.34 times per shift. When aggregated among all notes, physicians viewed handoff notes most frequently (207,849 times), followed by physician assistants (46,792) and nurses (40,593). A complete breakdown of the frequency of views by common clinical roles can be found in Figure 1. The data shows that while physicians account for the majority of note views, all other roles combined account for 36.41% of views. During the interviews, all participants explained that their handoff notes were written primarily for clinicians who are covering during the evening shift. However, all three participants also acknowledged that others, such as consulting physicians or bedside nurses also read the note.

Figure 1.

Monthly Handoff Note Views Among Common Clinical Roles.

Temporally, 38.70% of handoff views occurred during handoff periods (6am to 8am and 5pm to 7pm) as shown in Figure 2. This shows that while handoff notes are most frequently viewed during handoff periods, most views do not actually occur during time periods commonly associated with handoff.

Figure 2.

Monthly Handoff Views by Hour

Relationship Between Views and Updates

To analyze the data using linear regression and ANOVA, three outlier data points (notes where updates were greater than 50 during a shift), were removed. As shown in Figure 3A, notes with more updates were also viewed more frequently. In Figure 3A, the R² value is 0.2611 with a p-value < 2.2 × 10⁻¹⁶. The data was then stratified into three groups by updates (no updates, minimal updates, and frequent updates). The ANOVA test showed that notes in the different groups had significantly different frequency of views (F = 12893, p < 2.2 × 10⁻¹⁶). Post-hypothesis testing using Tukey’s HSD test reveals that all three groups are significantly different from each other (p < 2.2 × 10⁻¹⁶ for each instance). The greatest mean difference between groups is between frequently updated notes and notes not updated during that shift (difference = 3.39 views per shift). The mean difference in note views between notes that are minimally updated and frequently updated is also similarly large.

Figure 3.

Relationship Between Handoff Views and Updates

Given that notes in the frequently updated group were viewed more frequently than notes in the other two groups, those notes were analyzed further. In this analysis, the focus shifted from the frequency of note views to the frequency of updates. Table 2 shows the frequencies of note updates for different units of the hospital. The left hand side of the table shows the top 15 units that contributed the most frequently updated handoff notes. The table shows that frequently updated notes come from many different units. The right hand side of the table shows the top 15 units where a high proportion of the handoff notes were frequently updated. Notably, several of these units are intensive care units (B09N, B09S, MICA, M8GS, M4SI, AICU) that provide care for complex and unstable patients. This finding indicates that use of handoff notes as part of the clinical workflow may be at least partially dependent on unit practices and norms and patient severity.

Table 2.

Unit Characteristics of Frequently Updated Notes

Top Units Contributing Frequently Updated Notes			Top Units with Notes that are Frequently Updated
Unit	Service(s)	Percent	Unit	Services(s)	Percent
B06T	Pediatric Cardiology/Neurology/Pulmonolgy	4.88	CH 3 NORTH ENDO	Pediatric Endoscopy	96.67
B04T	Pediatric Medical/Surgical/GI/Transplant	4.63	B09T	Pediatric Cardiac Intensive Care Unit	87.37
M6GS	Adult Medicine/Cardiology/Neurology	4.48	H5CT	Adult Cardiothoracic Intensive Care Unit	84.44
M8GS	Adult Neurologic Intensive Care Unit	4.22	B09S	Pediatric Intensive Care Unit	76.17
M5CC	Adult Cardiac Care Unit	4.01	M8GS	Adult Medicine/Cardiology/Neurology	71.42
B09T	Pediatric Cardiac Intensive Care Unit	4.01	M4SI	Adult Surgical Intensive Care Unit	67.29
M7GS	Adult Cardiology	3.79	MICA	Adult Medic Intensive Care Unit A	65.61
B05T	Pediatric Hematology/Bone Marrow Transplant	3.66	B09N	Pediatric Intensive Care Unit	60.56
M5GN	Adult Medicine/Cardiology/Neurology	3.51	AICU	Adult Intensive Care	60.16
M5CT	Adult Cardiothoracic Intensive Care Unit	3.42	B06T	Pediatric Cardiology/Neurology/Pulmonolgy	59.70
M6GN	Adult Medicine	3.38	M5CC		58.38
M9GS	Adult Medicine	3.2	B04T	Pediatric Medical/Surgical/GI/Transplant	57.42
B09S	Pediatric Intensive Care Unit	3.17	B08S	Pediatric Medical/Surgical/GI/Transplant/Neonatal Critical Care Unit	55.34
M6HN	Adult Medicine	3.07	M5CT	Adult Cardiothoracic Intensive Care Unit	54.46
M6GS	Adult Medicine	3.02	BXXX	Pediatric Emergency Medicine	52.11

Bolded units overlap on both side of Table 2.

Matched Utilization of Handoff Notes

To further analyze the use of Handoff Tool from the team perspective, the relative note activity level was coded according to the schema in Table 1. Tables 3 and 4 show that matched cases of handoff views and updates are the most prevalent (43.44%). Within matched utilization, notes were most likely to be either not used in a shift at all (0 updates and views) or used frequently during a shift (Table 3). Between the two types of mismatched utilization, notes were more likely to have relatively more views than updates during a shift (Table 4).

Table 3.

Percentage of Notes By Note Activity Level

Updates/Views	None	Minimal	Frequent
None	18.10%	12.19%	4.10%
Minimal	19.60%	8.73%	6.98%
Frequent	7.08%	6.50%	16.71%

Table 4.

Percentage of Notes By Match/Mismatch Category

Matched	Mismatched (More Updates)	Mismatched (More Views)
43.44%	23.20%	33.10%

Qualitative Analysis

The qualitative interviews provided much insight into log data findings. All three interview participants explained that they primarily wrote handoff notes to provide information for night coverage teams and consulting physicians. Participant 3 expressed that bedside nurses frequently read the participant’s handoff notes at the beginning of the shift and often asked her follow-up questions based on the handoff note. This is consistent with qualitative findings from previous work by Jiang et al. showing that intensive care unit residents are aware that other team members, such as bedside nurses, are reading their handoff notes. Participant 2 explained that, “I think that if there had been a greater awareness of others reading [the handoff note], it would have been updated more.” This suggests that awareness of other’s actions acts as a positive feedback loop in regards to updating. However, participants 1 and 2 described little to no interactions with other clinicians prompted by the handoff note, suggesting that unit practice variations impacted the use of handoff note for information sharing. Expounding further upon the differences in units, participant 2 explained that ICU teams and general medicine teams operated differently. ICU teams often operated closely, while general medicine teams operated in a more diffuse manner. This difference may also provide one explanation for why frequently updated notes are also viewed more often.

Discussion

Previous studies have suggested that nurses and ancillary services used resident handoff notes as information sources^10,13,14. Further pursuing this line of investigation, this study investigated how handoff notes are used from a team perspective, and on their possible role in facilitating team communication. To complement previous qualitative studies on handoff notes¹⁰, this study took a quantitative focus. The findings showed that handoff notes are read throughout the day by multiple care team members. Most of these note viewers were physicians or physician assistants; however, nurses also made up a noticeable number of the user base. As highlighted by the interviews, the frequency of notes updates is at least partially due to the awareness that other team members, such as consulting physicians or nurses are referring to them for information.

Currently, many clinicians rely on verbal communication in transmitting or obtaining patient information because documentation is not updated sufficiently quickly¹⁵. This study found that units with the almost critical patients tend to have notes that are frequently updated and viewed. A possible interpretation of this finding is that those units are most likely to include Handoff Tool as one method to disseminate patient information to the rest of the team. This conclusion is mirrored in the interviews, which revealed that teams in ICUs often rely on Handoff Tool for documenting and sharing information more than general medicine units. This finding suggests a possible shift towards greater utilization of documentation, particularly electronic handoff notes, to pass on patient information.

Overall the study revealed that there exists a high degree of alignment in editing/viewing practices for electronic handoff notes. This is particularly the case for intensive care units, where patients’ conditions often change rapidly throughout a shift, and where members of patient care teams must work in close alignment with each other. Previously, researchers used the term social translucence to describe the degree of awareness between individuals about their respective practices in social settings. According to Kellogg et al, being aware of intentions and actions of others helps individuals to maintain a high degree of cohesion and coordination¹⁶. This study suggests that across the hospital, and particularly for intensive care units, electronic handoff tools are contributing to social translucence among members of patient care teams. Residents update their notes more frequently knowing that others refer to these notes for updated patient information, and other members of the teams view these notes more frequently with the expectation that they are updated. Handoff notes at our institution are carried over from the previous version and act more similarly to a single note that evolves over time; rather than traditional notes which are more similar to a collection of individual notes about a patient over time. Because of this difference, alignment of note update and view frequencies is an important indicator that note writers are providing and note viewers are reviewing updated information. However, the study also showed that there are cases when these practices are not aligned, suggesting an opportunity for improvement. For example, currently, clinicians are not informed of new updates to handoff notes that might indicate important changes in patients’ conditions. One possible solution to this limitation is the addition of an RSS-style subscription of handoff note updates that could help clinicians more reliably disseminate new patient information and maintain higher level of awareness within patient care teams.

Limitations

A key limitation to the study is that it only captured views of the electronic version of the handoff note, but not the views of its printed version. During the interviews clinicians repeatedly expressed that they frequently consulted the printed version of the handoff note throughout the day in lieu of the electronic version. Another limitation in the study is the small sample of participants in the interviews. However, these interviews were primarily conducted to help interpret the quantitative findings, rather than to draw novel conclusions. Lastly, this dataset represents a small temporal period within a single academic medical center. Handoff practices constantly evolve, and not all variations of these practices were captured with the current dataset.

Conclusion

This study examined and compared patterns of updates and views of electronic handoff notes based on the usage log data. The study revealed that use of handoff notes was commonly aligned between care team members, with frequently updated notes being also frequently viewed. Moreover, frequently updated notes commonly originated from one of the intensive care units. This indicates that the electronic resident handoff note is emerging as a new method for disseminating patient information, in addition to facilitating transitions of care. Given that there is a large proportion of mismatched utilization of handoff notes, more work can be done to help teams reach a consensus about using handoff notes as an alternative information source.