Handoffs in the Intensive Care Unit: Are Off Hours a Vulnerable Time?

Abstract

Operating room (OR) to intensive care unit (ICU) handoffs are complex and known to be associated with adverse events and patient harm. The authors hypothesized that handoff quality diminishes during nights/weekends and that bedside handoff practices are similar between ICUs of the same health system. Bedside OR-to-ICU handoffs were directly observed in 2 surgical ICUs with different patient volumes. Handoff quality measures were compared within the ICUs on weekdays versus nights/weekends as well as between the high- and moderate-volume ICUs. In the high-volume ICU, transmitter delivery scores were significantly better during off hours, while other measures were not different. High-volume ICU scores were consistently better than those in the moderate-volume ICU. Bedside handoff practices are not worse during off hours and may be better in ICUs used to a higher patient volume. Specific handoff protocols merit evaluation and training to ensure consistent practices in different ICU models and at different times.

Patient care handoffs are pivotal junctures in every patient’s clinical course, as these moments define caregiver understanding of the patients’ preceding course and anticipated trajectory. Miscommunication in the exchange of this crucial information may lead to preventable morbidity and mortality.^1–8 Up to two thirds of sentinel events reported to the Joint Commission³ and nearly half of all adverse surgical events may be attributed to errors in communication, the majority of which occur during handoffs.⁴ Even malpractice claims and avoidable readmissions have been traced to information loss during handoffs.⁵

Handoffs are thus an ideal setting for quality improvement (QI) initiatives, and numerous health care organizations are focusing QI efforts on transitions of care. The Joint Commission identified improvement of communication effectiveness as one of its first National Patient Safety Goals in 2002, required implementation of standardized handoffs in 2006, and in 2009 initiated a project focused solely on improving handoff communications.^6,9–11 The US Department of Health and Human Services Agency for Healthcare Research and Quality highlights Handoffs and Signouts among its key concepts in patient safety,¹¹ and the World Health Organization named handover communication one of its High 5 patient safety initiatives in 2008.¹² In response to these priorities as well as findings in a 2008 Institute of Medicine report on resident duty hours and implications for patient safety, the Accreditation Council for Graduate Medical Education now requires residency programs to include education regarding and oversight of handoffs.^8,13,14

A potentially significant contributor to handoff-related error is the extensive variability of handoff procedures among hospitals, services, units, and even individual providers.^2,6,15–19 This variation is especially apparent in patients transitioning from the operating room (OR) to the surgical intensive care unit (ICU), at which time multiple teams (anesthesia, surgery, and critical care) face the challenge of simultaneously communicating multifaceted patient information in addition to orchestrating complex patient and equipment transfer.²⁰

Although a handful of studies have investigated an association between time of admission and hospital morbidity and mortality (the so-called weekend effect), none has identified a causal relationship.^21–26 Given that house staff and hospital workers are generally fewer during off hours and that communication errors may be more manifest with smaller teams, the research team chose to investigate handoffs of surgical ICU admissions from the OR at different times.

Accordingly, the team hypothesized that bedside OR-to-ICU handoff quality diminishes during night and weekend hours and further postulated that these practices are similar in high- and moderate-volume surgical ICUs of the same health system.

Methods

ICU Setting

The research team developed a bedside handoff assessment metric based on current literature as well as input from surgical, anesthesia, and critical care experts. After obtaining institutional review board (IRB) approval, the team carried out a prospective observational double-cohort study in order to assess OR-to-ICU bedside handoffs in 2 surgical ICUs of different hospitals within the same tertiary care university health system, wherein both ICUs are staffed by the same surgery and anesthesia residents and attending intensivists.

ICU₁ averages 210 admissions per month (high volume) and includes patients from trauma, general, vascular, transplant, obstetrics and gynecology, orthopedic, oncology, otorhinolaryngology, plastic, and urologic surgery services. During weekday hours, ICU₁ is divided into 2 teams managing 10 to 12 patients each. The weekday teams comprise a critical care attending, 1 to 2 critical care fellows, 1 to 2 nurse practitioners, and 2 to 3 residents. During night and weekend hours, the ICU₁ teams merge and comprise one critical care fellow and 2 residents.

ICU₂ averages 125 admissions per month (moderate volume) and includes patients from general, vascular, orthopedic, and cardiac surgery services. During weekday hours, the ICU₂ team manages 10 to 14 patients with a critical care attending and a junior resident. Cardiac surgery patients are additionally managed by a cardiac surgery attending, a cardiac surgery physician assistant, and occasionally an anesthesia fellow, while noncardiac surgery patients are occasionally managed by an additional provider (nurse practitioner or anesthesia/surgery resident of variable training level). During night and weekend hours, ICU₂ is managed by 1 to 2 residents, hospitalists, or physician assistants.

Handoff Assessment Tool

Using a previously developed handoff assessment tool,²⁷ trained research staff practiced observing OR-to-ICU handoffs with physician oversight. The handoff assessment tool incorporates a checklist of items essential to reports from the transmitting OR team to the receiving ICU team, including past medical history, reason for ICU admission, allergies, airway, breathing/ventilation, circulation/hemodynamics, inputs, outputs, drains/lines, complications, plan, team contact information, and family information. Individual items were checked “Yes” as long as any mention of that item was made. The start and end times of each report were recorded. The surgeon (attending, resident, fellow) or anesthesiologist (attending, resident, fellow, nurse anesthetist) transmitting the report (the transmitter) was scored on the quality of his or her delivery as superior, satisfactory, or unsatisfactory. Superior delivery was defined as concise, clear, and organized, and included confirming receiving provider understanding and/or eliciting and fully answering receiving provider questions. Satisfactory delivery was defined as mostly concise and clear with adequately answered receiver questions. Unsatisfactory delivery was defined as hurried, disorganized, distracted, confusing, or vague. The ICU provider (fellow, resident, nurse practitioner) receiving the report (the receiver) was scored using a modified version of a validated scoring instrument assessing recipient listening: eye contact, affirmatory statements, head nodding, note taking, and question asking.²⁸ An attribute was checked “Yes” as long as that attribute was demonstrated one or more times during the handoff. Teamwork during the bedside handoff was scored as superior, satisfactory, or unsatisfactory. Superior teamwork was efficient, coordinated, and cooperative; the handoff reports took place only after physical transfer of the patient and equipment and with any immediate patient needs taken care of first, and the reports were not interrupted; both surgery and anesthesia transmitters were present and listening during the other’s report. Satisfactory teamwork was mostly efficient, coordinated, and cooperative; the handoff reports may have started during physical transfer of the patient, and the reports involved limited or no interruption; both surgery and anesthesia gave reports, but they may not have been present or actively listening to the other’s report. Unsatisfactory teamwork was inefficient, uncoordinated, or uncooperative; it may have been unclear who was transmitting and/or receiving reports, multiple conversations may have occurred simultaneously, or providers devoted attention to other tasks during the reports. Professionalism also was scored on a superior, satisfactory, or unsatisfactory basis. Superior professionalism entailed only appropriate discussion and behavior during the handoff. Satisfactory professionalism included minor distractions such as laughter or unrelated conversation. Unsatisfactory professionalism involved inappropriate comments or actions such as joking about the patient. Finally, the number of people in the room during the handoff (excluding the patient) and whether or not the receiving provider performed any physical examination of the patient also were noted.

Bedside Handoff Observations

As described elsewhere,²⁷ trained research staff directly observed OR-to-ICU bedside handoffs at all times of day. Research staff consisted of 4 students who were trained in observational assessment and who practiced observing OR-to-ICU handoffs with physicians prior to the beginning of the study. In order to maintain scoring consistency in the portions of the metric requiring subjective assessment, observations were piloted by research staff in pairs to promote discussion and agreement on scoring technique that was subsequently verified by an attending physician. On patient arrival to their ICU room, a single research staff member dressed in scrubs observed the bedside handoff practice from within the patient room, taking detailed field notes and audio-recording surgery and anesthesia reports using a miniature pen recorder. As per the study IRB, all observations and recordings were exempted from consent, were used exclusively for this observational purpose, and were destroyed within a month of collection. Immediately after the reports ended, the research staff member then scored the bedside handoff using the aforementioned metric to grade content transfer, communication, teamwork, and professionalism. Research staff reviewed their field notes and audio-recordings to ensure accurate completion of the metric and also reviewed field notes together to ensure consistency in evaluation of scores graded on an unsatisfactory-satisfactory-superior scale. A small subset of handoffs was assessed by 2 research staff during the training process. In these cases, the assessment by the research staff member who had evaluated more handoffs overall was used for data analysis.

This study was conducted in concert with a companion pre-post study of ICU handoff standardization.²⁷ Sample size for both studies was selected based on the number of observations needed to reach thematic saturation, the point at which additional observations add no new information.²⁹ In the researchers’ experience, saturation is generally reached within 15 to 20 observations, a somewhat smaller number than that needed for saturation with interviews.³⁰ The strata for analysis were (a) the ICU (#1 vs #2) and (b) time of handoff (weekday vs night/weekend). Therefore, the target sample size was 48 to 60 observations. The observed handoffs are a convenience sample based on the days and nights during which research staff were available in the ICU. The primary outcome was the quality of transmitter characteristics during the handoff.

Data Analysis

For data collected from within each ICU, characteristics of handoffs occurring during weekday hours (Monday-Friday, 0700–1700) were compared to those of handoffs occurring during all other times. The Mann-Whitney U test was used to compare mean scores between groups in the following categories: number of content items omitted from the reports, total duration of the handoff reports, number of passive (eye contact, head nodding, and affirmative statements) and active listening skills (note taking and question asking) exhibited by the receiver, and number of people in the room during the handoff. Pearson’s χ² test was used to compare both ICUs for percentage of ratings considered satisfactory or better for transmitter delivery quality, teamwork, and professionalism. Pearson’s χ² test also was used to compare both ICUs for the presence or absence of physical examination by the receiver. Because 2 reports are expected to occur in an OR-to-ICU handoff (one from surgery and one from anesthesia), a given handoff could receive up to 2 scores each for passive listening skills, active listening skills, and transmitter delivery quality. Means for these measures also were compared between the 2 ICUs using the same statistical tests. A P value of <.05 was considered statistically significant.

Results

A total of 58 bedside handoffs observed over 11 weeks were needed to achieve thematic saturation with respect to handoff quality, teamwork, and professionalism: 40 handoffs in ICU₁ and 18 in ICU₂.

ICU₁ (High Volume)

In ICU₁, 11 weekday handoffs (20 reports) were compared to 29 weeknight and weekend handoffs (54 reports). Satisfactory or better transmitter delivery scores were significantly more frequent on nights and weekends (91%) than on weekdays (65%, P < .01). Other metrics also tended to be better on night/weekend hours, but these differences were not significant (Figure 1).

Figure 1.

ICU₁ OR-to-ICU handoffs on weekdays versus nights/weekends.

Comparison of handoff metrics in high-volume ICU₁ on weekdays versus nights/weekends. (A) Histograms indicate the proportion of handoffs in which a physician or advanced practitioner on the ICU team performed a physical examination of the patient (Physical Exam), and the proportion of handoffs rated satisfactory or better for teamwork, professionalism, and transmitter (surgeon or anesthesiologist) delivery. Error bars represent SEM. ** versus night and weekends. P < .01. (B) Histograms indicate the mean number of content items omitted from the handoff reports (Content Omit), total length of combined anesthesia and surgery reports (Length [minutes]), number of people in the room during the handoff (Number in Room), number of passive listening skills (maximum = 3) demonstrated by the provider receiving the handoff (Pass List), and number of active listening skills (maximum = 2) demonstrated by the provider receiving the handoff (Act List). For all values in the figure, higher scores are better, except for “Content Omit,” in which a lower score indicates better performance. Error bars represent SEM.

Abbreviations: ICU, intensive care unit; OR. operating room; SEM, standard error of the mean.

ICU₂ (Moderate Volume)

In total, 13 weekday handoffs (16 reports) were compared to 5 weeknight handoffs (7 reports). No weekend handoffs were observed in ICU₂, because of the rare occurrence of weekend OR-to-ICU transfers in ICU₂. Differences between weekday and weeknight hours were much smaller for all measures, none of which reached statistical significance (Figure 2).

Figure 2.

ICU₂ OR-to-ICU handoffs on weekdays versus weeknights.

Comparison of handoff metrics in moderate-volume ICU₂ on weekdays versus nights. Categories are the same as described in Figure 1.

Abbreviations: ICU, intensive care unit; OR, operating room.

ICU₁ Versus ICU₂

A total of 26 weekday and weeknight handoffs (48 reports) in ICU₁ were compared to 18 weekday and weeknight handoffs (23 reports) in ICU₂. As there were no observed weekend handoffs in ICU₂, weekend handoffs in ICU₁ were excluded from this comparison. All but one category was scored higher in ICU₁ as compared to ICU₂ (Figure 3). ICU₁ had fewer than half as many content omissions as ICU₂ (P < .001) and teamwork scores rated satisfactory or better 5.9 times more often than in ICU₁ (P < .001).

Figure 3.

OR-to-ICU handoffs in ICU₁ versus ICU₂.

Comparison of handoff metrics between high-volume ICU₁ and moderate-volume ICU₂. Categories are the same as described in Figure 1.

*P < .05 versus ICU₂, **P < .01 versus ICU₂.

Abbreviations: ICU, intensive care unit; OR, operating room.

Discussion

OR-to-ICU bedside handoff quality was similar or superior during night and weekend shifts as compared to weekday daytime shifts in a high-volume academic surgical ICU. There also was a trend toward better teamwork and professionalism, and a more frequent hands-on exam by the receiving team. OR-to-ICU handoff quality was significantly different in 2 surgical ICUs within the same health system, despite staffing with overlapping sets of physician providers.

The finding that OR-to-ICU bedside handoff quality may be better during off hours is unexpected, as the known reduction in personnel volume available during those times would intuitively imply otherwise. That this finding is seen only in the high-volume ICU is also notable, as the drop in staff-to-patient ratio between days and nights/weekends is more prominent in the high-volume ICU than in the moderate-volume ICU.

Although the present study only looks for association rather than causality, one still is compelled to ask why a relative decrease in staffing ratio could lead to improvement in bedside handoff quality. First, one must consider whether the night and weekend personnel are trained differently or have different levels of expertise in bedside handoffs than personnel who work during regular daytime hours. In ICU₁, the ICU receivers are medical team members intermittently rotating on day or night schedules. As such, time of day should not confer any difference in handoff training, and any differences between individual providers should be accounted for by the use of mean scores for analysis.

On the other hand, the OR transmitters on nights and weekends may possess higher levels of training than their weekday counterparts, as the surgery and anesthesia house staff at those times are often midlevel or senior residents responsible for covering the entire operative ease, whereas the weekday transmitters can be more junior house staff who are only partially involved in the operative case and may be less familiar with salient intraoperative details. This is particularly noteworthy, as the only metric within ICU₁ that achieved significance was the transmitter delivery score.

Another explanation for the difference noted in bedside handoff quality between working and off hours includes the possibility that the relative drop in volume of surgical cases and other patient interventions from weekdays to nights/weekends allows both transmitting and receiving staff to spend more time and effort on each bedside OR-to-ICU handoff. One also could argue that with fewer personnel present, each team member bears a greater capacity and sense of responsibility to focus on information transfer, as they cannot rely on others to capture the same information. The phenomenon of individual members of small teams exerting more effort with better collaboration than equivalent members of larger teams was first demonstrated by Ringelmann in 1913,^31,32 and since has been widely captured in psychology, economics, and business literature, where there is an increasing focus on identifying the features of the most efficacious teams.^31–34 Numerous authors have concluded that smaller teams are in fact more effective, with the caveat that the ideal size and lower limit of any given team depends on the task involved.^31–34 Since the Institute of Medicine first highlighted the impact of medical errors and lessons to learn from industry in 2000, the medical world has become more attentive to cultivating high-quality teamwork,³⁵ though optimal team sizes in different services and care settings remain difficult to define.

The present comparison between different volume ICUs revealed a significant difference in bedside handoff quality in most parameters studied. This may point to the value of repetition in engendering consistent technique in handoff practice in the higher volume setting. It is also possible that overall handoff strategies in ICU rely more heavily on non–bedside forms of information transfer, such as by phone, electronic medical record, or face-to-face communication in a separate room. The post hoc evaluation of these alternate practices found that such non–bedside handoffs often occurred in ICU₂. However, the researchers could not determine whether this was consistent for all their patient handoffs, or how this compared to the use of non–bedside handoffs in ICU₁. Last, with regard to team structure, consistency of team members has been demonstrated in multiple nonmedical settings to positively impact team effectiveness.³³ It is notable that team staffing is more irregular in ICU₂, where providers often inconsistently fill shifts rather than being stable members of the ICU team for weeks to months at a time, as are the providers in ICU₁.

Overall, the findings of the current study suggest some opportunities to help create safer and more efficient ICU provider handoffs. Regarding providers handing over patients to the ICU, perhaps junior house officers might benefit from practicing transfers alongside their senior colleagues to potentially learn better technique. As to the teams giving or receiving a handoff, optimal team size may need to be further investigated, and greater emphasis could be placed on improving continuity of transmitting and receiving team members. Higher volume ICUs may provide the advantage of repeated practice with handoffs, but lower volume ICU models might benefit from implementing more protocolized handoffs following a checklist. The I-PASS Study Group recently demonstrated the profound positive impact of protocolized handoff training on handoff quality as well as patient outcomes, decreasing medical errors by 23% and preventable adverse events by 30%.¹⁶ Multiple other studies have demonstrated similar improvements with standardized handoff training in the OR-to-postanesthesia care unit and OR-to-ICU setting.^2,36–40

As an observational trial, the present study has some important limitations. First, as the study aimed simply to characterize the current quality of bedside handoffs, the researchers did not investigate the current approach to handoff training or use of handoff tools at the study institution, but the results suggest an opportunity to tackle this in the future. Second, there is inherent variation and subjectivity of handoff scoring despite using a well-constructed handoff scoring tool. The researchers attempted to assuage this variability by incorporating paired pilot observations and scoring field note discussion into the iterative scoring metric development process. Also, the scorers clearly could not be blinded to the location of their observation, thereby adding to the potential for observer bias to confound results. In addition, despite training and supervision by an attending physician, research staff were students and not surgical or anesthesia experts, and as such may not have known the subtleties of transmitting essential and less essential medical information when scoring subjects. Nonetheless, this bias would have occurred to the same extent in both ICUs and across all time frames. Last, the number of observed handoffs was smaller in ICU₂ than in ICU₁, in part because of reduced admission volume in that ICU. Nevertheless, this study remains one of the largest comparative studies of handoffs across sites and across admission times.

Conclusions

Contrary to the primary hypothesis, OR-to-ICU bedside handoff practices appear to be no worse during off hours as compared to weekdays, and are potentially slightly better during off hours in a high-volume academic surgical ICU. Additionally, handoff practices vary significantly between 2 surgical ICUs within the same health system despite involving many of the same attending surgeons, anesthesiologists, and their respective house staff. Specific handoff practices across different ICU models merit evaluation and implementation of team training to ensure consistent ICU practices at different times and different locations.