Abstract
Purpose
To analyze in-room video recordings of operating room (OR) to intensive care unit (ICU) handoffs to determine tempo and quality of team interactions on nights and weekends compared with weekdays, and to demonstrate how existing telemedicine technology can be used to evaluate handoffs.
Method
This prospective observational study of OR-to-ICU bedside handoffs was conducted in the surgical ICU of the Hospital of the University of Pennsylvania in July 2014–January 2015. Handoff video recordings were obtained for quality improvement purposes using existing telemedicine cameras. Evaluators used adapted validated in-person assessment measures to analyze basic characteristics and quality measures (timing, report types, report duration, presence of physical exam, teamwork skills, engagement, report delivery skills, listening skills, interruptions, unprofessional comments or actions).
Results
Sixteen weekday and 16 night and weekend handoffs were compared. There were no significant differences in basic characteristics. Most quality measures were similar on weekdays compared with nights and weekends. Surgeons demonstrated better report delivery skills and engagement on nights and weekends (P = .002 and P = .04, respectively), whereas OR anesthesiologists’ scores were similar during both time frames.
Conclusions
This study presents a novel approach of assessing handoff quality in OR-to-ICU handoffs using an existing telemedicine infrastructure. Using this approach, quality measures of night and weekend handoffs were found to be no worse— and sometimes better—than those during weekdays. Video analysis may emerge as an ideal unobtrusive quality improvement methodology to monitor handoffs and improve education and compliance with institutional handoff policies.
Communication failures among busy hospital services, particularly those occurring during patient handoffs, are a major source of preventable adverse events in health care.1–5 The Joint Commission recently reported that nearly two-thirds of all sentinel events, or unexpected patient safety events causing death or serious harm, can be attributed to failures in communication.4 Among such communication breakdowns in surgical departments, nearly two-thirds involve an inadequate handoff.3 Since 2011, the Accreditation Council for Graduate Medical Education (ACGME) has required residency training programs to provide formal housestaff education in handoffs, and to use a system to monitor handoff quality.6 Even though evidence-based reports on how to fulfill these requirements are numerous,5,7–10 recommendations remain limited, and universally accepted methods are lacking.
The day-to-day functioning of surgical services is noteworthy in that it often involves care in disparate hospital locations including the emergency department, the operating room (OR), the postanesthesia care unit (PACU), and the surgical intensive care unit (ICU). Surgical team members often have multiple demands for their time and attention, and therefore OR-to-ICU handoffs often compete with issues elsewhere. This observation is important as OR-to-ICU handoffs are complex, involving communication among multiple teams (surgery, anesthesia, critical care, respiratory care, and nursing). In addition, these transitions often involve the simultaneous physical transfer of both patient and equipment.11 Indeed, studies have shown OR-to-ICU handoffs to be error prone and associated with inappropriate multitasking, poor teamwork and communication, and distracting interruptions.12 Some studies have drawn analogies between handoffs and tasks in other complex, multiprofessional settings such as Formula 1 motor racing or square-rigger sailing, suggesting that the prioritization of safety and teamwork that has worked to reduce errors in these settings could serve as an example of how a similar approach might benefit patient care.13,14
Independently, a number of studies have investigated the concept of the “weekend effect.” This theory suggests that patients presenting for care outside of regular working hours are at risk for greater morbidity and mortality,15–21 particularly for conditions with high baseline case fatality rates.21 Specifically, off-hour presentation of patients with myocardial infarction or stroke has been associated with higher risk-adjusted mortality than weekday presentation.16,18 Although the causal relationship remains unclear, the authors of these studies propose numerous contributing factors to explain these findings, including staffing, provider coverage practices, resources, and intensity of care.15,16,18,19,21 Other fields such as trauma surgery may not suffer from this diurnal variation partly because of their standardized 24/7 care.20
The increasing prevalence of telemedicine with in-room video cameras offers an appealing opportunity to capture handoffs and evaluate multiple quality measures of OR-to-ICU handoffs. In this study, we analyzed in-room video recordings of real OR-to-ICU handoffs to determine the tempo and quality of team interactions. We sought to demonstrate how existing telemedicine technology can be used to evaluate handoffs, and we hypothesized that handoff quality would be worse on nights and weekends than during regular weekday hours.
Method
Setting
The Hospital of the University of Pennsylvania is a 700-bed, quaternary, urban teaching hospital. Its 24-bed surgical ICU cares for patients originating from a variety of primary surgical services, excluding cardiac and neurological surgery patients. The ICU care team is composed of registered nurses; critical care fellows; surgical, emergency medicine, and/or anesthesia residents; attending intensivists (anesthesiologists or trauma surgeons); and advanced practice providers (APPs). The ICU averages 210 admissions per month. During weekday hours (Monday–Friday, 0700–1700), the ICU care team is divided into two teams managing 10 to 12 patients each; each team comprises an attending intensivist, 1 to 2 critical care fellows, 1 to 2 APPs, and 2 to 3 residents. During night and weekend hours, there is one ICU care team that comprises 1 critical care fellow and 2 residents. During both weekday and night and weekend hours, the nurse- to-patient ratio in the ICU is either 1:1 or 1:2, with the precise ratio determined by patient acuity.
Handoff education
Per ACGME requirements, all new housestaff undergo a didactic handoff curriculum organized at the departmental level that differs among specialties and is not ICU specific in content, format, or location. APPs and nurses do not receive formal handoff training apart from what they received in physician assistant or nursing school or from observing their unit peers.
Telemedicine services and performance improvement video recordings
Each surgical ICU patient room is equipped with a camera and microphone, which transmit a high-fidelity video and audio signal to the University of Pennsylvania Penn E-lert remote monitoring service housed in a separate location. The cameras are located above the foot of the patient’s bed, mounted on the wall close to the ceiling. In December 2008, the hospital established a performance-improvement-driven institutional policy that allows video recording of any ICU bedside event requiring the immediate presence of a physician or APP through the existing bedside telemedicine technology. The majority of recordings are initiated at the discretion of the bedside nurse but can also be triggered by any ICU or telemedicine provider. Recorded events most often involve cardiac arrests, procedures, and rapid deteriorations but also frequently include admissions and readmissions of patients to an ICU room. Unit nurses are encouraged to record all such events, and the practice of recording events is prevalent, with an average of two to five recordings per day in the ICU. All ICU care providers are aware that video recording could occur in any room and at any time to monitor events related to patient care but are not alerted that they are actively being recorded prior to or during the recordings. The hospital obtains consent for video recordings from the patient or a legally authorized representative as part of a global ICU admission consent form, which encompasses the admission, bedside procedures, telemedicine monitoring, and video recording for quality improvement (QI) purposes.
Per institutional QI policy, these digital recordings are stored on a secure, encrypted computer server only accessible to QI officers. Unrelated to this study, a selection of these recordings is regularly presented at monthly multidisciplinary video review conferences under the auspices of QI activities. Per the institution’s policy, all video recordings must be destroyed within 28 days of the recording date.
When videos of admissions or readmissions from the OR are obtained, they are initiated on the patient’s entry into the room and last for 45 minutes, regardless of in-room activity. In the first recorded minute, the camera is positioned facing the door to accurately capture who was entering and exiting the room during this high-traffic interval immediately following patient entry. After the first minute and thereafter, the camera is positioned to face the patient’s bed. Every 30 seconds the camera pans once around the room to better capture all events in the room. During the last 30 minutes of recordings, camera pans occur once every minute.
Only recordings of handoffs involving patients who came directly from the OR were included in the study.
Selection of handoff recordings
We obtained institutional review board approval from the University of Pennsylvania to review only recorded admissions and readmissions from the OR. This approval allowed the project to be conducted exempt from additional consent as the recordings were obtained under the auspices of QI activities and under the argument that acquiring consent from every provider, visitor, and allied personnel that could momentarily appear in the recorded segments would not be practicable before or after the event was recorded.
From July 2014 to January 2015, study personnel (M.E.B., D.Z.) rotated per their schedule availability and accessed all admission and readmission videos that were on the server prior to deletion by a QI officer per institutional policy. In total, this amounted to a convenience sample of 32 recordings of OR-to-ICU bedside handoffs available for study that M.E.B. and D.Z. scored in a prospective observational manner (see below).
Video-recorded handoff assessments
Video review of handoffs differs from in- person observation and scoring in terms of the information that can be easily captured, so we focused our assessment on characteristics and quality measures such as timing, teamwork skills, and communication skills. As there are no validated handoff assessment tools for video-recorded, naturally occurring (i.e., not simulated) handoffs, we adapted validated in-person assessment measures from a combination of published reports as detailed below.
Two of the authors evaluated video footage offline: a senior medical student with previous training and experience in OR- to-ICU handoff assessment (M.E.B.), and a senior critical care APP whom M.E.B. trained (D.Z.). The evaluators were not blinded to the time of handoffs because the presence of windows on camera provided a general sense of day versus night.
M.E.B. analyzed all videos for basic handoff characteristics, including number of people in the room in the first recorded minute; total time spent in the room by the OR anesthesia provider; total time spent in the room by the surgery provider; time interval after patient entry at which all providers (ICU physician [intensivist, fellow, or resident] or APP, surgery, and OR anesthesia providers) had left the room (defined as no provider returning within five minutes of leaving); time interval after patient arrival to commencement of first verbal handoff report; total duration of handoff reports, as well as the duration of the surgery provider’s and OR anesthesia provider’s reports; the presence or absence of a report from the surgery provider to an ICU provider (ICU physician or APP) or nurse; the presence or absence of a report from the OR anesthesia provider to an ICU provider or nurse; and the presence or absence of a physical examination by an ICU provider or nurse (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A430).
Both evaluators independently analyzed the recorded videos for measures of handoff quality, including teamwork skills; report delivery skills (scored separately for the surgeon and anesthesiologist); passive and active listening skills of the ICU provider receiving each report; provider and nursing engagement in the handoff; number of interruptions; and number of unprofessional comments or actions during the report. See Table 1 for the operational definitions for teamwork skills, report delivery skills, and engagement; we adapted these definitions from performance criteria used in a prior study by Catchpole et al.13 For a given handoff, the evaluators gave each teamwork skill a score of 1 if it was met or 0 if it was not met, and calculated a sum of the teamwork skills (maximum = 5). Evaluators rated report delivery skills and engagement as superior, satisfactory, or unsatisfactory, per the descriptions in Table 1. The evaluators also assessed the presence or absence of listening skills using a validated scoring instrument for physician listening behaviors, including eye contact, affirmatory statements, head nodding, note taking, and question asking and/or interactive comments (e.g., read-backs).22 Each handoff could have up to two reports—one each from surgeon and the OR anesthesiologist—and the evaluators scored these reports separately for listening skills. Finally, the evaluators tallied the number of interruptions (pages, messages, phone calls, or side comments that interfered with handoff continuity) and unprofessional comments or actions (inappropriate comments about patients or staff, or distractions such as laughter or unrelated personal conversation) (see Supplemental Digital Appendix 1 at http://links.lww.com/ACADMED/A430).
Table 1.
Operating Room to Intensive Care Unit Handoff Quality Measures, Used in a Study of Surgical Service Handoffs, Hospital of the University of Pennsylvania, July 2014–January 2015
| Handoff quality measure | Operational definitiona |
|---|---|
| Teamwork skillsb | |
|
| |
| Task prioritization | Equipment transfer, physical handover of patient, and immediate patient needs are taken care of prior to the handoff of information. |
|
| |
| Clear role identification | The transmitter(s) and receiver(s) of the handoff clearly identify themselves as such. Introductions may not always have been necessary given that many providers are familiar with each other, but it should be clear that the transmitter knows who is responsible for taking over patient care and speaks directly to that person. |
|
| |
| Attention to reports | The transmitter(s) and receiver(s) of the handoff attend solely to the conversation during the information handover. They are not engaging in other tasks or side conversations. |
|
| |
| Coordination of reports | The reports from anesthesia and surgery either occurred together as a team or sequentially. They did not occur simultaneously in different parts of the room. |
|
| |
| Transmitter collaboration | The anesthesiologist and surgeon were present in the room and actively listening to the other’s report. |
|
| |
| Report delivery skills | Superior: The transmitter uses concise, clear, organized speech, and confirms receiving provider understanding and/or elicits and fully answers questions. |
| Satisfactory: The transmitter uses mostly concise and clear speech, and adequately answers questions. | |
| Unsatisfactory: The transmitter uses hurried, disorganized, distracted, confusing, or vague speech, and does not answer questions adequately. | |
|
| |
| Engagement | Superior: The individual appeared interested and interactive, faced the transmitter or receiver, and paid attention throughout the handoff. |
| Satisfactory: The individual appeared somewhat interested and interactive, at times faced the transmitter or receiver, and was paying attention for most or all of the handoff. | |
| Unsatisfactory: The individual appeared disinterested, was not interactive, did not face the transmitter or receiver, and/or was not paying attention. | |
The authors adapted these definitions from performance criteria used in a prior study by Catchpole et al.13
Each teamwork skill was scored as present (score of 1) or absent (score of 0), and a sum was calculated (maximum = 5).
M.E.B. evaluated all 32 videos for these quality measures. D.Z. only evaluated 10 of these videos because of decreased availability to evaluate videos prior to their deletion per institutional policy. Both evaluators followed the aforementioned handoff evaluation guide with a list of the operational definitions to improve consistency in evaluating handoffs (see Table 1).
Data analysis
We compared mean scores of basic characteristics and quality measures of handoffs occurring during weekday hours (Monday–Friday, 0700–1700) with those occurring at night and weekend hours. Of note, late-night handoffs (those occurring after 2400) were uncommon. For basic handoff characteristics, we used the Mann–Whitney U test to compare means for continuous variables (number of people in the patient’s room and durations). For measures of time and handoff report duration, we excluded outliers greater than two standard deviations from the mean. This resulted in the exclusion of no more than two data points per comparison. We used Pearson chi-square test to compare categorical variables (types of handoff reports, number of handoffs in which a physical examination was performed by the ICU provider or nurse).
For handoff quality measures, 10 of the handoffs were scored by both evaluators (see above); for these handoffs, we used averages for measures in which the two evaluators assigned different scores. We used Pearson chi-square test to compare categorical variables (number of handoffs meeting a given teamwork or listening skill, number of handoffs with satisfactory or superior ratings for report delivery skills and engagement). We used Fisher exact test in place of Pearson chi- square test in cases where an expected frequency in the chi-square was 5 or lower. We used the Mann–Whitney U test to compare the sum of teamwork skills met and the number of interruptions and unprofessional comments and actions. We calculated P values using the exact two-tailed significance and considered a P value < .05 significant. We used SPSS Statistics Version 23 (IBM, Armonk, New York) for all data analyses.
Results
Basic handoff characteristics
We compared 16 weekday handoffs to 16 night and weekend handoffs. There were no significant differences in basic characteristics between the two groups (Table 2). The mean duration of the surgeon’s handoff report tended to be 36% longer on nights and weekends (mean 2 minutes 8 seconds) than on weekdays (mean 1 minute 34 seconds), but this failed to reach significance (P = .10).
Table 2.
Basic Characteristics of Operating Room to ICU Handoffs on Weekdays Versus Nights and Weekends, Surgical Service, Hospital of the University of Pennsylvania, July 2014–January 2015
| Basic handoff characteristic | Weekdays (n = 16) | Nights and weekends (n = 16) | P value |
|---|---|---|---|
| Number of people in room, mean (SD) | 6.8 (1.4) | 6.8 (1.4) | .95 |
| Total time (in mm:ss) in room, mean (SD) | |||
| Anesthesia | 09:21 (04:05) | 08:23 (02:31) | .71 |
| Surgery | 03:26 (02:49) | 04:15 (02:44) | .37 |
| Receiving ICU provider | 08:38 (07:36) | 08:54 (06:59) | .89 |
| Time (in mm:ss) to first handoff report, mean (SD) | 01:11 (01:01) | 01:15 (00:55) | .55 |
| Time (in mm:ss) to room empty of providers, mean (SD) | 13:14 (08:13) | 11:53 (05:52) | .96 |
| Handoff report duration (in mm:ss), mean (SD) | |||
| Anesthesia | 03:26 (01:55) | 02:58 (01:52) | .53 |
| Surgery | 01:34 (01:21) | 02:08 (01:09) | .10 |
| Total | 04:42 (02:39) | 05:20 (02:41) | .53 |
| Handoff report type, no. (%) | |||
| Anesthesia to ICU provider | 9 (56) | 9 (56) | — |
| Anesthesia to nurse | 13 (81) | 13 (81) | — |
| Surgery to ICU provider | 8 (50) | 10 (63) | .48 |
| Surgery to nurse | 8 (50) | 12 (75) | .14 |
| Handoff with patient physically examined, no. (%) | |||
| By ICU provider | 8 (50) | 7 (44) | .72 |
| By nurse | 10 (63) | 9 (56) | .72 |
| By either ICU provider or nurse | 12 (75) | 13 (81) | .67 |
Abbreviations: ICU indicates intensive care unit; SD, standard deviation; mm, minutes; ss, seconds; ICU provider, ICU physician (intensivist, fellow, or resident) or advanced practice provider.
Handoff quality measures
We compared 16 weekday handoffs (with 28 reports) to 16 night and weekend handoffs (with 29 reports). The teamwork, engagement, and report delivery skills quality measures were not found to be inferior on nights and weekends compared with weekdays (Table 3). Notably, on nights and weekends, handoff report delivery skills were more likely to be rated as satisfactory or superior (P = .03); this was primarily due to reports delivered by the surgeon (P = .002), as OR anesthesiologists’ delivery skills were scored similarly for both time frames. There was also a trend toward improved engagement among providers on nights and weekends (P = .07), again largely due to higher scores among the surgeons during this time (P = .04), as OR anesthesiologists’ engagement was scored similarly for both time frames.
Table 3.
Quality Measures of Operating Room to ICU Handoffs on Weekdays (n = 16) Versus Nights and Weekends (n = 16), Surgical Service, Hospital of the University of Pennsylvania, July 2014–January 2015
| Handoff quality measure | Weekdaysa | Nights and weekendsa | P value |
|---|---|---|---|
| Teamwork skills, no. (%)b | |||
| Task prioritization | 3/16 (19) | 3/16 (19) | — |
| Clear role identification | 13/16 (81) | 13/16 (81) | — |
| Attention to reports | 7/16 (44) | 9/16 (56) | .18 |
| Coordination of reports | 10/14 (71) | 14/16 (88) | .16 |
| Transmitter collaboration | 0/14 (0) | 3/16 (19) | .23 |
| Sum, mean (SD) | 2.1 (1.0) (n = 14) |
2.8 (1.3) (n = 16) |
.13 |
| Engagement, no. (%)c | |||
| Anesthesia | 14/16 (88) | 14/16 (88) | — |
| Surgery | 7/15 (47) | 13/16 (81) | .04 |
| ICU provider | 8/10 (80) | 10/10 (100) | .47 |
| Nurse | 7/16 (44) | 9/16 (56) | .58 |
| All | 36/57 (63) | 46/58 (79) | .07 |
| Report delivery skills, no. (%)c | |||
| Anesthesia | 9/14 (64) | 8/15 (53) | .61 |
| Surgery | 4/14 (29) | 12/14 (86) | .002 |
| All | 13/28 (46) | 20/29 (69) | .03 |
| Listening skills, no. (%)b | |||
| Eye contact | 28/29 (97) | 28/33 (85) | .42 |
| Affirmatory statements | 25/29 (86) | 28/32 (88) | .37 |
| Head nodding | 18/24 (75) | 25/29 (86) | .09 |
| Note taking | 23/31 (74) | 23/33 (70) | > .99 |
| Question asking and/or interactive comments | 25/31 (81) | 24/31 (77) | .24 |
| Number of interruptions, mean (SD) | 0.8 (0.8) | 0.6 (0.6) | .45 |
| Number of unprofessional comments or actions, mean (SD) | 0.3 (0.5) | 0.3 (0.5) | .84 |
Abbreviations: ICU indicates intensive care unit; SD, standard deviation; ICU provider, ICU physician (intensivist, fellow, or resident) or advanced practice provider.
Denominators are listed in each cell. Numbers less than the total number of handoffs are due to missing transmitters or receivers or inability to evaluate a skill accurately.
Numbers and percentages reflect presence of skill. Listening skills are scored separately for reports from the surgeon and the operating room anesthesiologist.
Numbers and percentages reflect ratings of satisfactory or superior.
Because 6 of the 16 handoffs in both the weekday and night and weekend time frames involved a handoff received by a nurse without a receiving ICU physician or APP present, we performed a subgroup analysis excluding those handoffs (Table 4). The results remained similar with better surgeon report delivery skills on nights and weekends (P = .03), while other measures remained noninferior.
Table 4.
Operating Room to ICU Handoff Quality Measures on Weekdays (n = 10) Versus Nights and Weekends (n = 10) for Handoffs With an ICU Provider Present, Surgical Service, Hospital of the University of Pennsylvania, July 2014–January 2015
| Handoff quality measure | Weekdaysa | Nights and weekendsa | P value |
|---|---|---|---|
| Teamwork skills, no. (%)b | |||
| Task prioritization | 1/10 (10) | 1/10 (10) | — |
| Clear role identification | 7/10 (70) | 9/10 (90) | .58 |
| Attention to reports | 7/10 (70) | 7/10 (70) | — |
| Coordination of reports | 5/9 (56) | 9/10 (90) | .05 |
| Transmitter collaboration | 0/9 (0) | 3/10 (30) | .21 |
| Sum, mean (SD) | 2.1 (1.3) (n = 9) |
3.1 (1.1) (n = 10) |
.12 |
| Engagement, no. (%)c | |||
| Anesthesia | 10/10 (100) | 10/10 (100) | — |
| Surgery | 7/9 (78) | 9/10 (90) | .58 |
| ICU provider | 8/10 (80) | 10/10 (100) | .47 |
| Nurse | 4/10 (40) | 5/10 (50) | .71 |
| All | 29/39 (74) | 34/40 (85) | .38 |
| Report delivery skills, no. (%)c | |||
| Anesthesia | 6/8 (75) | 7/10 (70) | > .99 |
| Surgery | 4/9 (44) | 9/9 (100) | .03 |
| All | 10/17 (59) | 16/19 (84) | .06 |
| Listening skills, no. (%)b | |||
| Eye contact | 20/20 (100) | 21/22 (95) | > .99 |
| Affirmatory statements | 15/18 (83) | 19/21 (90) | .32 |
| Head nodding | 14/15 (93) | 18/19 (95) | > .99 |
| Note taking | 19/20 (95) | 16/22 (73) | .10 |
| Question asking and/or interactive comments | 18/20 (90) | 19/22 (86) | > .99 |
| Number of interruptions, mean (SD) | 0.85 (1.0) | 0.5 (0.7) | .45 |
| Number of unprofessional comments or actions, mean (SD) | 0.2 (0.4) | 0.2 (0.5) | > .99 |
Abbreviations: ICU indicates intensive care unit; ICU provider, ICU physician (intensivist, fellow, or resident) or advanced practice provider; SD, standard deviation.
Denominators are listed in each cell. Numbers less than the total number of handoffs are due to missing transmitters or receivers or inability to evaluate a skill accurately.
Numbers and percentages reflect presence of skill. Listening skills are scored separately for reports from the surgeon and the operating room anesthesiologist.
Numbers and percentages reflect ratings of satisfactory or superior.
For the 10 videos assessed by both evaluators, there was strong agreement overall with Cohen’s kappa values of 0.82 for report delivery skills, 0.72 for engagement, 0.52 for all listening skills combined, 0.62 for number of interruptions, 0.64 for number of unprofessional comments or actions, and ranging from 0.62 to 1.0 for each of the teamwork skills.
Discussion
This study demonstrates a technique to unobtrusively monitor the quality of OR-to-ICU handoffs at all hours of the day and night using existing in-room telemedicine technology. Contrary to our hypothesis, the current study demonstrates that bedside OR-to-ICU handoffs on nights and weekends in an academic hospital are not inferior to those occurring during workday hours.
Basic handoff characteristics and most quality measures were similar between weekdays and nights and weekends, and two subjective aspects of surgeon participation—report delivery skill and engagement in the handoff process— were rated more favorably on nights and weekends than on weekdays.
Although there are a number of reasons why we had hypothesized that night and weekend handoffs would be of lesser quality, including provider fatigue and decreased schedule predictability, there are also several possible explanations for why handoff performance on nights and weekends remained similar, if not slightly better. First, it may be that the lighter operative caseload on nights and weekends resulted in surgeons and other providers being less rushed or distracted and thus more capable of engaging in the handoff process. Second, although we did not differentiate the level of training among surgeons giving handoffs, it may be that surgical providers who played a greater role in the operation or who had more experience with handoffs were more commonly handing over patients on nights and weekends.
Previous reports on the quality of handoffs on different days and at different times of the day are limited and reveal conflicting results.23,24 We recently evaluated OR-to-ICU handoffs during the weekdays versus nights and weekends using study personnel present within the patient’s room, who evaluated the handoffs in full view of the clinical providers. In that setting, we found significantly better transmitter communication scores on nights and weekends compared with weekdays; however, a Hawthorne effect may have been present.23 The current study may have gauged handoff practices more accurately, as the evaluation was carried out essentially without the transmitters or receivers being aware that they were being evaluated. Another study of change-of- shift handoffs in obstetrical inpatients found that resident handoffs were better in the mornings than evenings, while the opposite was true of the nursing handoffs.24
Only a handful of studies have reported on the use of video recording to assess the quality of naturally occurring handoffs.25–28 One study used a conference room camera to assess end-of-the-week handoffs on Fridays between attending physicians and found that these handoffs rarely followed commonly recommended communication schemes.25 Two studies recorded handoffs in the PACU and concluded that the use of a checklist improved OR-to-PACU handoffs and that patient information and equipment were transferred simultaneously in 65% of cases.26,27 Only one other study has assessed OR-to-ICU handoffs; in this study, after implementing a standardized handoff protocol in a pediatric cardiac surgery ICU, the authors used video to demonstrate long-term sustainability.28 It is notable that in all but one of these studies28 the subjects were fully cognizant that their handoff performance was being recorded, potentially affecting their behavior and the quality of the information transmitted.
This study highlights the unobtrusive nature of leveraging existing telemedicine infrastructure to video record handoffs for analysis of quality measures. Video recording of naturally occurring handoffs may serve as an effective means for operationalizing the ACGME’s requirement to provide formal education on and monitor patient handoffs by housestaff. Indeed, other studies have demonstrated that standardized or simulated video recordings are very useful in teaching and monitoring handoff skills as part of a structured curriculum.29,30 These types of recordings, however, lack the individualized assessment, feedback, and coaching that videos of real handoffs could provide. The power of video recording in improving quality of care has already been demonstrated in other areas, for example, with hand washing compliance, colonoscopy, and surgical technique in the OR.31–34 In these studies, improvements were partly attributed to an awareness of video recording,33 group feedback via the posting of compliance rates,34 or individualized coaching sessions with a peer-judged surgical expert.32 These methods are viable options that could be applied in tandem with video recordings to improve handoff quality.
As an observational evaluation, our study has important limitations. First, given the preliminary nature of this work, our sample size is limited and reflects a convenience sample of transitions of care in a single surgical ICU. Although we were unable to randomly select handoffs for recordings, we included all videos capturing admissions and readmissions from the OR for QI purposes that we were able to evaluate before their deletion per institutional policy in the study. Second, the scoring of quality measures studied such as teamwork skills, engagement, report delivery skills, and listening skills is inherently subjective, and there was no available validated scoring instrument for video review. The evaluators used a scoring tool with operational definitions to improve standardization, and although double review could not be conducted for all handoffs due to availability, the few handoffs that did receive two evaluations demonstrated good interrater agreement. Third, because the wall-mounted cameras capture only events occurring within a patient’s room, only handoffs occurring at the bedside were observed; other forms of information transfer, such as those completed by phone or electronic medical record, were not included in our assessment. Finally, the audio quality of some video recordings was limited, and sometimes simultaneously occurring conversations impaired the ability of the evaluator to understand what verbal information was being exchanged.
Despite these limitations, this study is the first to use existing telemedicine video technology to record handoffs and allow quality measures to be assessed unobtrusively.
Conclusions
This study presents a novel approach of assessing handoff quality in OR- to-ICU handoffs using in-room video recording via an existing telemedicine infrastructure. Using this approach, the quality of night and weekend handoffs in a single urban teaching hospital was found to be no worse and sometimes better than those during weekday hours. Video analysis may emerge as an ideal unobtrusive QI methodology to monitor handoffs and improve education and compliance with institutional handoff policies.
Supplementary Material
Acknowledgments
The authors wish to thank Molly Elson and Isabelle Oka for their assistance with video recording and data acquisition, as well as Amanda Martin, MSN, ACNP-BC, for her assistance with handoff evaluation.
Funding/Support: This work was supported by grants from the Anesthesia Patient Safety Foundation and the University of Pennsylvania Provost’s Office to M.B. Lane-Fall.
Footnotes
Please see the end of this article for information about the authors.
Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A430.
Other disclosures: None reported.
Ethical approval: The authors obtained approval from the University of Pennsylvania Institutional Review Board (protocol number 813837).
Previous presentations: This work was presented at the Scientific Forum of the 2015 Clinical Congress of the American College of Surgeons, Chicago, Illinois, October 7, 2015.
Contributor Information
Mark E. Barry, A medical student, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania
Beth R. Hochman, Assistant professor, Division of Acute Care Surgery, Department of Surgery, NewYork-Presbyterian Hospital/Columbia University Medical Center, New York, New York.
Meghan B. Lane-Fall, Assistant professor, Department of Anesthesiology and Critical Care, Perelman School of Medicine at the University of Pennsylvania, and senior fellow, Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania.
Denise Zappile, Acute care nurse practitioner, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.
Daniel N. Holena, Assistant professor, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, and senior fellow, Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, Pennsylvania.
Brian P. Smith, Assistant professor, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.
Lewis J. Kaplan, Associate professor, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.
Ann Huffenberger, Director of operations, Penn E-lert Telemedicine Program, University of Pennsylvania Health System, Philadelphia, Pennsylvania.
Patrick M. Reilly, Chief and professor, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.
Jose L. Pascual, Associate professor, Division of Traumatology, Surgical Critical Care, and Emergency Surgery, Department of Surgery, Perelman School of Medicine at the University of Pennsylvania, Philadelphia, Pennsylvania.
References
- 1.Horwitz LI, Moin T, Krumholz HM, Wang L, Bradley EH. Consequences of inadequate sign-out for patient care. Arch Intern Med. 2008;168:1755–1760. doi: 10.1001/archinte.168.16.1755. [DOI] [PubMed] [Google Scholar]
- 2.Kitch BT, Cooper JB, Zapol WM, et al. Handoffs causing patient harm: A survey of medical and surgical house staff. Jt Comm J Qual Patient Saf. 2008;34:563–570. doi: 10.1016/s1553-7250(08)34071-9. [DOI] [PubMed] [Google Scholar]
- 3.Gawande AA, Zinner MJ, Studdert DM, Brennan TA. Analysis of errors reported by surgeons at three teaching hospitals. Surgery. 2003;133:614–621. doi: 10.1067/msy.2003.169. [DOI] [PubMed] [Google Scholar]
- 4.Joint Commission. Sentinel event data: Root causes by event type 2004–2015. https://hcupdate.files.wordpress.com/2016/02/2016-02-se-root-causes-by-event-type-2004-2015.pdf. Accessed December 21, 2016.
- 5.Starmer AJ, Spector ND, Srivastava R, et al. I-PASS Study Group Changes in medical errors after implementation of a handoff program. N Engl J Med. 2014;371:1803–1812. doi: 10.1056/NEJMsa1405556. [DOI] [PubMed] [Google Scholar]
- 6.Nasca TJ, Day SH, Amis ES, Jr, ACGME Duty Hour Task Force The new recommendations on duty hours from the ACGME Task Force. N Engl J Med. 2010;363:e3. doi: 10.1056/NEJMsb1005800. [DOI] [PubMed] [Google Scholar]
- 7.Riesenberg LA, Leitzsch J, Massucci JL, et al. Residents’ and attending physicians’ handoffs: A systematic review of the literature. Acad Med. 2009;84:1775–1787. doi: 10.1097/ACM.0b013e3181bf51a6. [DOI] [PubMed] [Google Scholar]
- 8.Wohlauer MV, Arora VM, Horwitz LI, Bass EJ, Mahar SE, Philibert I, Handoff Education and Assessment for Residents (HEAR) Computer Supported Cooperative Workgroup The patient handoff: A comprehensive curricular blueprint for resident education to improve continuity of care. Acad Med. 2012;87:411–418. doi: 10.1097/ACM.0b013e318248e766. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.DeRienzo CM, Frush K, Barfield ME, et al. Duke University Health System Graduate Medical Education Patient Safety and Quality Council Handoffs in the era of duty hours reform: A focused review and strategy to address changes in the Accreditation Council for Graduate Medical Education common program requirements. Acad Med. 2012;87:403–410. doi: 10.1097/ACM.0b013e318248e5c2. [DOI] [PubMed] [Google Scholar]
- 10.Gordon M, Findley R. Educational interventions to improve handover in health care: A systematic review. Med Educ. 2011;45:1081–1089. doi: 10.1111/j.1365-2923.2011.04049.x. [DOI] [PubMed] [Google Scholar]
- 11.Bonifacio AS, Segall N, Barbeito A, Taekman J, Schroeder R, Mark JB. Handovers from the OR to the ICU. Int Anesthesiol Clin. 2013;51:43–61. doi: 10.1097/AIA.0b013e31826f2b0e. [DOI] [PubMed] [Google Scholar]
- 12.Segall N, Bonifacio AS, Schroeder RA, et al. Durham VA Patient Safety Center of Inquiry Can we make postoperative patient handovers safer? A systematic review of the literature. Anesth Analg. 2012;115:102–115. doi: 10.1213/ANE.0b013e318253af4b. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Catchpole KR, de Leval MR, McEwan A, et al. Patient handover from surgery to intensive care: Using Formula 1 pit-stop and aviation models to improve safety and quality. Paediatr Anaesth. 2007;17:470–478. doi: 10.1111/j.1460-9592.2006.02239.x. [DOI] [PubMed] [Google Scholar]
- 14.Henkind SJ, Sinnett JC. Patient care, square-rigger sailing, and safety. JAMA. 2008;300:1691–1693. doi: 10.1001/jama.300.14.1691. [DOI] [PubMed] [Google Scholar]
- 15.Cavallazzi R, Marik PE, Hirani A, Pachinburavan M, Vasu TS, Leiby BE. Association between time of admission to the ICU and mortality: A systematic review and metaanalysis. Chest. 2010;138:68–75. doi: 10.1378/chest.09-3018. [DOI] [PubMed] [Google Scholar]
- 16.Saposnik G, Baibergenova A, Bayer N, Hachinski V. Weekends: A dangerous time for having a stroke? Stroke. 2007;38:1211–1215. doi: 10.1161/01.STR.0000259622.78616.ea. [DOI] [PubMed] [Google Scholar]
- 17.Laupland KB, Shahpori R, Kirkpatrick AW, Stelfox HT. Hospital mortality among adults admitted to and discharged from intensive care on weekends and evenings. J Crit Care. 2008;23:317–324. doi: 10.1016/j.jcrc.2007.09.001. [DOI] [PubMed] [Google Scholar]
- 18.Kostis WJ, Demissie K, Marcella SW, Shao YH, Wilson AC, Moreyra AE, Myocardial Infarction Data Acquisition System (MIDAS 10) Study Group Weekend versus weekday admission and mortality from myocardial infarction. N Engl J Med. 2007;356:1099–1109. doi: 10.1056/NEJMoa063355. [DOI] [PubMed] [Google Scholar]
- 19.Ensminger SA, Morales IJ, Peters SG, et al. The hospital mortality of patients admitted to the ICU on weekends. Chest. 2004;126:1292–1298. doi: 10.1378/chest.126.4.1292. [DOI] [PubMed] [Google Scholar]
- 20.Carr BG, Reilly PM, Schwab CW, Branas CC, Geiger J, Wiebe DJ. Weekend and night outcomes in a statewide trauma system. Arch Surg. 2011;146:810–817. doi: 10.1001/archsurg.2011.60. [DOI] [PubMed] [Google Scholar]
- 21.Bell CM, Redelmeier DA. Mortality among patients admitted to hospitals on weekends as compared with weekdays. N Engl J Med. 2001;345:663–668. doi: 10.1056/NEJMsa003376. [DOI] [PubMed] [Google Scholar]
- 22.Greenstein EA, Arora VM, Staisiunas PG, Banerjee SS, Farnan JM. Characterising physician listening behaviour during hospitalist handoffs using the HEAR checklist. BMJ Qual Saf. 2013;22:203–209. doi: 10.1136/bmjqs-2012-001138. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Hochman BR, Barry ME, Lane-Fall MB, et al. Handoffs in the intensive care unit: Are off hours a vulnerable time? [Published online ahead of print December 8, 2015.] Am J Med Qual. doi: 10.1177/1062860615617238. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Goff SL, Knee A, Morello M, Grow D, Bsat F. Handoff quality for obstetrical inpatients varies depending on time of day and provider type. J Reprod Med. 2014;59:95–102. [PMC free article] [PubMed] [Google Scholar]
- 25.Ilan R, LeBaron CD, Christianson MK, Heyland DK, Day A, Cohen MD. Handover patterns: An observational study of critical care physicians. BMC Health Serv Res. 2012;12:11. doi: 10.1186/1472-6963-12-11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Salzwedel C, Bartz HJ, Kühnelt I, et al. The effect of a checklist on the quality of post- anaesthesia patient handover: A randomized controlled trial. Int J Qual Health Care. 2013;25:176–181. doi: 10.1093/intqhc/mzt009. [DOI] [PubMed] [Google Scholar]
- 27.van Rensen EL, Groen ES, Numan SC, et al. Multitasking during patient handover in the recovery room. Anesth Analg. 2012;115:1183–1187. doi: 10.1213/ANE.0b013e31826996a2. [DOI] [PubMed] [Google Scholar]
- 28.Northway T, Krahn G, Thibault K, et al. Surgical suite to pediatric intensive care unit handover protocol: Implementation process and long-term sustainability. J Nurs Care Qual. 2015;30:113–120. doi: 10.1097/NCQ.0000000000000093. [DOI] [PubMed] [Google Scholar]
- 29.Arora VM, Berhie S, Horwitz LI, Saathoff M, Staisiunas P, Farnan JM. Using standardized videos to validate a measure of handoff quality: The handoff mini- clinical examination exercise. J Hosp Med. 2014;9:441–446. doi: 10.1002/jhm.2185. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30.Telem DA, Buch KE, Ellis S, Coakley B, Divino CM. Integration of a formalized handoff system into the surgical curriculum: Resident perspectives and early results. Arch Surg. 2011;146:89–93. doi: 10.1001/archsurg.2010.294. [DOI] [PubMed] [Google Scholar]
- 31.Makary MA. The power of video recording: Taking quality to the next level. JAMA. 2013;309:1591–1592. doi: 10.1001/jama.2013.595. [DOI] [PubMed] [Google Scholar]
- 32.Hu YY, Peyre SE, Arriaga AF, et al. Postgame analysis: Using video-based coaching for continuous professional development. J Am Coll Surg. 2012;214:115–124. doi: 10.1016/j.jamcollsurg.2011.10.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33.Rex DK, Hewett DG, Raghavendra M, Chalasani N. The impact of videorecording on the quality of colonoscopy performance: A pilot study. Am J Gastroenterol. 2010;105:2312–2317. doi: 10.1038/ajg.2010.245. [DOI] [PubMed] [Google Scholar]
- 34.Armellino D, Trivedi M, Law I, et al. Replicating changes in hand hygiene in a surgical intensive care unit with remote video auditing and feedback. Am J Infect Control. 2013;41:925–927. doi: 10.1016/j.ajic.2012.12.011. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.