Resident Experiences With Implementation of the I-PASS Handoff Bundle

Abstract

Background

The I-PASS Handoff Study found that introduction of a handoff bundle (handoff and teamwork training for residents, a mnemonic, a handoff tool, a faculty development program, and a sustainability campaign) at 9 pediatrics residency programs was associated with improved communication and patient safety.

Objective

This parallel qualitative study aimed to understand resident experiences with I-PASS and to inform future implementation and sustainability strategies.

Methods

Resident experiences with I-PASS were explored in focus groups (N = 50 residents) at 8 hospitals throughout 2012–2013. A content analysis of transcripts was conducted following the principles of grounded theory.

Results

Residents generally accepted I-PASS as an ideal format for handoffs, and valued learning a structured approach. Across all sites, residents reported full adherence to I-PASS when observed, but selective adherence in usual practice. Residents adhered more closely when patients were complex, teams were unfamiliar, and during evening handoff. Residents reported using elements of the I-PASS mnemonic variably, with Illness Severity and Action Items most consistently used, but Synthesis by Receiver least used, except when observed. Most residents were receptive to the electronic handoff tool, but perceptions about usability varied across sites. Experiences with observation and feedback were mixed. Concern about efficiency commonly influenced attitudes about I-PASS.

Conclusions

Residents generally supported I-PASS implementation, but adherence was influenced by patient type, context, and individual and team factors. Our findings could inform future implementation, particularly around the areas of resident engagement in change, sensitivity to resident level, perceived efficiency, and faculty observation.

Introduction

Responding to increased attention to harmful effects of resident physician fatigue¹ and pressure to limit resident work hours,² medical educators^3–5 have called for structured handoff training to limit potential untoward effects of discontinuity in care. The I-PASS Handoff Bundle is an evidence-based and theory-informed comprehensive handoff program.^5–7

The multi-center I-PASS Handoff Study⁸ evaluated the handoff improvement program across 9 residency programs. At each site, implementation was carried out by a local team led by a study site leader who was a faculty attending physician. Residents experienced bundle implementation in the following way: preparations included sharing of background information at an educational event or grand rounds, faculty development activities for attending physicians, then a go-live that constituted a 3-hour training workshop, transition to an I-PASS–formatted electronic handoff tool, and a stated expectation by program leadership to use the I-PASS verbal handoff structure. This was reinforced through an awareness campaign and regular observation and feedback over at least 12 months (Table 1).^9–11 This intervention was associated with improved communication and a 30% reduction in preventable patient harm events without negative impact on workflow.¹² We conducted a parallel qualitative study in order to explore the implementation experiences of the end-users (ie, pediatrics residents). Several studies reported descriptions of elements of the I-PASS program. This is the first to report the experience from the resident perspective.

Table 1.

Elements of I-PASS Handoff Bundle

Methods

Study Design

As our objectives were to explore the implementation experiences of end users, this study design followed principles of grounded theory.¹³

Recruitment and Data Collection

We created an interview guide sensitized by a framework for evaluating diffusion of innovation in health care¹⁴ (provided as online supplemental material). Site-level investigators solicited volunteers from the full residency program using an e-mail invitation, resulting in a convenience sample. One to 2 investigators previously unknown to the residents (physicians [M.C. and Z.B.] and social scientists [K.T. and A.A.]) conducted focus groups lasting 40 to 90 minutes with 2 to 11 participants. After obtaining informed consent, we audio-recorded participants' perceptions of handoff quality, experience with I-PASS implementation, and reflections on sustainability of the bundle (Table 1). We conducted focus groups at 8 of the 9 residency programs involved in the study in the United States and Canada, including a total of 50 participants (Table 2). All sites implemented I-PASS on general pediatrics inpatient units. Typically, handoffs occurred in a team fashion, with 2 to 4 junior residents (postgraduate year 1 [PGY-1]) and 1 to 2 senior residents (PGY-2 to PGY-4). Handoffs also included medical students at most sites; attendance of fellows and attendings at handoff was less common. One site was not included due to logistical barriers. Focus groups occurred July 2012 through December 2013, 12 to 24 months after intervention began.

Table 2.

Study Participant Demographics

Institutional Review Board approval was obtained from The Hospital for Sick Children's Research Ethics Board.

Data Analysis

We transcribed audio-recordings of focus groups verbatim and imported them into NVivo 10.0 (QSR International Pty Ltd, Doncaster, Victoria, Australia).

We used inductive content analysis to analyze data, beginning with open coding on a subset of transcripts, highlighting important quotes and noting salient concepts. Open coding is defined as “the analytic process through which concepts are identified and their properties and dimensions are discovered in data.”¹³ Open coding began transcript by transcript (ie, site by site), but initially focused on the individual resident participant. Because participants emphasized how they adapted their actions to how others on the team performed, we ultimately considered both individuals and teams. The primary coders (M.C., K.T., S.-A.L.) held regular discussions to develop preliminary codes and intermittently reviewed transcripts with other investigators to validate interpretations and further refine codes. As new concepts emerged from subsequent transcripts, we continually revised the codes. To facilitate axial coding, we assigned each text segment attributes of site, resident level (PGYs 1–4), and I-PASS bundle component, and then sorted across sites by attribute, comparing, for example, all junior resident comments across sites and comments within a particular site. Finally, all investigators reviewed a representative sample of coded data to establish consensus on the validity of the analysis.

Results

Strong similarities between sites emerged and persisted throughout data collection. We reached saturation after analysis of approximately 5 transcripts, but continued collection and analysis to further confirm findings and to ensure representation from a broad range of sites. The data presented here are organized according to how the focus group script flowed. We started by asking participants about their perceptions of handoffs prior to implementation, inquired about practices over the course of implementation, and concluded by asking about sustainability. We present the most common or dominant sentiment for each element, but note disconfirming perspectives that exist.

Perceptions of Handoff Quality Prior to I-PASS

Most residents agreed that prior to program implementation, handoffs were not ideal. They recalled situations of poor handoffs, and readily acknowledged the value of learning a structured approach, particularly for junior trainees. In most cases, residents described their prior approach to learning the handoff as “by osmosis.”

“In my first couple of years of residency . . . we would participate in standard handoffs, which were of variable qualities, and variable techniques were used so it was unpredictable often. And it definitely was frustrating to not have a standardized method to hand off to people . . .” (Site R, R01, PGY-4).

Experience With Resident Handoff Training Workshop

Recollections of handoff training tended to be neutral to positive. Participants felt that the training provided a solid framework for performing an ideal handoff. Junior residents almost universally welcomed a structured approach. However, many senior residents questioned the need for training, given their level of experience.

“I think at first most of the senior residents felt that . . . this is like teaching us how to breathe, you don't need to teach us how to do this. . . . But it does seem like the junior residents really enjoyed it because it is a good standardized way to go through things” (Site R, R01, PGY-4).

With respect to simulation training, residents acknowledged the difficulty in simulating a realistic handoff, but valued the opportunity to practice the skill. Some residents commented that the training was too long.

At most sites, implementation of I-PASS handoff occurred immediately after training. At a few sites, there was significant delay (months) before using I-PASS during actual handoffs, which resulted in confusion and, in some cases, negative perceptions of the program.

Electronic Handoff Tool

We observed differences by site regarding experience with the electronic handoff tool. Prior to I-PASS, all residents had used some form of a written tool. At sites where the option was available, residents appreciated auto-importing of data into the I-PASS tool. Residents experienced technical difficulties with the tool at individual sites. For instance, printing of the tool often resulted in a long document. Such technical issues engendered negative feelings about the I-PASS program that persisted despite residents' acknowledgement of improvements made to overall document length and usability.

Residents reported variations in handoff document structure and content across sites. All followed the I-PASS format, but some sites separated the 5 subfields of the patient summary, and residents at these sites expressed greater frustration with perceived time spent completing the document.

Perception that the electronic handoff tool should match the verbal handoff structure was consistent across sites, and was cited as the strongest driver for using verbal I-PASS structure in practice. Another theme was user dependence of the I-PASS written tool, which relies on daily manual updates to ensure care plans are current. Themes included hesitation to edit colleagues' entries, ambiguity around accountability for document maintenance, and personal preference regarding level of detail, which contributed to variability in how residents approached tasks, along with some degree of frustration. When residents encountered high workload and competing tasks, updating the handoff list was a lower priority.

“And it is the last thing on your list, like you have parents that want to get discharged, you have kids to admit, you have people to call; you are not going to update your I-PASS when you have all of that other stuff to do, and updating your I-PASS isn't going to affect anything at that moment” (Site Y, R02, PGY-1).

Handoff Environment

Many residents raised handoff context and process issues. For example, the curriculum emphasized the importance of having dedicated space, reliable start time, and minimal interruptions. However, some sites had difficulty achieving an optimal handoff setting, due to local processes or organizational culture issues. Residents experienced frustration with interruptions during handoffs at some sites, while it was a relative nonissue at others.

Reinforcement Phase: Incorporation Into Daily Practice and Observation/Feedback

Residents at all sites and PGY levels expressed concerns about efficiency. Although residents lacked data regarding handoff length before and after implementation, most assumed that structured handoff was more time-consuming and, at times, was considered “redundant,” “cumbersome,” or required “unnecessary detail.” In contrast, some residents reported that I-PASS may actually improve efficiency.

“Then for 3 days . . . I would time every team kind of secretly, and actually it was way more efficient even with read-back. Even though it felt slower, it actually went much faster, people stayed focused and thorough. I was surprised and I was actually shocked” (Site Q, R04, PGY-3).

When asked which elements of structured verbal handoffs were used regularly by residents, the pattern of adherence was remarkably similar across sites and levels. While residents largely accepted I-PASS as an ideal way to perform handoffs, they felt it should not be applied to every patient. They regarded the “full I-PASS” as most beneficial for complex or sicker patients, junior learners, and unfamiliar teams, but unnecessary for straightforward patients or morning handoffs back to the day team.

“I used some of the elements of I-PASS, but not as sort of like consistently. For me it was when I have more complicated patients that is when I definitely sort of like I felt like a structure really helped. . . . Whereas patients that are much simpler . . . following the whole I-PASS program was a little too cumbersome sometimes” (Site T, R03, PGY-2).

Universally, residents reported more complete adherence to I-PASS when being observed by a faculty member.

“It did take longer, sign-outs were definitely longer when [faculty] were watching, and I think a lot of it was because we felt like we needed to say something about every box, we needed to have an action in every box when we did this situation monitoring every box . . .” (Site Y, R01, PGY-1).

What was known and gap

The effectiveness of the I-PASS handoff bundle has been assessed in a multi-site study, yet little is known about the experience of residents with this new approach.

What is new

A qualitative study explored residents' experience with and perception of the I-PASS bundle.

Limitations

Single specialty study may reduce generalizability; potential for social desirability responding.

Bottom line

Resident adherence to I-PASS was influenced by patient type, context, and individual and team factors. This could inform strategies that increase engagement in the use and improvement of I-PASS.

Residents often described that a team leader (ie, senior resident) would set expectations regarding the degree of I-PASS adherence. They described a typical pattern of use of the mnemonic's individual elements, with a consistent rationale for this pattern.

Illness Severity (first stating whether the patient is “stable,” “a watcher,” or “unstable”) was seen as valuable for flagging sick patients, but less likely to be repeated for stable patients.

“I . . . think the only important thing that I've taken away or implemented is the severity of the patient . . . that I think gives me a good immediate snapshot of what I should be paying more attention to” (Site T, R02, PGY-3).

Patient Summary and Action Items were considered obvious, intuitive handoff components. However, many residents perceived that completeness was overemphasized compared to conciseness, and that they were expected to include more details than necessary. In terms of Action Items, residents agreed strongly with the program's emphasis on clarity and specificity.

Residents expressed varied opinions on Situation Awareness/Contingency Planning and Synthesis by Receiver. Participants regarded contingency planning as something they should be doing already; following I-PASS served as reinforcement. There were some divergent opinions. For example, at 1 site, the contingency planning field became known derogatorily as the “be a doctor” box (Site Y, R06, PGY-2).

Synthesis by Receiver (restatement of key information to ensure accuracy) was universally the most challenging element reported by all sites. In some cases, whether synthesis was performed became the defining feature that, to residents, indicated whether I-PASS was being used. Some resistance stemmed from residents' perceived expectation to repeat the full handoff rather than confirming key points, as training had intended.

During faculty observations, residents valued the constructive feedback on handoff skills. However, there was significant variation in the experience across individuals at and between sites. Global feedback tended to be appreciated, while detailed feedback on mnemonic compliance was deemed “nitpicky.” Some residents resented faculty observation of handoffs and felt competing demands on their time were unappreciated.

“. . . when we get observed we get feedback from it and sometimes it's annoying because we'll get like little details like I preface that everybody was stable, but I didn't say that every single patient was individually stable . . . when we have 10 people on the census and you're saying everyone's absolutely stable I don't feel like I need to say ‘stable' 10 times” (Site W, R07, PGY-1).

There were mixed opinions about the I-PASS campaign. Some residents thought it was a good reminder, while others did not feel it changed their behavior.

“We have these computers in the work room . . . on the computers they put this little tab that says I-PASS, so you see it frequently, I just think that that is a good reminder because it makes you think that the hospital takes it seriously” (Site Q, R05, PGY-1).

“I don't think it's the button that we wear or the little cards that are stuck to the computer, I know that's there but I don't know what it says. It's the process of doing it every day that is reminding me to do it, not the printed things” (Site W, R08, PGY-3).

Sustainability

For some residents, sustainability of the I-PASS program rested on whether efficiency concerns could be addressed; others felt program success ultimately depended on whether it improves patient safety. Residents indicated that they intended to continue using those aspects of the I-PASS bundle they found most useful and had been widely adopted already.

“And I think what I intend to do as a senior is keep the stable and watcher thing and then have read-back only with like complicated patients” (Site Q, R03, PGY-1).

Residents suggested that sustainability was likely to be enhanced if refinements were made that more closely aligned the program with those components of the program that reflected residents' feedback: “I keep kind of fantasizing about I-PASS 2.0 . . .” (Site D, R07, PGY unknown; Table 3).

Table 3.

Resident Suggestions for Future Refinements to the I-PASS Program

Discussion

While residents accepted explicit training in handoff skills, they did not consistently enact structured handoff communication for all patients. They experienced a conflict between the program's focus on adherence to a standardized approach, their perceptions of unnecessary rigidity, and their expectations to be able to use judgment, particularly at the senior level, with regard to how to use elements of the format for different patients and situations. The validity of our results is supported by the inclusion of a large sample size from all years and multiple sites; facilitation by investigators unknown to residents, resulting in frank discussion; and the high degree of saturation we achieved.

We were initially guided by the diffusion of innovations framework,¹⁴ but as we progressed through analysis we determined that normalization process theory (NPT)¹⁵ offered a better framework for our findings and clearer recommendations for program improvements and future I-PASS implementation. NPT proposes that implementing a complex intervention occurs through 4 main processes: coherence, cognitive participation, collective action, and reflexive monitoring (Table 4). Our study illuminated several issues related to these 4 processes.¹⁵

Table 4.

Generative Mechanisms in Normalization Process Theory

• 1. Coherence: The curriculum helped to define a shared purpose, and most residents were aligned with the goal of improving handoff and patient safety; however, senior residents perceived that they were already doing effective handovers.
• 2. Cognitive Participation: A subset of residents were involved in developing I-PASS, but not all residents were engaged in overseeing how the handoff bundle would be incorporated into daily practice.
• 3. Collective Action: Residents experienced conflict between the ideal handoff taught and the necessity to use all elements consistently, as well as uncertainties around ownership, quality, and daily updating of the document.
• 4. Reflexive Monitoring: Residents felt that knowing the impact on patient safety outcomes would augment buy-in to the program and enhance sustainability, but lacking these data or handoff duration data hindered addressing concerns around efficacy and efficiency in this study, which occurred in parallel to the study of I-PASS impact on patient safety.

Using the 4 processes of NPT as a guide, medical educators may promote coherence in practice by prompting residents to discuss the local meaning of I-PASS and how it adds to achieving high-quality care. To optimally embed I-PASS into the culture, medical educators can consider initiatives to promote more resident participation in the implementation and monitoring and, ideally, to increase their level of engagement and commitment at the outset. To promote a sense of resident ownership, it would be important to establish roles for residents in the use and modifications of I-PASS in their daily practice. The fact that in the quantitative I-PASS study there was no difference observed in the length of handoffs before versus after implementation of the program, yet residents experienced frustration with the duration of handoffs, emphasizes the importance of reflexive monitoring. Residents may misattribute their general frustrations with time pressures and handoff duration to a new handoff program. Although I-PASS was associated with significant safety improvements, with no increased time cost to residents,¹² given the intense level of concern about efficiency, future implementation efforts may need to make this more explicit in the training and provide real-time data on time spent on handoffs.

Together, this and the quantitative I-PASS study¹² both provide robust complementary information on the experiences and outcomes of implementing a handoff bundle. Process data from the quantitative study revealed that written and verbal handoff quality improved significantly in all areas, but residents did not consistently adhere to the I-PASS format, with a pattern of use matching what residents describe in this study. One explanation for the observed improvement in patient safety despite inconsistent adherence to the mnemonic during verbal handoff is the presence of other important active ingredients in this complex intervention, such as auto-importing of patient data into the electronic handoff document, improvements to the handoff environment, and improved teamwork and communication skills. Strict adherence may not be necessary to achieve desired outcomes. What is not known is whether strict adherence would result in greater improvements in patient safety.

One limitation of this study is that the use of a true grounded theory approach was precluded by logistical constraints, which did not allow for theoretical sampling. As a result we present themes that emerged from our content analysis of the data rather than a “grounded theory.” In addition, focus groups are subject to social dynamics within the group, and minority perspectives may be relatively silenced. This is particularly important in the context of residency, in which hierarchical relationships exist between junior and senior trainees. Another limitation is that each resident group was only studied at 1 point in time and at different points relative to the start of the implementation. This may have contributed to resident ambivalence about some aspects of the program.

These findings must be taken in the context of when they occurred; the landscape with respect to handoff education in the United States and Canada is rapidly evolving. Time is particularly important in the context of resident education; new trainees may be more receptive to educational initiatives or practice changes than senior trainees, and several years may be required to fully realize intended benefits of implementation.

Conclusion

Our study provides useful insights for education and patient safety leaders planning to implement a handoff improvement program. Residents' perspectives on the experience and their suggestions for modifications could be incorporated into future refinements of this program or the development of new programs.