Abstract
Background
A formal handoff process, such as the I-PASS handoff program, can improve communication about patients among residents. Faculty observation of resident handoffs has served as the primary method for documenting adherence to I-PASS, and little is known about residents' use when they are not being observed.
Objective
We determined how frequently pediatric residents use I-PASS when not being observed.
Methods
We implemented I-PASS in the 2016–2017 academic year and anonymously surveyed residents (December 2016 and June 2017), asking them how they perceive the effectiveness of I-PASS at enhancing patient safety, their frequency of I-PASS use when not observed, co-residents' frequency of use, and open-ended questions regarding factors affecting use.
Results
Fifty-one (52%) and 50 (51%) of 99 eligible residents completed the December and June surveys, respectively. All respondents thought I-PASS had some effectiveness in enhancing patient safety. In December, only 6 (12%) residents stated they used I-PASS more than 75% of the time and reported providing a synthesis statement during handoffs more than 75% of the time. The results were similar for both surveys. Commonly cited reasons for not using I-PASS included time (n = 30), prior knowledge of patients (n = 20), and patients with limited complexity (n = 9).
Conclusions
While most residents thought I-PASS was effective at enhancing patient safety, many reported that they do not use all 5 elements in most of their handoffs when not being observed. Barriers reported included time, familiarity with patients, and limited patient complexity.
Introduction
The I-PASS handoff program (I-PASS) is an evidence-based handoff program created to improve communication during patient handoffs between health care providers. It uses a mnemonic as an organizing framework for communication: Illness Severity, Patient Summary, Action List, Situational Awareness and Contingency Planning, and Synthesis by Receiver.1 In a large multicenter study, the implementation of I-PASS in 9 pediatric residency programs in North America was associated with improved communication and reduction in medical errors and preventable adverse events (eg, errors that harmed patients).2–5 Since the original I-PASS handoff study, the program has been disseminated to more than 500 institutions across the world.6
For residency programs, maximizing the impact of I-PASS relies on training residents and changing resident behavior in daily handoff practices. To date, resident adherence to the I-PASS structural elements has largely been determined by direct observation of handoffs by either faculty members or research assistants. There is evidence from the initial 2014 multicenter I-PASS handoff study to suggest that residents may not be using I-PASS consistently when they are not being observed.7 Understanding resident handoff practices has important implications for how educators ensure resident handoff communication competency outside of an initial intensive training period.
At the Johns Hopkins Children's Center, residents generally exhibited excellent adherence to I-PASS during faculty-observed verbal handoffs. However, during these observations, residents sometimes revealed that their handoff behaviors were different when faculty were not present. The primary aim of this study was to determine how frequently pediatric residents at our institution used I-PASS during handoffs when not observed by faculty. Our secondary aims were to determine trends in I-PASS use over time, which aspects of the mnemonic were used most frequently, and barriers to use of I-PASS. We hypothesized that residents are less adherent to I-PASS when not being observed by faculty.
Methods
We conducted a single center study that collected handoff observation data as well as quantitative and qualitative survey data at 2 time points during our program's participation in the Society of Hospital Medicine (SHM) I-PASS Mentored Implementation Program (June 2016–June 2017). We anonymously surveyed all pediatric residents at the Johns Hopkins Children's Center in December 2016 (mid-implementation) and June 2017 (end of implementation).
The postgraduate year 1 (PGY-1) class received training as part of the SHM I-PASS mentored implementation program6 in June 2016. Residents (PGY-2 and above) had received in-person training on I-PASS8 during their respective intern orientations (June 2014 and June 2015) with an online refresher module in July 2016. I-PASS faculty champions were trained on I-PASS9 and observing resident handoffs during the summer of 2016. In October 2016, we held a departmental I-PASS grand rounds featuring national I-PASS leaders and a noon conference session on the I-PASS written handoff document.
Faculty observations of resident handoffs on 2 general pediatric inpatient units began in September 2016 and continued for the duration of the academic year. During observations of resident handoffs on the inpatient teams, faculty completed the I-PASS Handoff Assessment Tool10 on paper, and responses were entered into REDCap.11 The primary process measure tracked by our program was percent adherence by the handoff giver to all 5 elements of I-PASS with a goal of 75% adherence. Adherence for Synthesis was noted if the receiver provided a synthesis statement (regardless of whether they needed a reminder from the giver). Run charts of this measure were reviewed at monthly faculty champion meetings, posted in resident team rooms, and communicated in person and via e-mail to resident teams and via e-mail to the entire residency program. Incentives such as snacks in workrooms and breakfast for the team with the highest adherence were provided.
The authors, with input from I-PASS Study Group leaders, developed a new survey instrument (provided as online supplemental material), which included questions related to perceptions about I-PASS, self-assessment of frequency of I-PASS use when not observed by faculty, frequency of I-PASS use by co-residents, and open-ended questions regarding factors affecting use. The new survey was not tested prior to use. Surveys were distributed by e-mail via Qualtrics software (Qualtrics LLC, Provo, UT) to residents in December 2016 and June 2017. Three e-mail reminders for survey completion were sent over 2 weeks. Each reminder included an individualized link to respond to the survey. Responses were anonymized so that participants' answers could not be linked with their e-mail address. After survey completion residents were entered into a lottery for a gift card.
For quantitative outcomes, we calculated descriptive statistics for characteristics of participants as well as the frequencies of each response. For open-ended questions, 2 investigators (H.K.H. and T.L.N.) reviewed all responses and devised a coding scheme informed by Framework Analysis.12 All statements were coded by both investigators, and disagreements were resolved by discussion and a consensus-building approach.
This study and our program's participation in the SHM I-PASS mentored implementation were determined to be exempt by the Johns Hopkins School of Medicine Institutional Review Board.
Results
In handoffs observed by faculty on 2 general inpatient teams, residents consistently demonstrated greater than 75% adherence to all 5 elements of I-PASS (Figure) after October 2016. Fifty-one of 99 (52%) eligible residents completed the December 2016 survey and 50 of 99 (51%) eligible residents completed the June 2017 survey. Residents from all training years participated. All respondents thought that I-PASS had some effectiveness in enhancing patient safety (Table 1).
Figure.
Giver Adherence to All 5 Elements of the I-PASS Mnemonic During Observed Handoffs
Note: Johns Hopkins site-specific data from SHM I-PASS Mentored Implementation Program in 2016–2017. Adherence to all 5 elements of the I-PASS handoff mnemonic documented by faculty champions during observed handoffs among residents on general pediatric inpatient teams. One asterisk denotes September 1, 2016 implementation of “Go Live.” Two asterisks denote written handoff noon conference session and department of pediatrics I-PASS grand rounds.
Table 1.
Survey Respondent Characteristics and Perception of I-PASS
| Characteristics and Perception |
December 2016 (n = 51), No. (%) |
June 2017 (n = 50), No. (%) |
| Training year | ||
| PGY-1 | 19 (37) | 17 (34) |
| PGY-2 | 14 (27) | 16 (32) |
| PGY-3+ | 18 (35) | 17 (34) |
| Training program | ||
| Categorical pediatrics | 39 (76) | 39 (78) |
| Combineda | 12 (24) | 11 (22) |
| How effective do you think I-PASS is at enhancing patient safety? | ||
| Extremely effective | 3 (6) | 2 (4) |
| Very effective | 25 (50) | 30 (60) |
| Moderately effective | 20 (40) | 16 (32) |
| Slightly effective | 2 (4) | 2 (4) |
| Not effective at all | 0 (0) | 0 (0) |
Combined programs include medicine-pediatrics, pediatric anesthesia, pediatric genetics, and pediatric neurology.
Abbreviation: PGY, postgraduate year.
In the December 2016 survey, only 12% (6 of 51) of respondents stated that they used all 5 elements of the I-PASS mnemonic more than 75% of the time when not being observed, with no difference by level of training. Furthermore, 24% (12 of 51) reported using all 5 elements less than 25% of the time when not being observed. Only 12% (6 of 51) reported providing a synthesis statement during handoffs more than 75% of the time. Residents reported similar frequency of I-PASS use by their co-residents. Residents reported similar adherence to I-PASS in the June 2017 survey at the end of the yearlong program-wide implementation. In June 2017, only 8% (4 of 50) of respondents reported using all 5 elements more than 75% of the time (all interns) and 22% (11 of 50) reporting using all 5 elements less than 25% of the time (Table 2).
Table 2.
Resident Self-Reported Use of I-PASS When Not Being Observed by Faculty
| Questions | December 2016 (n = 51), No. (%) | June 2017 (n = 50), No. (%) |
| How often do you use all 5 elements of I-PASS when giving a verbal handoff to another resident (when not being formally observed)?a | ||
| > 75% of the time | 6 (12) | 4 (8) |
| 25%–75% of the time | 33 (65) | 35 (70) |
| < 25% of the time | 12 (24) | 11 (22) |
| How often do you provide a patient synthesis statement when receiving a verbal handoff?a | ||
| > 75% of the time | 6 (12) | 5 (10) |
| 25%–75% of the time | 30 (59) | 30 (60) |
| < 25% of the time | 15 (29) | 15 (30) |
| How often do your colleagues use all 5 elements of IPASS during a verbal handoff?a | ||
| > 75% of the time | 2 (4) | 5 (10) |
| 25%–75% of the time | 37 (73) | 32 (64) |
| < 25% of the time | 12 (24) | 13 (26) |
| How often do your colleagues provide a patient synthesis statement when receiving a verbal handoff?a | ||
| > 75% of the time | 4 (8) | 3 (6) |
| 25%–75% of the time | 28 (55) | 30 (60) |
| < 25% of the time | 19 (37) | 17 (34) |
In the June 2017 survey, this question was asked in reference to the last 3 months.
Table 3 describes characteristics of resident statements regarding their use of I-PASS when not being observed. Reasons residents chose to use I-PASS included improved communication (43%, 22 of 51), complex patients (27%, 14 of 51), and the synthesis statement enhancing understanding (20%, 10 of 51). Commonly cited reasons for not using I-PASS during handoffs included time constraints (59%, 30 of 51), familiarity with patients (39%, 20 of 51), and straightforward patients (18%, 9 of 51).
Table 3.
Characteristics of Resident Statements Regarding Use of I-PASS When Not Being Observed
| Category | Examples |
| Reasons residents choose to use I-PASS | |
| Improved communication | Allows for optimal and efficient communication |
| Complex patients | Helps organize information on complex patients |
| Synthesis statement promotes understanding | Often while synthesizing, I will come up with questions about contingency planning or active issues that wouldn't have occurred to me otherwise |
| Patient safety | It provides safety in care of patients |
| Contingency plans | Understanding contingency plans [is] helpful |
| Prevent forgetting information | So [I don't] forget vital info |
| Expected by receiver | Suspicion that receiving team interested in I-PASS style handoffs |
| Clarify goals | [When] unclear about goals of care |
| Reasons residents choose not to use I-PASS | |
| Time constraints | Time consuming when many patients need to be signed out |
| Familiarity with patient | [Patient] already known to service |
| Straightforward patient | Sometimes, with straightforward patients for whom there is nothing to do, it does not seem necessary |
| Uncomfortable with I-PASS | Lack of familiarity/practice using I-PASS |
| Forget mnemonic/forget to use | Simply not remembering certain elements |
| Complex patient | Very complicated patient, many active issues, need for systems-based approach |
| Interruptions | Interruption by pages |
| Redundancy | It can be very redundant on simple [patients] (especially the read back) |
| Perception that I-PASS is ineffective | I don't believe it actually helps |
| Elements of I-PASS mnemonic residents choose not to use | |
| Illness severity | Always in written handoff but not always verbally stated |
| Patient summary | For well-known patients I was picking up again on night shift, we did not always do summary statement/hospital course but focused on intervening events |
| Situational awareness/contingency plans | [Do not say] contingency plans if they were still the same from prior days |
| Synthesis by receiver | Easy to forget to do or seems redundant in simple patient |
Of the 5 elements of the I-PASS mnemonic, the majority of respondents (58%, 29 of 50) described that they chose not to use Synthesis by receiver, indicating that it is easy to forget, takes too much time, or may seem redundant if the patient is not complex (Table 3).
Discussion
Most pediatric residents at our institution recognize that I-PASS is associated with enhanced patient safety and adhere to the I-PASS mnemonic during faculty observed handoffs; yet, they do not consistently use all elements during unobserved handoffs. Rates of I-PASS adherence when not being observed did not improve at the end of an intensive, residency program–wide implementation, which included regular faculty observations and incentives for using the mnemonic during observed handoffs. Residents provided logistical reasons (eg, time), patient factors (eg, complex patient), and rotation factors (eg, intensive care unit), which affect the use of the program. These findings are in contrast to high (> 75%) I-PASS adherence rates during faculty-observed handoffs among our residents.
Coffey et al analyzed focus group interviews of resident participants from 8 of the 9 participating sites in the original I-PASS study,5 and concluded that, “Universally, residents reported more complete adherence to I-PASS when being observed by a faculty member.”7 Despite significant investment in training and observation through participation in the SHM I-PASS Mentored Implementation Program,6 we have observed similar findings related to resident adherence.
The lower levels of reported I-PASS mnemonic adherence, despite intensive training and recognition of patient safety benefits, can be interpreted in 2 ways. First, this may signal a need to place even more emphasis on observation, feedback, and culture change in order to improve adherence and realize the full patient safety benefits of this standardized handoff process. On the other hand, the data suggest that residents are tailoring their use of I-PASS to specific circumstances, and may still be providing high-quality handoffs without strict adherence to the mnemonic. These 2 interpretations of our data have different implications for the future of handoff training and for how to approach the sustainability of I-PASS. Training should include a focus on helping residents and faculty members to adapt the program to the individual needs of their institution and patient population.
Major limitations include the distribution of the survey at a single institution with a response rate of approximately 50%, reducing generalizability to other programs. As this new survey is not supported by validity evidence, respondents may not have interpreted survey questions as intended. Surveys responses were anonymized, thus we were limited in comparing the 2 time points given similar but not identical participants between the 2 samples. All rates of I-PASS use were assessed via self-report and may be affected by recall bias.
Next research steps may include investigating whether handoff training during medical school and faculty development13 to actively model handoff skills improve adherence to I-PASS during residency. Studies of different observation and feedback strategies during residency or the routine inclusion of handoff competencies in resident assessments may further enhance understanding of I-PASS use by residents. Adaptations of the method itself, such as modifications for stable or non-complex patients, may also enhance resident adherence, yet require further study for effects on patient safety. As residents may respond best to evidence that full adherence to I-PASS elements reduces patient harm, an additional next step is to examine the association between I-PASS adherence and patient safety outcomes at our institution.
Conclusion
Most pediatric residents at our institution recognize that the I-PASS handoff program improves patient safety, yet report that they do not use all elements of the mnemonic in the majority of handoffs when not being observed. Barriers include time required, low patient complexity, and familiarity with patients.
Supplementary Material
What was known and gap
The I-PASS handoff program can improve resident communication during patient handoffs. Little is known about how often residents use it when they are not being observed.
What is new
Anonymous surveys of residents regarding their perceptions about the effectiveness of I-PASS and how often they use it when they are not being observed.
Limitations
The survey was distributed at a single institution, limiting generalizability, and lacked validity evidence. Rates of I-PASS use were self-reported and may be affected by recall bias.
Bottom line
Most residents surveyed thought I-PASS improved patient care, but they reported not using it in the majority of handoffs performed when they were not being observed.
References
- 1.I-PASS Patient Safety Institute Inc. 2019 About us. History. https://ipassinstitute.com/history Accessed April 17.
- 2.Starmer AJ, Spector ND, Srivastava R, Allen AD, Landrigan CP, Sectish TC. I-PASS a mnemonic to standardize verbal handoffs. Pediatrics. 2012;129(2):201–204. doi: 10.1542/peds.2011-2966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Starmer AJ, Sectish TC, Simon DW, Keohane C, McSweeney ME, Chung EY, et al. Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA. 2013;310(21):2262–2270. doi: 10.1001/jama.2013.281961. [DOI] [PubMed] [Google Scholar]
- 4.Starmer AJ, O'Toole JK, Rosenbluth G, Calaman S, Balmer D, West DC, et al. Development, implementation, and dissemination of the I-PASS handoff curriculum: a multisite educational intervention to improve patient handoffs. Acad Med. 2014;89(6):876–884. doi: 10.1097/ACM.0000000000000264. [DOI] [PubMed] [Google Scholar]
- 5.Colligan L, Brick D, Patterson ES. Changes in medical errors with a handoff program. N Engl J Med. 2015;372(5):490–491. doi: 10.1056/NEJMc1414788. [DOI] [PubMed] [Google Scholar]
- 6.Starmer AJ, Spector ND, West DC, Srivastava R, Sectish TC, Landrigan CP, et al. Integrating research, quality improvement, and medical education for better handoffs and safer care: disseminating, adapting, and implementing the I-PASS Program. Jt Comm J Qual Patient Saf. 2017;43(7):319–329. doi: 10.1016/j.jcjq.2017.04.001. [DOI] [PubMed] [Google Scholar]
- 7.Coffey M, Thomson K, Li SA, Bismilla Z, Starmer AJ, O'Toole JK, et al. Resident experiences with implementation of the I-PASS handoff bundle. J Grad Med Educ. 2017;9(3):313–320. doi: 10.4300/JGME-D-16-00616.1. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Spector N, Starner A, Allen A, Bale J, Bismilla Z, Calaman S, et al. I-PASS handoff curriculum: core resident workshop. MedEdPORTAL. 2013;(9) doi: 10.15766/mep_2374-8265.9311. [DOI] [Google Scholar]
- 9.O'Toole JK, Starmer A, Calaman S, Campos ML, Hepps J, Lopreiato JO, et al. I-PASS mentored implementation handoff curriculum: champion training material. MedEdPORTAL. 2019;15:10794. doi: 10.15766/mep_2374-8265.10794. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Starmer A, Landrigan C, Srivastava R, Wilson K, Allen A, Mahant S, et al. I-PASS handoff curriculum: faculty observation tools. MedEdPORTAL. 2013;(9) doi: 10.15766/mep_2374-8265.9570. [DOI] [Google Scholar]
- 11.Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–381. doi: 10.1016/j.jbi.2008.08.010. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Ritchie J, Lewis J. Qualitative Research Practice: A Guide for Social Science Students and Researchers. Thousand Oaks, CA: SAGE Publications Ltd;; 2003. [Google Scholar]
- 13.O'Toole JK, West DC, Starmer AJ, Yu CE, Calaman S, Rosenbluth G, et al. Placing faculty development front and center in a multisite educational initiative: lessons from the I-PASS handoff study. Acad Pediatr. 2014;14(3):221–224. doi: 10.1016/j.acap.2014.02.013. [DOI] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.