Effect of Patient Safety Curriculum for Internal Medicine Residents on a Health Care System

Abstract

Background

Healthcare organizations seeking to promote a safety culture depend on engaged clinicians. Academic medical centers include a community of physicians-in-training; however, medical residents and fellows are historically less engaged in patient safety (PS) than are other clinicians. Increased attention has been focused on integrating PS into graduate medical education. Nonetheless, developing curricula that result in real-world system changes is difficult.

Objective

To develop an interactive PS curriculum for internal medicine (IM) residents that analyzes real-word PS problems.

Methods

A multidisciplinary group developed a five-session, case-based PS curriculum for IM residents in the context of a 3-year, longitudinal quality-improvement, PS, and high-value-care curriculum. The curriculum was facilitated by a PS analyst and incorporated mock root cause analysis (RCA) based on actual resident-reported PS events. Each mock RCA developed an action plan, and outcomes were tracked. Pre- and postcurriculum assessments with participating residents were conducted to evaluate the curriculum.

Results

Twenty-eight IM residents completed the curriculum during four iterations from 2017 to 2020. The curriculum identified multiple potential PS risks, led to tangible changes in clinical processes, and enhanced resident confidence in improving systems of care.

Conclusions

We describe an active-learning PS curriculum for IM residents that addressed actual resident-reported PS problems. Through RCA, action items were identified and meaningful system changes were made. Leveraging the expertise of local PS experts in the design and delivery of PS curricula may improve the translation of learner recommendations into real system changes and cultivate a positive PS culture.

Medical error remains a leading cause of morbidity and mortality in the United States (1, 2) and imposes a significant economic burden on the healthcare system (3). Healthcare organizations have implemented educational initiatives and patient safety (PS) reporting systems (4) with the goal of transforming the culture and paradigm around PS (5). Nonetheless, medical trainees underreport PS concerns and are less engaged in PS than are other clinicians (6, 7).

Teaching hospitals incorporate physicians-in-training into many aspects of patient care. Therefore, training residents and fellows in PS is critical to advancing a safety culture. Quality improvement (QI) and PS have been specifically integrated throughout medical education (8–10), with QI and PS being linked to all six Accreditation Council for Graduate Medical Education (ACGME) core competencies (11), specifically in the ACGME Program Requirements for Pulmonary Disease and Critical Care Medicine (PCCM) (12). Prior studies demonstrated improvement in resident engagement in PS by addressing barriers, using formal didactic sessions, and even offering incentive programs (13–17). Others described root cause analysis (RCA) workshops or curricula (17–22,23), although system changes resulting from these educational initiatives are rarely reported (15, 16, 22, 23).

We developed a novel, interactive, case-based learning PS curriculum for internal medicine (IM) residents at the University of Vermont (UVM) Medical Center to meet ACGME milestones and engage residents in identifying safety problems and designing system changes to ameliorate those problems. This curriculum represents a partnership between the UVM Medical Center Office of PS and faculty from the UVM Robert Larner M.D. College of Medicine. In this report, we describe the implementation of this PS curriculum, system changes resulting from the curriculum, and resident perspectives on the curriculum.

Methods

The PS curriculum was designed to address the gap in meeting ACGME milestones (24), specifically Systems-based Practice 2 milestones: recognizing system error and advocating for improvement. Furthermore, there was an institutional transition toward active learning throughout both undergraduate medical education and graduate medical education (GME) during curriculum development, which underscored the case-based learning construct used throughout the curriculum. The curriculum was developed by an interprofessional group of (IM) physicians and pharmacist faculty members in the UVM Department of Medicine and a PS analyst from the UVM Medical Center Office of PS. PS analysts are hospital employees responsible for using the hospital’s PS reporting system to develop and analyze safety metrics, guide PS activities, and assist with the implementation of the organization’s PS program. The PS analyst involved with this curriculum was experienced in leading institutional PS committees, RCAs, and PS educational initiatives; however, this analyst did not have any formal training in curriculum development or GME. The PS curriculum was delivered to Postgraduate Year 3 (PGY-3) IM residents during the final year of a longitudinal, 3-year QI, PS, and high-value-care (HVC) curriculum, in which residents learned the basics of improvement science and QI, conducted mentored QI projects (25), and performed a financial analysis of resident-reported clinical cases by using the principles of HVC. The faculty members involved in the PS curriculum were also integral in the development and delivery of the longitudinal QI, PS, and HVC curriculum and had prior experience in GME curricular design and implementation science.

This active-learning curriculum (Table 1) was facilitated by the PS analyst and delivered in five 1-hour sessions during a 4-month period to four different resident cohorts from 2017 to 2020. The sessions consisted of interactive discussions about PS principles, including system error and RCA, followed by review and analysis of real near-miss PS reports submitted by their peers. Residents then selected one near-miss case to investigate through mock RCA. Cases were selected on the basis of high-risk, high-impact problems with relevance to resident work. During the mock RCAs, which were moderated by the PS analyst with experience leading real RCAs, each resident played the role of a key stakeholder (e.g., a nurse, pharmacist, radiology technician, or physician). The following are brief summaries of the cases selected for mock RCA, which were based on actual resident-reported PS events in which no harm reached the patient.

Table 1.

Patient safety curriculum overview

Session	Topic	Learning Objectives
1	Introduction to patient safety principles	• Describe the framework of just culture and how this learning culture relates to patient safety.
		• Review the institution’s patient safety reporting system.
2	Common causes of error, including performance and system errors and cognitive biases	• Identify common causes of medical error.
		• Differentiate among performance error, system error, and cognitive biases.
		• Review near-miss, resident-reported patient safety events and select one case to investigate.
3	Preparation for mock RCA	• Recognize the approach to event identification and investigation.
		• Review roles and responsibilities for mock RCA.
4	Mock RCA	• Analyze the patient safety event and identify the root cause of the event systematically by employing RCA principles.
		• Integrate RCA findings and formulate recommendations for system changes.
5	Course wrap-up and debriefing session	• Summarize patient safety principles, causes of error, and RCA process.
		• Evaluate the patient safety curriculum and explain how this curriculum will inform future practice.

Definition of abbreviation: RCA = root cause analysis.

The patient safety curriculum was based on Accreditation Council for Graduate Medical Education Systems-based Practice 2 milestones (17): identifies systemic causes of medical error and navigates them to provide safe patient care; advocates for safe patient care and optimal patient care systems; activates formal system resources to investigate and mitigate real or potential medical error; and reflects on and learns from own critical incidents that may lead to medical error.

Summary of Cases

2017

A patient scheduled for percutaneous gastrostomy tube placement was ordered contrast to be administered per interventional radiology instructions. Ultimately, preprocedural contrast was not delivered as intended, leading to a delay in the procedure.

2018

A patient was admitted with a subdural hematoma and inadvertently given prophylactic anticoagulation on admission.

2019

Patients directly admitted from outside hospitals who were receiving intravenous heparin infusion for management of serious problems such as acute coronary syndrome experienced unintended interruptions in heparin therapy when infusion pumps were taken away by the transport teams.

2020

Because of ongoing PS concerns, residents chose to readdress the 2018 case in which prophylactic anticoagulation was inadvertently administered in a high-risk situation.

Residents underwent pre- and postcurriculum assessments during the 2018–2020 iterations of the curriculum to ascertain the degree of comfort with core PS principles, as well as general perceptions regarding PS. Comparisons were made between pre- and postcurriculum assessments by using the Fisher exact test. Analyses were performed by using Stata 16.1 (Stata Corporation).

This work did not constitute research on the basis of the definition of research activity adopted by the UVM Institutional Review Board and was exempt from ethics review.

Results

Twenty-eight (100%) PGY-3 residents participated in the curriculum over the four iterations described. During each iteration, residents reviewed an average of 12 resident-reported, near-miss PS reports and selected one case to be investigated further through mock RCA. Near-miss PS events were identified by the PS analyst from an average annual pool of 190 resident-reported PS events.

During the mock RCAs, residents generated recommendations for system improvements that were delivered to hospital leadership by the PS analyst. The recommendations resulted in numerous system changes, as summarized in Table 2. The system changes were coordinated by the organization’s PS program outside of the resident PS curriculum and engaged interprofessional groups, including QI partners, electronic medical record analysts, nurses, pharmacists, and physicians.

Table 2.

Summary of near-miss patient safety events reviewed during patient safety curriculum mock root cause analyses

Case (yr)	Patient Safety Event Summary (Session 2)	Mock RCA Conclusions (Session 4)	Mock RCA Recommendations	Resultant System Improvement
Contrast administration before PEG tube placement (2017)	A resident called IR regarding contrast administration before PEG tube placement. IR suggested administering contrast through the patient’s nasogastric tube the night before the planned procedure. The resident placed the order with these instructions, only to realize the day of procedure that the contrast had not been administered.	Residents identified the root cause as the default administration instructions in the EMR that autopopulated when contrast was ordered. These administration instructions called for contrast to be given in the hours before a scan, rather than the night before the procedure as intended.	Collaborate with IR and other stakeholders to learn about current recommendations and best practices for administration of contrast before PEG tube placement.	The PEG tube order panel in the EMR was modified to include specific contrast administration instructions, eliminating the need to use two separate orders and ensuring appropriate contrast administration in the future.
			Revise the PEG tube order panel to include the above recommendations.	The improvement plan was achieved through discussions with IR, EMR analysts, and nursing and was available for use within 3 mo of the mock RCA.
			Minimize the number of orders that need to be placed by combining orders into an order panel.
Pharmacologic prophylaxis against VTE in hospitalized patients (2018)	A patient was admitted with a subdural hematoma and inadvertently given prophylactic enoxaparin on admission.	Residents identified a potential root cause as a hard stop in the EMR admission orders requiring that VTE prophylaxis be addressed. This hard stop was sometimes encountered before a patient’s overall clinical picture was fully understood. Findings were validated through discussions with pharmacy, nursing, ICU attending physicians, and QI and EMR analysts.	Change the admission order set such that physicians do not feel pressured to select pharmacologic VTE prophylaxis in situations of uncertainty.	In the months after the mock RCA, a “defer until bleeding risk assessed” option was added to the admission order set, allowing admitting providers the option to defer decisions about VTE prophylaxis for 12 h.
			Implement an educational initiative for residents and attending physicians who routinely use the admission order sets, including an update about the order-set change, best practices for use of pharmacologic VTE prophylaxis, and when the new order-set options should be used.
Direct hospital admissions with intravenous heparin infusions (2019)	Several patients directly admitted from outside hospitals encountered unintended interruptions in heparin therapy upon arriving to the hospital as a result of transport teams disconnecting the infusion and taking their own pump delivery systems with them. Many of these patients were transferred for management of serious problems such as myocardial infarction. If admitting physicians were unable to place admission orders immediately, re-initiation of the heparin infusion could be delayed.	Residents identified that a nurse-initiated order set had previously been developed to address this scenario but was inconsistently used because of transitions in the workforce and waning awareness.	Implement an educational initiative for physicians, nurses, and other members of the care team on floors that allow for nurse-driven heparin infusion protocols.	Newsletters and e-mail communications were shared with pharmacists, nurses, and physicians, promoting more consistent awareness and order-set use.
				Residents were educated about the order set and encouraged to remind nursing staff to use it when a new admission arrives.
				These interventions were developed and implemented just several months after the mock RCA, and the order set is currently being used.
Pharmacologic prophylaxis against VTE in hospitalized patients (2020)	Because of ongoing patient safety concerns regarding inadvertent administration of pharmacologic VTE prophylaxis (similar to those of the 2018 cohort), residents chose to readdress the case of a patient admitted with intracranial hemorrhage who was unintentionally given prophylactic enoxaparin.	Residents identified the root cause as an incongruency between the admission workflow and the EMR order set. Providers may feel compelled to write admission orders in a timely fashion, but often, the information needed to make an informed decision about pharmacologic VTE prophylaxis is not available at the time a hard stop requires pharmacologic VTE prophylaxis orders to be placed.	Optimize admission order-set language.	The order-set language was changed to “differential diagnosis includes a contraindication for anticoagulation; defer until clinical picture is more clear”; this recommendation was similar to 2018 recommendations.
			Consider a default start time for VTE prophylaxis at several hours after admission to provide a safety margin against inadvertent ordering.	These system changes were in conjunction with other organizational quality initiatives targeted at improving VTE prophylaxis.
			Educate providers about the aforementioned changes, and try to standardize these across all service lines and admission order sets.

Definition of abbreviations: EMR = electronic medical record; ICU = intensive care unit; IR = interventional radiology; PEG = percutaneous gastrostomy; QI = quality improvement; RCA = root cause analysis; VTE = venous thromboembolism.

Pre- and Postcurriculum Assessments on PS Principles

A total of 14 residents completed the precurriculum assessment, and 26 completed the postcurriculum assessment (Table 3). Entering the curriculum, 93% of residents agreed or strongly agreed that participation in the PS curriculum was important. All residents felt confident they would use PS principles during their careers, with no significant change in attitudes being shown upon completion of the curriculum.

Table 3.

Pre- and postcurriculum assessments of patient safety curriculum

Survey Question: I feel confident . . .	Pretest (n = 14) [n (%)]	Posttest (n = 26) [n (%)]	P Value
Identifying system causes of medical error	9 (64)	24 (92)	0.04
Advocating for safe patient care	12 (86)	26 (100)	0.12
Reporting errors and/or near misses through the patient safety reporting system	10 (71)	24 (92)	0.16
Teaching others about recognizing and mitigating system error*	8 (57)	21 (84)	0.12
Improving systems of care	7 (50)	23 (89)	0.02
That participating in the curriculum is important to me	13 (93)	26 (100)	0.35
I will use patient safety principles in my career	14 (100)	25 (96)	1.0

A comparison of the percentage of agreement among survey respondents between the pre- and postcurriculum assessments is shown. Original survey responses of “agree” or “strongly agree” were categorized as “agreed”; responses of “disagree,” “strongly disagree,” and “neutral” were categorized as “did not agree.”

^*n = 25 for posttest.

Before the curriculum, only 64% of residents felt confident in identifying system error, which improved to 92% (P = 0.04) after the curriculum. Similarly, residents reported improved confidence in improving systems of care (precurriculum: 50%, postcurriculum: 89%; P = 0.02). There were trends toward improvement in confidence in teaching others about recognizing and mitigating system error (precurriculum: 57%, postcurriculum: 84%), as well as in advocating for safe patient care and using the PS reporting system, which were areas with high baseline amounts of confidence before the curriculum (86% and 71%, respectively).

Resident Perspectives on the Curriculum

Postcurriculum meetings were held after the first two iterations of the curriculum to obtain feedback from residents for internal QI purposes. All residents who took part in the curriculum were eligible, and 11 (79%) were present for the sessions. Participating residents reported that time spent in the curricular sessions was highly worthwhile and advocated for continuing the curriculum. They appreciated the relevance of the curriculum to their daily practice and emphasized the importance of incorporating a “real-world” case. In addition, they gained appreciation for the process of PS event review and RCA, which led to reported increased use of the PS reporting system. Representative quotations included “It was very gratifying to see changes occur” and “[It was] super easy to be engaged.” They frequently cited the abilities to collaborate and effect system change as key learning points: for example, “People do come together and cooperate to improve care” and “Individuals can help make changes to the system.”

Discussion

We describe an active-learning PS curriculum for IM residents that analyzed resident-reported safety events through mock RCAs and incorporated resident recommendations into action plans. Mock RCA recommendations were then successfully translated into tangible system changes, a curricular outcome that is difficult to demonstrate (16, 17). Assessments of the curriculum indicated that residents were engaged in the curriculum and learned that system-based approaches can be effective for improving care, as well as for enhancing confidence in understanding and improving systems of care.

One of the strengths of our program is the interprofessional group of faculty and PS experts involved in the development and delivery of the curriculum. Although other curricula have been primarily delivered by using online modules (14, 20), by chief residents (13, 18, 20, 26), by resident QI–PS councils (15, 27), by faculty with QI–PS interests (13, 18, 22, 28), or by including residents in institutional QI–PS frameworks (23, 29), a novel feature of our curriculum is the direct integration of a local PS expert from our institution’s office of PS into the classroom. The PS analyst was intimately involved with the design of the curriculum and served as the moderator for the active-learning sessions. As real-world PS is an interprofessional process, we believe it is critical that future curricula leverage existing PS experts and infrastructure in the design and delivery of PS curricula. Not only does inclusion of PS experts improve the quality of education around use of local PS systems, but it also provides a natural mechanism for implementing the system changes recommended through mock RCAs, leveraging the PS analyst’s institutional knowledge and skills to facilitate change. Furthermore, learner interaction with the PS analyst cultivates a positive and collaborative PS culture that will be critical for trainees as they transition to independent practice. Although this PS curriculum was delivered to IM residents, we believe that the curriculum is generalizable to PCCM fellows-in-training who must receive QI–PS training and participate in PS event analysis (12). In summary, for programs seeking to develop a PS curriculum we recommend the following key steps:

• 1.Assemble motivated interprofessional faculty with QI–PS experience, including personnel from the institutional PS office.
• 2.Design the curriculum by using an active-learning construct and deliver it during protected educational time.
• 3.Use actual PS events as the basis for discussion (ideally trainee-submitted and -selected events).
• 4.Leverage expertise from local PS faculty with RCA experience to lead mock RCAs and implement RCA recommendations.

Implementation of some mock RCA recommendations were met with real-world challenges. This was perhaps best illustrated when the 2020 cohort chose to revisit the safety concern previously analyzed in 2018. Through analysis of the interval successes and shortcomings of the recommendations from the 2018 mock RCA, the 2020 cohort gained first-hand appreciation of the practical challenges that come with the iterative approach to enacting system changes. Further study is needed to understand the effect of the PS curriculum on safety culture, resident use of the PS reporting system, and engagement in PS activities.

Limitations of this institutional curriculum include the small sample size, as well as the timing and continuity, which are common barriers in GME curricula (19, 29). Resident attendance was variable because of time off, vacation, and night-shift responsibilities, which also affected curricular evaluations because only those present for the respective session took part in the assessment. As such, this precluded a paired precurriculm and postcurriculum assessment analysis among individual residents. Although each author independently reviewed the assessment transcripts, the selected quotes and reflections are not intended to represent formal qualitative research. The curriculum was targeted to PGY-3 residents because they were believed to have the institutional experience and insight to effectively analyze PS reports, identify complex system problems, and develop solutions. However, because these residents were in the final months of their training, it was not possible to assess any subsequent change in the culture, PS event reporting, or clinical practice after completion of the curriculum. Nonetheless, during the assessments, residents reflected on changes in their attitude toward medical error and the PS reporting system: for example, “[PS] reports are important, and do lead to change,” “I’ve filed more [PS] reports in the last 6 months than I have in the last 2 years,” and “It’s a complex system and it’s not just one person’s fault. It’s a system failure, not an individual failure.”

Conclusions

An active-learning PS curriculum for IM residents delivered by a PS expert and based on mock RCAs of actual resident-reported PS events resulted in high amounts of resident engagement and meaningful system changes. Although this curriculum was targeted toward IM residents, the content and approach are generalizable to all GME programs including PCCM.