Teaching Quality Improvement in Emergency Medicine Training Programs: A Review of Best Practices

Abstract

International graduate medical accreditation bodies are placing increasing emphasis on resident education and competency in the principles of quality improvement and patient safety (QIPS). Current QIPS educational curricula are heterogeneous and variably attain stated objectives. We have conducted a review of QIPS curricular best practices and barriers to implementation of successful QIPS curricula and provide clear solutions aimed at overcoming these barriers. Emergency medicine programs provide fertile ground for QIPS initiatives and can become world leaders in QIPS curricular development and education.

Background

In 1999, the Institute of Medicine published “To Err is Human,”4 a report that was the first broadly disseminated measurement of medical harm in the United States.1 Over 15 years later, graduate medical education bodies internationally have made explicit the requirement that residents should complete their training with an aptitude for performing quality improvement and patient safety (QIPS) work. The ACGME have included a resident‐centered “practice‐based learning and improvement” competency, as well as mandatory resident integration into interdisciplinary QIPS programs as part of the Common Program Requirements.1 These are further discussed in the program requirements for emergency medicine5 as well as the ACGME/ABEM Milestones project.6 As such, the case described in Box 1 is one that will likely resonate with many teachers in emergency medicine.

Box 1. A Vignette.

Dr. Vaughan, the emergency medicine program director at a large academic institution, had recently returned from the International Conference on Residency Education (ICRE). She decided to attend the preconference summit on quality improvement (QI) as this seemed to be gaining much traction in her local health system and in academic circles. They discussed the Accreditation Council for General Medical Education (ACGME) requirements for QI competencies,1 the new CanMEDS 2015 framework,2 and the General Medical Council's (UK) QI framework.3 Most program directors understood that they needed to develop learner competency in QI to meet accreditation standards but many questions remained unanswered. What is the best way to teach QI? How do I develop faculty along with residents to acquire the skill set? How do I ensure that the departmental and hospital leadership will support these initiatives? How do I find room in the curriculum to deliver the content? How do I evaluate QI competency?

As she pondered, she overheard her clinical chief, Dr. Patel‐Zhao, discussing a recent patient case. The patient has severe urosepsis leading to prolonged intensive care unit, which was the result of a missed resistant sensitivity on urine culture. Dr. Vaughan recalled that it was their physician group's responsibility to follow‐up on their own tests, including culture results. However, with her new QI hat on, she wondered if there might be a better way to modify the system to treat the problem of missed sensitivities and involve the residents in this process along the way.

In Canada, the Royal College of Physicians and Surgeons released CanMEDS 2015, which explicitly requires QIPS training. Within CanMEDS 2015, a few concepts (“Apply the science of quality improvement to improving systems of patient care” and “Contribute to a culture that promotes patient safety”) explicitly emphasize QIPS skills.2 In the United Kingdom, the General Medical Council published a Quality Improvement Framework that is designed to govern undergraduate and graduate medical education training in QIPS.3 Their framework document outlines the responsibilities of medical schools, training programs, and individual practitioners in ensuring QI competency for all learners. A comparison of the ACGME and CanMEDS 2015 requirements is outlined in Figure 1.

Figure 1.

CanMEDS 2015 and Accreditation Council for General Medical Education quality improvement and patient safety educational requirements in emergency medicine graduate medical education.

The emergency medicine context

Despite a clear call for improvement from the literature and a strong mandate to educate medical trainees about QI, the full potential of strong QIPS trainees and faculty improving clinical outcomes has yet to be achieved.7 Studies have consistently demonstrated relatively low adherence to best practices and quality of care metrics across the spectrum of health care, which includes some elements of emergency department (ED) care.8, 9 Other authors have measured temporal trends in harm incidents within 10 institutions over a 5‐year period and have noted no significant changes.10 Some studies have documented a 7‐day ED return rate of 6.8% and, within this group, a 5.7% incidence of adverse event rate of which 56.6% were deemed preventable.11 Others have documented almost 11,000 deaths annually within 1 week of discharge from the ED using claims data form the U.S. Medicare program.12 This study focused exclusively on healthy people living in the community and excluded individuals with morbid illness and poor functional status.

Additionally, the practice environment in the ED has significantly changed. Tang and colleagues13 reported a 37.7% increase in U.S. national ED volume between 1997 and 2007 from 94.9 million to 116.8 million using the National Health Ambulatory Medical Care Survey (NHAMCS). In 2013, the NHAMCS reported 130 million ED visits nationally in the United States, an additional increase of 11.6%.14 Increasing volumes and growing wait times have led many researchers to develop indices of ED crowding. Although there is no consensus on a single measure of crowding, it has been associated with a reduction in the quality of ED care15 further compounding the need for safety and quality efforts. This study identified number of patients in the waiting room, ED occupancy, and the number of admitted patients awaiting inpatient beds as crowding measures reflecting quality of care.

Although there is a clear need for QIPS in the ED, we still lack clear methods for building capacity for improvement. Publications demonstrating teaching methods that result in long‐term understanding of QIPS principles by trainees or cultural/attitudinal changes toward improvement are not available.16 Capacity‐building programs designed to add competency at the faculty level are also missing from the literature on the subject. These elements are essential in supporting QIPS projects at all levels and furthering a culture of improvement in the academic and clinical environments.

This article provides a synthesis of best practices in QIPS education and some insights in design of QIPS curricula for optimal outcomes. We have also identified the most common barriers to curricular implementation and we offer solutions aimed at creating an environment designed for and suited to high‐impact QIPS education and projects.

The state of qips curricula

There has been a wide range of teaching methods used in QIPS curricula including didactic lectures, small‐group discussion, case discussion, experiential learning, project/presentations, simulation/role‐play, and multimedia/Web‐based modules.17, 18 A systematic review in 2010 showed that often QIPS curricula usually focused on improving trainee's knowledge.17 Topics covered have included:17, 18

• Patient‐safety overview (e.g. covering terminology, rationale);

• Continuous QI (e.g., plan‐do‐study‐act cycles);

• Communication and teamwork;

• Audit and feedback;

• Process mapping;

• Change management and culture change;

• Systems thinking;

• Root cause analysis;

• Systems‐based analysis;

• Human factors engineering;

• Error/incident reporting;

• Dealing with errors;

• “Just culture” (i.e., avoiding blame/shame);

• Disclosure of error and medication safety.

Although many of these topics are included under the umbrella of QIPS, the required breadth of QIPS education is not made clear in accreditation requirements. Additionally, some of these elements may be better suited to other parts of a residency curriculum (e.g., “just culture” may fit better within an ethics curriculum), while other topics are merely a small part of broader QI tools such as Lean or the Model for Improvement. Due to the highly variable nature of QIPS content, there is a need to identify high‐quality adoptable methods of implementing QIPS curricula.

High‐quality qips curricular elements

Experiential learning is continuously identified as a best practice feature of high‐quality QIPS curricula.19, 20 Experiential learning is a teaching model in which residents are immersed in real‐life experiences that double as a learning environment. In the case of QIPS, this could mean that they are part of larger multidisciplinary teams who are tasked with solving specific clinical quality and safety problems under the guidance of a quality expert. Aligned with best practices and Kolb's learning cycle theory,21 most studies in the a recent systematic review18 showed that experiential learning modalities were more frequently used than in the previous review by Wong and colleagues in 2010.17 Unsurprisingly, trainees found experiential learning more effective when compared to other didactic components.22 Although an experiential learning component for high‐quality QIPS training has been suggested as being important within the literature,20, 23 implementation and evaluation of such curricula have been met with varying levels of success.24

Another recent systematic review applied Pawson and Tilley's realist evaluation framework25 to analyze what works for whom, under what circumstances to achieve which outcomes. Applying the realist evaluation framework allows us to understand which elements of a curriculum work for whom and what outcomes one might expect when implementing a similar program.

Jones and colleagues24 advise that since residents are front‐line workers, they may have great insights into processes to improve local systems. The authors note that successful QIPS teaching requires both educational and clinical leadership support the trainees.

Meanwhile, Jones and colleagues also note that there is a wide variety of approaches for recruiting faculty to teach these sessions, explicitly noting that it may be worthwhile to engage interprofessional teachers. Multiple successful modalities are promoted including a broad, diverse group of teachers or a small dedicated QIPS educational team. An example of a local curricula that has shown some of these key elements has recently been disseminated through Academic Life in Emergency Medicine (ALiEM) Ideas in Didactics and Educational Activities (IDEA) series.26

Curricular outcomes

Experts in medical education have suggested that we should be cautious when inferring medical education quality from patient outcomes.27 While this may be the case for certain aspects of medical education, in QIPS programs, using patient outcomes may indeed be the correct approach for evaluating QIPS curricula. With most educational interventions, it may be exceedingly difficult to show the relationship between a trainee's learning and a systems‐level outcome due to problems such as downstream effects or difficulty with proving causality. In QIPS curricula, however, generating systems or patient‐level improvements are the goal of these programs, and therefore, it is important to determine if these curricula achieve these ends. Simply knowing the terminology or concepts behind QIPS may be an easily measurable outcome, but determining whether students and members of the health care team can apply these concepts and perhaps create systems‐level changes that affect patient care are of great importance.

When examined using the Kirkpatrick evaluation framework, the bulk of the QIPS curricula program evaluations are still around learner satisfaction (Kirkpatrick Level 1 outcome), and improvement in trainee knowledge (Kirkpatrick Level 2) or changes in learner behaviors (Kirkpatrick Level 3).17, 18

Admittedly, it may be quite difficult to measure institution‐level changes. For convenience reasons, many studies of QIPS curricular outcomes have tended to test knowledge acquisition immediately after the completion of a curriculum.17, 18 Studies of more distal application and retention of QIPS concepts may be of use in strengthening the evidence behind a successful intervention.16

Those who engage in evaluating knowledge acquisition might also measure the application of said knowledge. The Quality Improvement Knowledge Application Tool (QIKAT) is a previously derived tool28, 29 that has been recently revised.30 The tool presents learners with three system‐level quality issues in narrative form and ask them to respond with an aim and measures and propose an initiative for improvement. These are graded by trained adjudicators. Using a standardized assessment tool for trainee knowledge acquisition can help to strengthen the evaluation components for QIPS projects.17

A few promising studies have led to persistent organizational changes (Kirkpatrick Level 4).22, 31, 32, 33, 34, 35 Examples of such organizational changes are the frequency of red‐flag symptoms identified in back pain presentations in a general practice, all patient receiving a physical examination within 6 hours of admission to a mental health unit, and implementation and use of central line kits and team and completion of discharge summaries before discharge from the institution. Educational initiatives which achieve organization improvements represent the application of learning in the clinical environment, a milestone that eclipses the simple synthesis of conceptual frameworks. All future QIPS curricula should seek to deliver didactic learning while providing learners with the ability to apply the learning, under the guidance of experts, in the clinical environment.

Barriers to implementation of qips curricula

The literature on QIPS education offers important insights into factors influencing the successful implementation of QIPS curricula.17 Subsequent publications have explicitly isolated specific challenges to be addressed to expand expertise and training in QIPS.16, 19 These factors affecting successful implementation are divided into four categories: learner factors, teacher factors, curricular factors, and learning environment factors.16, 17 These are listed in Table 1.

Table 1.

Barriers to Integrating QIPS and Potential Solutions

Domain	Barriers to Integrating QIPS	Potential Solutions
Learner factors	Level of learner enthusiasm or buy‐in toward curriculum	Exposure to QIPS concepts during UGME training
	Competing educational demands of medical students and residents	Use QIPS project to fulfill accrediting body's scholarly project requirement
		Divide clinical epidemiology project and QI project streams—learners to choose personal focus
		Develop need‐to‐know QI and clinical epidemiology curricula for all residents
		All trainees should be part of a QI project during their training
Teacher factors	Adequate number of faculty with expertise in teachinga	Enroll all core faculty members in basic QI training programs Faculty participation in experiential QI project curriculum with residents (colearning model)34 Enable interested faculty to pursue additional specialized training in QIPS35 Cultivate or recruit QIPS experts with advanced training36
	Involvement of faculty role models committed to patient safety
	Level of faculty enthusiasm or buy‐in toward curriculum
	Time burden on faculty to teach the curriculum
	Faculty recognition and support	Establish QIPS directorships in the department and lobby for academic recognition and promotion track
Curricular factors	Curriculum should combine experiential teaching methods	Preferentially select educational methods with an emphasis on experiential or mixed methods vs. didactic only
	Provide adequate time to carry out the curriculum (especially those involving QI projects)	Use novel scheduling modalities34 Integrate this work with current M&M rounds, administration blocks and interprofessional collaboration requirements
	Scheduling to optimize completion of the QI project
	Development of better assessment tools for competency in QIPSa	Consider use of the QIKAT‐R tool for resident knowledge evaluation28 (acknowledging limitations)
Academic environment factors	Institutional culture that does not support QI educational efforts	Inform senior institutional executives of the curriculum and intended effects in the clinical setting—obtain an explicit executive sponsor Align all QI projects with department and institutional agendas Seek early administrative support and establish clear and rapid channels to allow data acquisition QI curricula projects should balance scope with the need to develop interdepartmental working relationships and improvements to foster improved overall culture Discuss with IRB or REB regarding parallel processes for proposed QI projects while maintaining safety standards37
	No link of QIPS initiatives with hospital priorities or operational activities
	Lack of funding to promote and support changes that result from QI activities
	Lack of access to health system data to support QI work
	Limited or poor integration of QIPS work into the clinical learning environmenta	Use of experiential and longitudinal projects in concert with colearning initiatives

IRB = institutional review board; QI = quality improvement; QIPS = quality improvement and patient safety; REB = research ethics board; UGME = undergraduate medical education.

^aDenotes “Areas for further study” as identified by Wong.16

Overcoming qips educational barriers

As we have highlighted, there are a number of barriers that have been identified for integrating QI into graduate medical education (GME) curricula. From learner factors such as lack of exposure to QIPS concepts to institutional factors such as inadequate incentives for teachers, all of these barriers can make it difficult to integrate QIPS into local programs. Table 1 describes these barriers and possible ways to overcome these.30, 36, 37, 38, 39

A great opportunity for emergency medicine

Emergency medicine can help to identify and solve quality of care issues in health care. EDs often act as social safety nets and are therefore uniquely positioned to identify community patterns leading to worse health outcomes. The ED remains an environment in which most patients are subject to several transitions of care. Failures in communication, often during transitions of care, are the leading cause of adverse events in health care40 and the cause of over 60% of sentinel events according to The Joint Commission.41 Finally, the ED operates with increasing resource constraints, where demand often outstrips supply. These realities have become manifest because of increasing chronic disease burden within the populations we serve. For these reasons, and many more, quality‐of‐care issues and QIPS opportunities may be more easily observed in the ED environment. The application of QIPS modalities to these problems has the potential to create change ideas and solutions that may be applied throughout the health care spectrum. Examples of these may include reducing delays to diagnostic imaging (DI) using Lean methodologies or working with the DI group to shorten the time to test result. These issues are largely felt in high‐acuity and high‐volume areas but may demonstrate length‐of‐stay benefits for other settings in the hospital setting. Emergency medicine has the opportunity to lead all other specialties by instituting curricula that produced clinicians that are ready to engage in the difficult task of solving process issues throughout health care.

In keeping with the aforementioned best practices, literature review, postgraduate training requirements, challenges, and opportunities, the authors have summarized a curricular designed methodology which has been summarized in Data Supplement S1 (available as supporting information in the online version of this paper, which is available at https://doi.org/onlinelibrary.wiley.com/doi/10.1002/aet2.10052/full).

Conclusion: a challenge to emergency medicine

Considering the above, we feel that it is time to challenge our field to better integrate EM education and training with QIPS. Box 2 describes a possible positive outcome for our case vignette that would not be possible if GME educators do not take up the mantel of integrating QIPS into their programs. As such, we have three domains in which educators could begin this important work:

Box 2. Vignette Conclusion.

Dr. Vaughan returned to her institution and discussed the need for training with the ED chief, who identified another member of the faculty who had completed the Institute for Healthcare Improvement (IHI) Improvement Advisor course. Together, they liaised with the hospital and established partnerships with the performance and analysis group who, in turn, connected them with a network of faculty interested in QIPS work.

To build more local capacity, Dr. Vaughan, the ED chief and the faculty QIPS expert decided to develop an annual experiential QIPS curriculum for residents. Residents were asked to choose between longitudinal QIPS research or the current clinical epidemiology research model already existing in the department. The QIPS group looked to hospital adverse event registries, departmental quality of care, and hospital quality groups for possible QIPS project ideas in their department. They settled on the colearning experiential QIPS teaching model and began developing the curriculum.

Over the past few years, there had been a few cases of patients returning to the ED with urosepsis after culture sensitivities had not been verified. Both the ED and the hospital had identified this as a quality issue. The learners, as residents and staff in the graduate QI colearning curriculum, selected this clinical problem. They built a change team composed of residents, interested faculty, nurses in the ED, and representatives from the microbiology group. Over the course of the year, they would apply QIPS principles to solve the quality of care issue.

The ED chief recognized that this was meeting both educational and quality‐of‐care demands in the department and actively incentivized faculty participation in the coming years. Each year, the department now takes on two to three QIPS projects and has seen important safety gains.

Develop QI Capacity By Developing People and Future Leaders

Educational programs should develop longitudinal QIPS training programs for trainees, which align with institutional and departmental agendas. This should include interested faculty as previously described in the colearning model,36 allowing these individuals to also develop QIPS expertise. These faculty members will be available year after year for further initiatives and should be encouraged to seek deeper and broader knowledge in QIPS through formal training programs and channels.

Departments should additionally consider baseline training programs for all faculty members in QIPS concepts. These training programs should not include advanced application or statistical concepts, but should focus on how QIPS approaches and concepts intersect and overlap with the interests of faculty members (e.g., medical education, clinical epidemiology, administration). Good examples of such training programs are online models such as the IHI Open School certificate42 or regional training programs such as the IDEAS program.37 This would allow for a more universal understanding of its goals, methods, and tools. By investing in individuals in this way, training programs and clinical departments will recognize that they have trained their people to solve their problems.

Develop Innovative Ways to Train Residents in QIPS and to Solve Common ED Problems—Then Share Your Successes and Your Failures

Quality improvement and patient safety is a field in growth with many undiscovered applications, successes, and failures. Clinical quality and safety issues abound within the EM environment and clinical leaders are consistently looking for well‐designed solutions to these problems. Residents are an important part of these clinical environments, and as such, they need to participate within well‐guided multidisciplinary teams actively applying QIPS knowledge to solve safety and quality issues in the practice environment. Establishing large teams of faculty members, residents, nurses, and other ED staff increases the EDs capacity to perform more QIPS work in the future.

Thereafter, the dissemination of new data and change ideas will be essential for continued growth. Currently, several peer‐reviewed journals are publishing important QIPS work and some journals are devoted only to the dissemination of QIPS literature (BMJ Quality and Safety and BMJ Quality Improvement Reports).

Alternatively, emergency medicine has demonstrated itself a leader in online resources. In addition, several national and international conferences are beginning to highlight QI work. Academic emergency physicians should consider further dissemination of novel results and failures on sites such as JETem.org, MedEdPortal.org, ALiEM.com's IDEA series, International Conference on Residency Education (ICRE) QI abstracts, or scientific emergency medicine conference abstracts (e.g., Society for Academic Emergency Medicine [SAEM], Council of Emergency Medicine Residency Directors [CORD EM], Canadian Association of Emergency Physicians [CAEP]).

QIPS Is a Team Sport—Collaborate to Solve Bigger Problems

Quality improvement and patient safety project teams should be representative of the problem, which it is meant to solve. Many clinical delivery issues in the ED represent processes, which involve stakeholders from many departments, clinical specialties, and a variety of allied health providers. Broader collaboration between these groups dramatically increases the likelihood of an effective solution and implementation.

Emergency medicine practitioners are the quintessential team players in the hospital environment. QI training initiatives would allow educational programs to develop and evaluate learner collaboration, for faculty to further use their professional networks to address broad quality programs and to build on the concept of the ED as a system‐based specialty.

It is our duty to lead in the development of clinicians willing to take on the most pressing issues in health care delivery. In this pursuit, there are no silos of specialty or care, there are simply clinicians who are engaged in following the patient through his or her health care journey and breaking down barriers to the best possible care. Possibly our most pressing role is to ensure that emergency medicine trainees are ready to take up this challenge.

Supporting information

Data Supplement S1. A framework for evidence based QIPS curricula in emergency medicine.

AEM Education and Training 2017;1:301–309.