Residents Learn to Improve Care Using the ACGME Core Competencies and Institute of Medicine Aims for Improvement: the Health Care Matrix

Abstract

Objectives

This article describes how internal medicine residents at Vanderbilt University Medical Center learn to assess and improve care using the Institute of Medicine aims for improvement and the Accreditation Council for Graduate Medical Education core competencies combined in a tool called the health care matrix. The most important and popular use of the health care matrix has been with suboptimal care, in which care is not safe, timely, effective, efficient, equitable, or patient centered.

Background

The core competencies provide a means of defining why care was not safe, timely, effective, efficient, equitable, or patient centered. The Institute of Medicine aims for improvement are also important because they are used to frame most publicly reported measures of quality. Few residents have an understanding of these public measures and how their futures will be affected by the growing trend toward quality report cards.

Intervention

To help the residents understand the significance of public measures of quality, they learn to assess their patients as a “panel,” looking at the care they provide for patients with coronary artery disease and diabetes mellitus. Residents use the health care matrix to analyze 1 of their patients, and then as a group they select a health care matrix for their improvement project. The way the health care matrix is formatted and the sequencing of the core competencies allow for the analysis of the cells to lead to the final question “What was learned and what needs to be improved?” The residents are then taught the tools and methods of quality improvement and complete their project. Some of these projects have had a significant influence on external measures of quality for this organization. The article describes the 8-week course that residents complete, the use of the health care matrix, the analysis of the patient panel, and finally an example of a completed project in which they improve the timeliness of antibiotics administration to patients with pneumonia (a public measure of quality).

Introduction

The ultimate obligation in health care education is to develop clinicians who are competent to practice medicine, while teaching them to lead the transformation necessary in health care delivery and education. The Institute of Medicine (IOM) believes that our medical education systems have not kept pace with the changing face of health care.¹ To address this challenge, Vanderbilt University Medical Center began using a performance-based diagnostic tool called the health care matrix, which guides users to scrutinize the care of patients using the IOM aims for improvement and the Accreditation Council for Graduate Medical Education (ACGME) core competencies. The pilot work began with internal medicine residents in November 2003 and has continued since then. Each year, second-year residents during their ambulatory rotation spend 2 hours per week learning to analyze and improve care. To date, 1442 residents have taken this course. From December 2003 to December 2006, 674 residents focused on care of patients with coronary artery disease (CAD). From December 2006 to April 2009, 768 residents focused on patients with diabetes mellitus (DM).

Why Use the IOM Aims for Improvement?

The IOM aims for improvement are being used to frame most publicly reported measures of quality. Phase III of the ACGME Outcomes Project asks programs to focus on external measures of quality for programs and for individual residents. These measures will become vital to residents as they transition from their learning role to practicing clinicians. table 1 gives a description of the IOM aims for improvement, with examples of external metrics for which hospitals and their medical staffs are being held accountable.^2–8

Table 1.

Institute of Medicine (IOM) Aims for Improvement

Residents usually have little exposure to external measures of quality, despite the fact that as frontline clinicians they could have the most influence in improving them. This will be demonstrated in the example herein of improving the time to antibiotic therapy for patients arriving in the emergency department (ED) who have a diagnosis of pneumonia.

Academic medicine is being challenged to modify the structure and content of medical education, particularly the relationship between medical training and practice. It is necessary to teach clinicians to continually ask if the care they provide is safe, timely, effective, efficient, equitable, and patient centered (IOM aims for improvement) and to equip them with a method to capture their answers as data and report these analyses to organizational and educational leaders. As medical educators, we must ask ourselves how their education prepares them to face the health care crisis, how they will respond to demands for publicly reported measures of quality in their hospitals and of their own performance, and, most important, how they will learn to lead the transformation needed in the current health care culture. Ashton states that “When we treat our residents as if they are ‘invisible’ in our quality improvement programs, we in some measure abdicate our responsibility for their education and for the well-being of our current and future patients”.¹⁰ But who will teach these residents? Audet and colleagues found that “quality improvement still hasn't permeated the professional culture of medicine, although progress is evident.”^11(p843)

Documentation of health care improvement began in the early 1990s. Headrick and colleagues^12–16 have provided some of the best examples of clinical improvement in the setting of medical students and residents. Increasing curricula are being developed to teach systems-based practice and practice-based learning and improvement.^17,18 The ACGME core competencies have launched a “quiet revolution,”¹⁹ begun by those who understood the importance of teaching quality improvement to medical students and residents. There continues to be insightful literature on the core competencies in general and on improvements in particular; however, what is not as prevalent among the literature is how to use all the core competencies together in a way that clearly drives improvements.

Statement of Purpose

This article describes a means of assessing care using the IOM aims for improvement and the ACGME core competencies combined in the health care matrix.²⁰ This analysis provides valuable information that would not be captured using the core competencies alone. For example, the analysis begins with the question “Is care safe?” If it is not, the core competencies provide a means of identifying why care was not safe: was it medical knowledge, communication, professionalism, or system issues? However, the bottom-line question for practice-based learning and improvement if care was not safe is “What was learned and what needs to be improved?” Patient safety is a major issue in health care today, but each of the IOM aims for improvement highlights a different component of care. If care is not timely, this requires process analysis. If care is not effective, evidence-based medicine, guidelines, protocols, and variation must be examined. If care is not efficient, cost issues and the value of care must be considered. If care is not equitable, it is necessary to examine the cultural and socioeconomic issues. Finally, if care is not patient centered, the team must take a critical look at how it treats its patients. In essence, the health care matrix becomes a “forcing function” for quality improvement by bringing together 2 well-vetted sets of concepts. The residents are then taught the tools and methods of quality improvement and select a project that is often linked to public measures of quality, although some projects not directly linked to these measures are important to the residents. Residents can have a significant influence on the improvement of care in an immediate and pragmatic way.

Methods

The internal medicine residents at Vanderbilt University Medical Center are involved in an 8-week course (2 hours per week) during their ambulatory rotation in which they learn about the IOM aims for improvement and the ACGME core competencies. The course begins by asking the residents how their program provides education that develops each of the core competencies and the effectiveness of that training. Although the residents have had numerous evaluations based on the core competencies, their answers inform us that they still lack the understanding of systems-based practice and practice-based learning and improvement. The chief residents then introduce the health care matrix as a means of assessing the care of patients by using an example of a patient with a complex disease such as CAD and DM or a geriatric patient (figure 1).

Figure 1.

Curriculum Development Matrix for the Care of Geriatric Patients

This case sets the stage to ask the residents how effective they think they are in providing care to their patients with CAD and DM. Each resident identifies his/her patients with CAD or DM and enters the patients and information in a database. Once residents have their panels identified, they are asked to review the care using accepted evidence-based practice. A report is generated for each class, which is compared with classes before them.

Each resident is then told to select 1 patient to present to his/her colleagues using the health care matrix. This serves the following 2 objectives: (1) to teach the core competencies in the context of the IOM aims for improvement and (2) to provide valuable data to the program about care and educational issues. After all the residents have presented their patients, they select 1 case to become the group's improvement project. This provides an educational opportunity for residents to understand how to systematically improve the systems that govern the care of a population of patients.

Results

The teaching case identifies many issues that physicians face in caring for patients. For example, a particular patient may highlight that care is patient centered (she is happy with her physician and the attention she gets); she can see her physician when she wants (timely). However, her physician may be frustrated with the lack of progress in getting her DM and hypertension under control (not effective or efficient because she comes more often than she would need). She may need to rely on medication samples because of socioeconomic status (equity). In addition, her cultural lifestyle is one that revolves around church gatherings with a strong focus on meals. She desires only the care that is necessary to keep her out of the hospital, not to improve her glycosylated hemoglobin level or her hypertension, which is a dilemma for the physician. Therefore, much time is spent at each visit trying to get sample medications, taking valuable time away from dealing with her many health problems. This type of teaching case outlines the dilemma of measuring physician performance that does not take into account system and patient issues. This is especially important with the new requirement by the Joint Commission for ongoing professional performance evaluation.²¹

To teach residents about assessing their patients as a panel, they are taught to view patients in the aggregate. Residents from December 2003 to December 2006 focused on CAD, and residents from December 2006 to the present assess care of patients with DM. table 2 gives data on the performance of residents in the care of patients with CAD using aspirin, β-blockers, and angiotensin-converting enzyme inhibitors. For the care of patients with DM, residents look at glycosylated hemoglobin level and preventive care. Each class is compared with all previous classes. There exercise is usually the first time that residents are exposed to patients' data in this manner. Their first reaction is that they are not doing as well as they thought. However, they are reminded that the system either supports or hinders their efforts and reflect on how they must often work around the system to get care for their patients such as getting sample medications. The residents realize that they will need to take a proactive approach to measuring patient care once they are practicing clinicians. This can simply begin by placing their patients in a “panel” with identified measures of quality (process and outcomes).

Table 2.

Data From a Class of Residents Are Compared With Those From Previous Classes on the Care of Patients With Coronary Artery Disease Using Aspirin, β-Blockers, and Angiotensin-Converting Enzyme Inhibitors

The residents each present a patient using the health care matrix. By reflecting on the core competencies for each IOM aim and hearing similar stories from their colleagues, the residents soon become familiar with the core competencies and the contributions that these core competencies make to patient care. The residents then select 1 case as their improvement project. The following is an example of an improvement effort undertaken by residents for a patient with pneumonia in whom the first dose of antibiotics was delayed. An important “external measure of quality” for patients with pneumonia is the time to antibiotic therapy, and this metric is featured on the Centers for Medicare & Medicaid Services website (http://www.hospitalcompare.hhs.gov). We were not meeting the standard 240 minutes (at the time of this project, the metric was 240 minutes; it is now 360 minutes). A health care matrix on a patient with pneumonia had already provided information about the issues related to the care of these patients and the need for a system alert that a patient could have pneumonia. The first step was for the residents to flowchart the care of patients who came to the clinic having a possible diagnosis of pneumonia (figure 2). They knew that these patients could not be treated in the clinic because there was no quick access to intravenous antibiotics and no place to start treatment in the clinic while waiting for an inpatient bed. Like other patients who are very ill or febrile, these patients are always sent to the ED. Some faculty members thought that internal medicine residents should select the correct antibiotic; however, with the delay in getting an inpatient bed, this proved problematic for delivering the first dose of antibiotics within the specified time frame. In collaboration with his internal medicine colleagues, the ED resident initiated the writing of an algorithm for the order entry system so that ED physicians could indeed order the correct drug. Another ED resident and a few nurses became involved in the improvement project for patients with DM and helped flowchart the process from the perspective of the ED staff. They identified delays that could prevent the patient from getting the correct treatment within the prescribed 240-minute window.

Figure 2.

A Resident Worked With Nurses in the Emergency Department to Understand the Process for Patients With Pneumonia

This work attracted the attention of senior leaders, including the head of information technology, chief of staff, and chairs of internal medicine and emergency medicine. As a result, a workshop was held at which the residents' flowcharts of internal medicine and ED processes were used as the basis to create an “ideal” process flow. An action plan was created, and improvements were initiated. table 3 summarizes the performance metrics for the care of patients with pneumonia, which show improvement in time to antibiotic therapy from almost 600 minutes to the required 240-minute window. The continued improvement is posted on the Centers for Medicare & Medicaid Services website (http://www.hospitalcompare.hhs.gov).

Table 3.

Vanderbilt University Medical Center (VUMC) Performance on Publicly Reported Metrics of Quality Care for Patients With Pneumonia

Over the years, residents have been able to improve many processes of care, including time out for invasive procedures on the units, delays in discharging patients, Veterans Affairs medication issues, clinic efficiencies, working with home health nurses to prevent readmission within 30 days, and many others. Each of these improvements has helped reinforce how residents can improve care if given the proper tools and methods. At the end of the 8-week training program, the residents presented their project to their faculty, the head of internal medicine, the chief medical officer, and anyone else who was involved or could attend the session. If nurse colleagues or other clinicians participated in the improvement effort, they are also invited to the presentation. This “graduation” is important to the residents, as evidenced by the significant effort that went into their presentations.

Discussion

When residents use the health care matrix to assess patient care, they are able to identify problems with care and competency development issues that might have been previously unrecognized. Many traditional patient presentations or morbidity and mortality conferences focus primarily on medical knowledge. Even with a focus on the 6 core competencies, there is still a richness of detail that is missing if the IOM aims for improvement are not used. Residents have the most knowledge of the waste, delays, and work-arounds in our systems, so their voices (and those of all front-line clinicians) need to be encouraged, heard, documented, and acted on. The health care matrix is a tool that allows for this to happen. The best summary for this work is from a former chief resident who offered the following comments guiding postgraduate year 2 residents in using the health care matrix:²²

By considering the care of this patient in terms of the IOM aims for care and the ACGME core competencies in a systematic fashion, I learned some significant things about my care for the patient: I realized that on most levels I have been well trained from a standpoint of medical knowledge. It was in other areas where I observed either the holes in my training or the gaps in my understanding of how to care for my patients. I realized that, while I believed I was communicating with the patient well, I was not addressing her issues from the patient-centered perspective. It was eye-opening to realize that I might not know what her goals were and that they were probably strikingly different from my own. This might have played an enormous role in my effectiveness if we could have addressed shared goals. Finally, in comparing my care of this patient to the management of my entire patient panel, I realized that my care was much less systematic than I thought. It seemed that more often than not I was looking at each patient as an “outlier.” I have much to learn about systems-based practice, particularly regarding the management of a group of patients with chronic disease. Only when I understand how to apply this knowledge will I know how to effect systems-based learning and improvement.

The applications of the health care matrix are predicated on the notion that practice-based learning and improvement are the bottom line, the synthesis of all other cells. This must be linked to an action plan, with some individual or team being accountable for improvements. Another benefit of the health care matrix is that it can be used by clinicians in all disciplines. Nurses, respiratory therapists, audiologists, and others can use this tool to analyze their patients or to contribute to the analysis of patient care that might have been suboptimal. The residents learned this when they completed a health care matrix and realized that they did not have all the facts about the care of patients if they did not consult other team members.

Limitations

Validation of the health care matrix has not yet been undertaken because of the difficulties inherent in data collection and analysis without an Internet-based application. Funding has been provided by the University of Texas System to create an Internet-based application of the health care matrix. A plan has been created to validate the health care matrix with many other institutions who have been using it since 2004.

Next Steps

The health care matrix seems to be a practical heuristic for framing the care of patients and the core competencies in a way that leads to improvements. The residents who use the health care matrix remember the core competencies and can easily link them to issues of care. The most common use has been in morbidity and mortality conferences; however, additional applications are being discovered by those using this tool. For example, internal medicine residents in Taiwan are using the health care matrix to prepare for patient care, not just to analyze the care.²³ Once the health care matrix is Internet based, it will be possible to attach educational materials (such as a brief quality improvement course linked to practice-based learning and improvement), to input data (from the literature or from analysis of multiple matrices), and to produce reports by specialty, diagnosis, and other institutions that are using the health care matrix.