Teaching Internal Medicine Residents Quality Improvement Techniques using the ABIM’s Practice Improvement Modules

Summary

Introduction/aim

Standard curricula to teach Internal Medicine residents about quality assessment and improvement, important components of the Accreditation Council for Graduate Medical Education core competencies practiced-based learning and improvement (PBLI) and systems-based practice (SBP), have not been easily accessible.

Program description

Using the American Board of Internal Medicine’s (ABIM) Clinical Preventative Services Practice Improvement Module (CPS PIM), we have incorporated a longitudinal quality assessment and improvement curriculum (QAIC) into the 2 required 1-month ambulatory rotations during the postgraduate year 2. During the first block, residents complete the PIM chart reviews, patient, and system surveys. The second block includes resident reflection using PIM data and the group performing a small test of change using the Plan–Do–Study–Act (PDSA) cycle in the resident continuity clinic.

Program Evaluation

To date, 3 resident quality improvement (QI) projects have been undertaken as a result of QAIC, each making significant improvements in the residents’ continuity clinic. Resident confidence levels in QI skills (e.g., writing an aim statement [71% to 96%, P < .01] and using a PDSA cycle [9% to 89%, P < .001]) improved significantly.

Discussion

The ABIM CPS PIM can be used by Internal Medicine residency programs to introduce QI concepts into their residents’ outpatient practice through encouraging practice-based learning and improvement and systems-based practice.

INTRODUCTION

While the Accreditation Council for Graduate Medical Education (ACGME) core competencies include practiced-based learning and improvement (PBLI) and systems-based practice (SBP), incorporating curricula for teaching these principles in Internal Medicine residency programs may be challenging. Other studies have described curriculum developed to teach PBLI and SBP^1–8; however, few of them use an easily assessible curriculum that can be implemented in Internal Medicine residency programs. Furthermore, few of these curricula have been rigorously evaluated using both educational and clinical outcomes.⁹

Simultaneously, recent reports of suboptimal quality of care in the United States^10,11 and patient expectations that physicans continually assess their personal quality of care¹² have led to changes in the American Board of Internal Medicine’s (ABIM) maintenance of certification (MOC) process. Beginning in 2006, Internal Medicine physicians fufilling MOC requirements could complete a practice performance self-evaluation, applying key components of quality assessment and improvement in their practice through practice improvement modules (PIMs). A recent study showed that the use of the ABIM’s diabetes PIM by practicing physicians resulted in meaningful behavioral change.¹³ The ABIM has also adapted PIMs for use in Internal Medicine residency programs. While there are no published studies documenting the use or effectiveness of PIMs in resident education, a few are currently underway.¹⁴ The ABIM’s PIM may be an accessible tool for the faculty to use to teach residents about quality improvement (QI) and to prepare them for MOC.

AIM

The aim of this educational innovation is to describe a longitudinal curriculum using the ABIM’s Clinical Preventive Services Practice Improvement Module (CPS PIM), which teaches residents quality assessment and improvement techniques and to assess the curriculum’s effect on resident confidence in QI skills.

PROGRAM DESCRIPTION

We have incorporated a quality assessment and improvement curriculum (QAIC), which centers on the use of the ABIM’s CPS PIM, into 2 required 1-month ambulatory rotations during the postgraduate year 2 (PGY2; see Table 1). The residents participating in this curriculum have their continuity clinics at 2 clinical sites. Residents rotate through the ambulatory blocks in groups of 10–15. Each month-long ambulatory rotation includes 4 90-minute QAIC sessions, in addition to the residents’ other outpatient clinical responsibilities and lectures. During the first PGY2 ambulatory block, the residents were introduced to the ABIM’s CPS PIM, which included resident’s web-based¹⁵ completion of 5 chart reviews, 5 patient surveys, and a system survey. The web-based CPS PIM then returns group level quality data on over 25 U.S. Preventive Screening Task Force–based screening indicators, ranging from cancer screening to immunizations.¹⁶ The next segment of the CPS PIM directs residents to reflect on this data and brainstorm ideas for improvement.

Table 1.

Structure of Quality Assessment and Improvement Curriculum

Residency year	Ambulatory block summer/fall	Ambulatory block winter/spring
Postgraduate year 2	Quality assessment block*	Quality improvement (qi) project block*
	Week 1: Intro to ABIM Clinical Preventive Services Practice Improvement Module (PIM)	Week 1: Review PIM data, chose QI project, develop aim statement and process map
	Week 2: Introduction to Quality Lecture	Week 2: Review process map and stakeholder interviews
	Week 3: PIM System Survey	Week 3: Develop and implement small QI project
	Week 4: Review summary data from PIM, Brainstorm areas for improvement	Week 4: Residents report Plan–Do–Study–Act Cycle results

*Each ambulatory training block consists of 4 90-minute lecture slots during ambulatory didactic curriculum

In the second PGY2 QAIC block, the residents reviewed the group-level PIM data again and chose 1 low-performing area as the focus for a group QI project. Faculty supervisors helped residents to define QI goals by encouraging projects that were: (1) feasible, (2) manageable for clinic staff, (3) under resident control, and (4) could be completed in the curriculum time frame of 1 month. The residents received lectures about developing aim statements, tutorials about process mapping skills, and finally tools for developing small Plan–Do–Study–Act (PDSA) cycles. Residents worked as a team to: (1) formulate an aim, (2) create a process map, (3) select specific measurements, (4) complete stakeholder interviews, and (5) develop and implement a small project to improve quality in their resident continuity clinic. In addition to the PIM data, which served as a “needs assessment,” the residents chose specific measures for each project. The effectiveness of the interventions was assessed through resident-led chart audits before and after the implementation of the QI initiative. The faculty assisted residents in performing a 2-sample test of proportions to compare the preintervention and postintervention data for each group project.

PROGRAM EVALUATION

From July 2006 to June 2007, 34 PGY2 residents participated in the first and second QAIC blocks. During the first block, the residents each performed a prospective sequential sample of 5 patient chart reviews, asked 5 patients to complete the ABIM CPS PIM Patient Survey, and met with clinic leadership to complete the ABIM Systems Survey. A total of 170 of 170 (100%) chart reviews were performed, and 130 of 170 (76%) patient surveys were collected. During a formal feedback session, the residents received group-level data regarding the quality of care they provided and their patient’s opinions of the quality of care the clinic provided. Group level data included 18 patient demographic data points, 9 outcome measures (e.g., body mass index [BMI] < 25, low-density lipoprotein cholesterol at goal), 49 processes of care measures (e.g., height recorded, breast cancer screening performed), and 107 systems enhancements (e.g., record contains an up-to-date medication list, practice provided smoking cessation counseling).

Residents noted suboptimal performance on 2 outcome measures: (1) BMI at goal of less than 25 in only 7 of 170 (4%) charts and (2) tobacco cessation counseling documented in 46 of 154 (30%) of relevant charts. The residents realized that while documentation of weight was exceptional (163 of 170 [96%] charts), height was recorded only 25% of the time (42 of 170), which was a major barrier to calculating the BMI for their patients. Results from the patient survey highlighted that 20% (38 of 117) of patients reported difficulty obtaining medication refills. Using this data, residents were able to brainstorm QI goals including: (1) increasing percentage of charts that had height recorded as a step to increase BMI screening, (2) increasing percentage of patients receiving smoking cessation counseling, and (3) improving the refill process. The CPS PIM then asked residents to develop a plan for improvement and to report results of this improvement.

During the second QAIC block, the residents developed a group project that was implemented in their continuity clinics addressing 1 of the 3 areas of improvement highlighted by the CPS PIM. The 3 projects included: (1) BMI collaborative (January block), (2) tobacco cessation collaborative (March block), and (3) the medication refill collaborative (May block; see Table 2).

Table 2.

Title: Summary of Quality Improvement Projects Resulting from Quality Assessment and Improvement Curriculum

QI projects	BMI collaborative	Tobacco cessation collaborative	Refill collaborative
PIM measure addressed	Chart review	Chart review	Patient survey
	Weight recorded in chart, 163 of 170 (96%) charts Height recorded in chart, 42 of 170 (25%) charts BMI at goal <25, 7 of 170 (4%) charts	Smoking-related illness documented, 154 of 170 (90%) charts Smoking cessation counseling recommended, 46 of 154 (30%) charts	In the past 12 months, how much of a problem has it been to get a prescription refill from this practice? 38 of 117 (20%) patients answered either “a small problem” or “a big problem”
	System survey	Patients survey	System survey
	Physician provides nutrition/exercise counseling Other designated staff provides nutrition/exercise counseling	Patient reports practice excellent at smoking cessation counseling 18 of 131 (18%) surveys	Records contain an up-to-date medication list Practice documents communication with patients about medications
		System survey
		Practice has a formal smoking cessation program
Aim	To improve clinic documentation of height, weight, and BMI to 85%	To improve documentation of smoking status from 30% to 60% within the last year	To decrease the number of inaccurate medication list dictations from 25% to 12.5%.
Intervention	Nursing staff to ask or measure height and document on vital sheet Resident to calculate and dictate BMI	Nursing staff to ask smoking status and document on vital sheet Nursing staff to give patient “readiness to quit” handout if patient is a current smoker Resident to counsel and dictate tobacco cessation counseling	Proper medication list flyer posted in every resident clinic cubicle Local pharmacies faxed a list of current residents and the clinic medication refill center fax/phone number Resident business cards updated
Resident Education	Clinic curriculum developed about calculating BMI, obesity risks, and counseling options	Clinic curriculum developed about tobacco risks and counseling options	Clinic curriculum developed about refill process at pharmacy and clinic medication refill center
Publicity	BMI charts/stickers disseminated Project information given at lunch lecture and by mass email	Tobacco status documentation stickers added to BMI charts Project information given at lunch lecture and by mass email	Proper medication list flyer posted in clinic Project information given at lunch lecture and by mass email
Patient education	Obesity handout developed	Tobacco cessation handout developed	Patient medication list developed
Results	Documentation of height dictated improved from 22 of 206 (11%) charts to 143 of 163 (88%) charts (p=.001) Weight dictated improved from 185 of 206 (90%) charts to 158 of 163 (97%) charts (p=.008) BMI dictated improved from 9 of 206 (4%) charts to 129 of 163 (79%) charts (p<.001)	Documentation of smoking status improved from 31 of 75 (41%) charts to 56 of 84 (67%) charts (p<.001)	Inaccurate medication list improved from 54 of 216 (25%) charts to 27 of 296 (9%) charts (p<.001)
Future Work	Create referral source for exercise and nutrition counseling to decrease average BMI in clinic	Create a referral process for tobacco cessation clinic to help patients stop smoking	Consider fax to email refill converter to decrease paperwork burden for clinic medication refill center

To address the poor rate of height documentation and, by default, BMI documentation, the BMI collaborative worked with the nursing staff to integrate height into the triage process, posted BMI charts throughout the clinic, and educated residents on the importance of calculating and documenting BMI. Residents developed an educational module to be used by the clinic preceptors during the first 30 minutes of the continuity clinic, which was set aside for standardized clinic education. The BMI collaborative was able to increase documentation of height from 22 of 206 (11%) of all charts in the week before intervention to 143 of 163(88%) of all charts in the week after intervention (P = .001). Although not identified as a problem, it is noteworthy that dictation of weight also improved from 185 of 206 (90%) to 158 of 163 (97%; P = .008). Ultimately, rates of dictation of BMI improved from 9 of 206 (4%) to 129 of 163 (79%; P < .001).

The tobacco cessation collaborative worked with clinic nursing staff to document smoking status and to give patients a “readiness to quit” handout if the patient was currently smoking. The collaborative also educated residents on tobacco cessation counseling and documentation. The tobacco cessation collaborative improved documentation of smoking status from 31 of 75 (41%) charts before intervention to 56 of 84 (67%) charts after intervention (P < .001).

The refill collaborative worked with the continuity clinic medication refill center staff and the local pharmacies to streamline the refill process for both patients and residents. The key part of their intervention was to educate residents on the proper use of the clinic medication refill center and the importance of having an accurate and up-to-date medication list. The residents found in their preintervention data that many residents changed a medication dose/stopped a medication/started a new medication in the dictated plan but did not make the corresponding change in the dictated medication list that was used by the medication refill center. These outdated medication lists were deemed “inaccurate” as they had the potential to lead to improper refills for patients and increased paperwork for the residents. By educating their peers, the residents were able to decrease the number of “inaccurate medication lists” from 54 of 216 (25%) preintervention charts to 27 of 296 (9%) postintervention charts (P < .001).

Themes that were common to all 3 projects included: (1) interdisciplinary effort between residents and clinic staff, (2) resident education that focused on correct documentation, (3) a publicity campaign, and (4) a patient education component. The group QI projects also resulted in a spillover into the attending clinics and into future QI projects. For instance, the attending clinic’s patients also had height and smoking status documented on clinic vitals sheets. The resident groups tended to build on previously implemented initiatives to improve awareness of their project. For instance, the tobacco cessation and refill QI projects added stickers/reminders to the laminated BMI charts that were posted in cubicles around the clinic.

To evaluate the effect of this curriculum, we administered a previously described [2] self-assessment tool of QI skills to the residents before and after the first 2 blocks of the curriculum. The percent of residents reporting that they were comfortable writing a clear aim statement improved from 71% (24 of 34 residents) to 96% (27 of 28 residents; P < .01, chi-square test). The percent of residents reporting that they were comfortable using a PDSA cycle improved significantly from 9% (3 of 34 residents) to 89% (25 of 28 residents; P < .001, chi-squared test). Preliminary evaluation of resident’s qualitative responses has shown a positive reaction to the QAIC curriculum. The following comments were excerpted from the free-text commentary on the post-assessment:

“It helped me realize that I can change things when they don’t work well, instead of just criticizing the current system.”

“I learned how to break QI projects into small manageable steps.”

“I gained experience using the PDSA cycle which is essential to my research interests.”

DISCUSSION

Teaching Internal Medicine residents PBLI and SBP can be challenging because of limited availability of standardized curricula, inadequate faculty experience, and the complexity of QI, which is dependent on local culture and context. The ABIM’s PIM provides a standardized, web-based, evidence medicine–based, affordable, and efficient tool to teach residents principles of quality assessment. The QAIC discussed above integrates the ABIM’s PIM with basic QI lectures to engage residents in hands-on learning of QI principles. Group QI projects empower residents to reflect on the quality data summarized by the PIM and to make significant changes in their continuity clinic experience.

Resident confidence in QI skills appeared to improve as a result of this curriculum. Moreover, the residents were also able to make statistically significant changes to their practice in the continuity clinic with their group QI projects. Residents were able to disseminate their findings in scholarly work including: an internal resident research day, a hospital quality fair, a monthly Quality Improvement in Progress seminar series, and a regional Society of General Internal Medicine meeting.

Challenges to implementing this curriculum include finding funding, time, and faculty with QI experience. Finding time within preexisting ambulatory blocks and creating PBLI/SBP electives are 2 options for implementing this curriculum into already busy resident schedules. Funding issues included financial resources (about $25 per resident) to buy the ABIM PIM for each resident and funding for QI project support, including photocopying expenses, buying height tools, and laminating BMI charts. Other challenges include developing teams that include both residents and clinic staff members and choosing appropriate measures to document the effect of each QI intervention. Although institutions may not have faculty experienced in QI, the ABIM PIM is a simple tool that inexperienced faculty can adopt to teach QI.

The ABIM’s CPS PIM can be used by Internal Medicine residency programs to introduce QI concepts into their residents’ outpatient practice through encouraging PBLI and SBP. It can also be incorporated into preexisting ambulatory rotations in Internal Medicine residency programs.