A Joint Quality Improvement and High-Value Care Curriculum in a Limited-Resource Setting

Abstract

Introduction

Since the release of the Institute of Medicine's To Err Is Human, there has been an increased focus on quality improvement (QI). QI training is now a requirement monitored via ACGME's clinical learning environment review committees. Given the significant cost of health care waste, teaching physicians to incorporate costs and value into medical decision making is crucial. Increasing information is available on methods to teach high-value care (HVC), but there is little information on combining HVC with QI. As these topics are intimately linked in efforts to provide effective, efficient care, a joint curriculum is a feasible solution.

Methods

We adapted material from two online resources—(1) Institute of Healthcare Improvement Open School and (2) American College of Physicians High Value Cost-Conscious Care Curriculum—to create a combined curriculum for use in a limited-resource setting. Our curriculum is divided into 10 seminars, each including both QI techniques and HVC theories, which are reinforced using a series of patient scenarios. Residents apply their knowledge in self-directed projects presented in the final seminar. Evaluation includes a pre-/postexposure QI knowledge application test, survey of self-assessed knowledge, and anonymous course feedback.

Results

For the 46 residents who completed the series, a statistically significant improvement in both tests was measured, and feedback was positive overall. Tailoring our in-seminar patient scenarios allowed residents to demonstrate their HVC knowledge acquisition.

Discussion

This seminar-based curriculum can be adapted to the time availability in any residency program and transfer to other disciplines with modification of the patient scenarios.

Educational Objectives

By the end of this curriculum, the learner will be able to:

• 1.Describe common principles and techniques of quality improvement, including models for improvement, plan-do-study-act cycles, root cause analysis, and process mapping.
• 2.Define high-value, cost-conscious care using concepts of health care waste, overordering tests, and choice of medication and its effect on cost.
• 3.Implement a five-step model for determining necessary care and optimizing quality.
• 4.Demonstrate these quality improvement techniques and high-value care concepts in a small-group project.
• 5.Define culture of safety and review how to report adverse events at various clinical sites.

Introduction

Since the release of the Institute of Medicine's To Err Is Human—with its likely underestimated report of 44,000–98,000 annual deaths due to medical errors—attention has turned to quality improvement (QI) and the need for residents around the country to receive additional QI training.¹ As of 2012, the ACGME mandated that QI training be a requirement for all trainees, monitored via clinical learning environment review committees with a focus on resident participation in QI initiatives.² Evidence suggests that QI training during residency is associated with involvement in QI activities after graduation.³

As more information surfaces about the significant cost of health care waste, a focus on teaching physicians to incorporate costs and value into medical decision making becomes crucial, especially as doctors' decisions account for over 80% of wasteful spending.⁴ More and more information is available on effective ways to teach high-value care (HVC), but there is little information on the concept of combining HVC with a QI curriculum. At times, in an effort to definitively diagnosis patients, care providers order unnecessary tests that can pose significant risk and cause harm. Quality care delivery requires a firm understanding of evidence-based medicine, risk-benefit ratios, health care costs, and population health issues. As a result, QI and HVC are intimately linked, and with available time lacking due to duty hours and competing educational priorities, an integrated QI/HVC curriculum is an urgent necessity and a feasible solution for internal medicine residency programs.

Due to these demands, we have created a limited-resource curriculum combining core concepts of both QI and HVC, a natural combination given that both quality and cost of care factor into the ultimate goal to provide safe HVC. We started by researching how to develop a QI curriculum. We conducted a thorough literature review of various QI curriculum models at other internal medicine, emergency medicine, and family medicine residency programs (Appendix A). We also viewed the DVD version of the Mayo Clinic CME course entitled Teaching Quality Improvement and Patient Safety in Health Professions Education. We benchmarked with other internal medicine programs by reaching out to faculty already teaching QI at other institutions, including Duke University; University of California, San Francisco; University of California, Davis; and California Pacific Medical Center. We discovered that the Institute for Healthcare Improvement (IHI) offers free access to online modules that are highly recommended and widely used.⁵ At national professional meetings, we learned about the American College of Physicians (ACP) High Value Cost-Conscious Care (HVCCC) online curriculum,⁶ which also provides a free, comprehensive online resource for teaching core concepts. We adapted material from the IHI Open School online modules and the ACP HVCCC online curriculum to create and study a 2-week combined curriculum. We decided to also use the Quality Improvement Knowledge Assessment Tool (QIKAT), which has evidence of reliably demonstrating increased QI knowledge among residents who take part in other curricula, as a measure.^7,4 With this tool, we were able to show an increase in QI knowledge as a result of our curriculum and also noted clear HVCCC concept understanding among residents during the in-seminar patient scenario work. Although this curriculum was initially given in 2-week blocks, we have since transitioned it to be longitudinal, administered during residents' 10 ambulatory blocks over the course of the year.

Methods

To implement the curriculum, we recruited seven additional faculty members to form a team of nine faculty advisors to teach and guide the residents. Faculty members were recruited without means of reimbursement, either financially or through time, due to our limited resources. Since the course material included concepts new to most faculty, we asked all faculty to complete the IHI modules and review the ACP online curriculum as preparation for faculty development. During our faculty development sessions, we reviewed the course objectives and ran through each of the seminars and associated patient scenarios. We assigned each faculty member one seminar to teach consistently, but we also assigned each faculty member one backup seminar. This allowed for coverage if the primary faculty member was unavailable for a given session. We ensured that all faculty members would be familiar with all seminar topics and content, even if they would not be responsible for teaching those seminars, to enhance quality control throughout the curriculum. The program director and chair of the department supported our efforts by encouraging faculty to volunteer as instructors, ensuring that resident schedules were adjusted to accommodate the course, and supporting residents in their project work.

This curriculum is a series of 10 seminars that can be given over any allotment of time; although originally scheduled as a daily seminar over a 2-week elective, we transitioned it to a longitudinal curriculum given every 5 weeks over a year. The facilitator guide (Appendix B) provides a detailed overview of the curriculum to assist the faculty or facilitator in preparation. The 10 seminars introduce both QI and HVC concepts, often concurrently, and require 90 minutes to 2 hours each. The core didactic portion of each seminar (Appendices C–K) varies from 60 to 90 minutes, depending on the level of group participation and the time it takes each group to work through the patient scenario activities (Appendix L). We then allot 30 minutes to an hour for project application. The resident handout (Appendix M) should be given to participants prior to the first seminar to communicate expectations.

During the first seminar, residents take a preexposure QIKAT⁸ and are divided into small groups of two to four. In these groups, they apply different techniques learned throughout the course to develop a QI project. Residents have time during each seminar to discuss their project in relation to the seminar topic, and after the seminar concludes, residents can continue project work on their own time. The faculty members who teach the seminars should be available for advising and mentoring the projects as they are likely stakeholders.

Prior to each seminar, residents are assigned suggested IHI modules⁵ (detailed in Appendix B) to introduce or better illustrate the topics that are then summarized in the didactic PowerPoints. The supplemental modules are currently available without charge to academic institutions when registered as a student, resident/intern, or teacher/professor. Facilitators may wish to modify the course to encourage students to complete 16 required IHI Open School courses, after which they could earn the IHI Basic Certificate.

Each seminar includes QI techniques coupled with HVC theories, which we then apply through a series of inpatient and outpatient scenarios for a single patient (Appendix L) to yield a component of experiential learning. Curricular focus during these discussions includes reviewing health care waste expenses, access to care, decision-making processes, QI research techniques, and communication of these tenets with patients. To facilitate the seminars, no additional supplies or equipment are needed outside of a computer with projector, a whiteboard or flip chart, and a small conference room.

The series concludes with each resident group giving a 15- to 20-minute presentation on its proposed project to its colleagues and a faculty member, illustrating different concepts, the proposed influence on cost, and plans for how to carry the project forward. After the group discussion on the projects, residents take a postexposure QIKAT⁸ and provide anonymous feedback for the course.

To measure whether residents had an increase in knowledge as a result of the curriculum, we compared the results of paired pre- and postexposure QIKATs.^7,4 The QIKAT is a validated tool for measuring QI knowledge and uses three clinical scenarios with a total possible score of 15. It includes a survey rated on a 5-point scale that measures self-assessed knowledge and comfort with QI topics. Three faculty instructors graded QIKATs independently. The first 20 QIKATs were graded jointly, with a Pearson correlation score of .99–1 and Bland-Altman plots showing no consistent bias, ensuring interrater agreement. Of note, this validated test has since been updated to create a less cumbersome and more reliable means of evaluating QIKATs.⁹

Results

Of the 100 categorical and primary care residents we have had at our institution over the 12 cohorts (4 years) of study, 60 took at least part of the course. We obtained paired pre- and postexposure QIKAT scores for 77% of those 60 residents (N = 46). The Table includes mean pre- and postexposure QIKAT scores and survey results.

Table. Resident Pre/Post QIKAT and Self-Evaluation Survey Scores.

Variable	N	M	SD	Minimum	Maximum
Pre-QIKAT	46	7.02	2.74	1	12
Post-QIKAT	46	11.1	3.01	3.33	15
Presurvey	46	2.13	0.51	1	3.08
Postsurvey	46	3.26	0.36	2.5	3.92

Abbreviation: QIKAT, Quality Improvement Knowledge Assessment Tool.

Using a two-tailed paired t test with α = .05, both scores increased significantly from pre- to postexposure. The mean change for QIKAT was 4.08 (95% CI, 3.15–5.01; p < .0001)—an increase in mean total score from 7.02 to 11.10. QIKAT score improvement indicates improved knowledge of QI concepts. The mean change for the self-evaluation survey was 1.13 (95% CI, 0.99–1.28; p < .0001)—an increase in mean self-assessment score from 2.13 to 3.26. Survey score improvement indicates an increase in residents' self-assessment of their comfort with QI.

Residents expressed that the series of patient scenarios was a useful adjunct to the course, but reviews on the usefulness of the IHI modules were mixed. Most residents felt that more time was needed for project work, with a few commenting as follows:

• •“[Need] more time/guidance on QI project.”
• •“[Need] more time to work on projects and for data gathering.”
• •“[Strengths were the] small groups; ample time for projects; relevant.”
• •“Identifying small things in our daily work flow that has a huge impact on healthcare that can be addressed and improved; new concepts learned.”
• •“[Needs] faculty buy-in.”

Almost all residents expressed a better appreciation of QI and HVC issues in their everyday practice. Specific comments included the following:

• •“I really enjoyed learning about QI. Would like to do more with this in the future—maybe actually implement a project.”
• •“I now feel like making a change to improve something in the hospital is something I could do.”
• •“This should be taught to medical students.”
• •“I enjoyed learning about actual costs of testing; I would be much more conscious of how and why I use tests on my patients. I enjoyed the seminars.”
• •“[Would be helpful] to meet with QI department, risk management to learn more about QI jobs in the real world.”

Faculty facilitators noted clear HVCCC concept understanding among residents during the in-seminar patient scenario work, such as the role-play of how to talk to patients about not ordering tests and the medication reconciliation exercise. We also subjectively noted an increase in resident-initiated discussion of these topics during clinical rounds in both ambulatory and hospital settings following completion of the curriculum.

During the initial run of this curriculum, residents brainstormed processes surrounding multiple areas, including accessing medications on hospital discharge, expediting radiology ordering, and giving our medical assistants a greater role in Pap smear completion. However, as this was a 2-week elective and project completion was not required, only one resident chose to pursue her project on discharge medications. Since we transitioned the curriculum to a longitudinal course, residents routinely complete their QI projects. Examples of successful projects include incorporating macro templates for cardiology discharge instructions to improve note quality and decrease time spent on their completion, creating a template to provide patients with introduction sheets including descriptions and pictures of team members so as to improve patient recognition of their physicians, and resident-led performance audits to identify systematic ways to increase microalbuminuria screening in diabetic patients.

Discussion

After offering this course 12 times and evaluating the residents' feedback and QIKAT scores, we have concluded that the overall design of combining QI and HVC was successful and feasible. This course can be offered in multiple formats and has the flexibility to be altered based on time available in each residency program, even transferring to other disciplines with modification of the patient scenarios. Our once-daily seminar in a 2-week structure could be adjusted to one seminar weekly over 10 weeks or two seminars a day over 1 week.

Lessons Learned

We initially administered the ACP survey about HVCCC knowledge, but we found that the survey focused on very concrete facts (e.g., What was the approximate annual expenditure of health care costs in the US in 2010?) as opposed to the broader HVCCC concepts that we included in our objectives, such as causes of health care waste, overordering of tests, and choice of medications. Therefore, we did not use the ACP survey as a measure of course objectives for HVCCC. Instead, we tailored our in-seminar patient scenario work to allow residents to show their knowledge acquisition regarding these concepts. The use of a series of longitudinal patient scenarios enhances trainee education on the topics, giving a chance for the residents to apply their new knowledge. After the third session, we also changed the order of the seminars (format presented) to progress the subject matter optimally and enhance flow in parallel with the patient scenario.

Our financial and time resources were limited. Our administration encouraged faculty involvement in teaching and allowed for resident scheduling in the elective, but we still had several issues retaining both parties. Our secretarial support was also limited; we had some assistance with room reservations and ordering of supplies, but most logistical issues were self-managed.

Faculty retention

Faculty were not provided any protected time or financial compensation for teaching, and as a result, faculty retention was one of the most difficult aspects of our curriculum. In a rotation that ran as frequently as every 2 weeks, continued faculty commitment was trying, in part due to burnout from time constraints, lack of compensation, and lack of structured faculty development. Another significant difficulty was the relative curricular isolation of QI and HVC topics to this 2-week period. Our institution, like many others, still struggles with culture change surrounding error reporting and health care waste management. As a result, residents expressed that they did not always feel comfortable discussing errors and cost concerns with attendings outside of our core faculty. We recommend QI and HVC topics be presented to all faculty in order for a shift in an institution's approach.

As an incentive for faculty recruitment, our institution now weighs participation in this curriculum towards individual career advancement and promotion. Faculty may now use their participation to accrue ABIM Maintenance of Certification points. Other possible incentives would include giving financial compensation or protected time to faculty. We are also transitioning to a 3-year curriculum with the core lectures to be given by PGY3s in order to both solidify the PGY3s' knowledge and partially off-load direct faculty responsibility.

Resident retention

In the first few cycles of the curriculum, residents were frequently pulled from the elective prior to completing their 2 weeks to provide necessary backup coverage on other services, and therefore, we have paired pre- and postexposure data for only 77% of the participants. The remaining 23% of participants did not complete the entire course. Similarly, despite overwhelming interest among residents about getting involved in QI projects, they struggled with finding time to continue their projects within the constraints of inpatient rotations once they left the protected time of the QI block.

To address retention issues for residents, we transitioned to a more longitudinal structure with 1 half-day every other week. We believe this better facilitates knowledge retention and protected time for project work on a monthly basis. The longitudinal approach also allows all residents to take the course, as opposed to 60% previous participation. We have also worked with the department and university to provide more forums for presentation of the resident projects. We encouraged leaders of an already-existing institutional research day to add a QI category, providing a place for residents to showcase their work (and thus have it prepared to submit to regional and national conferences). We also worked with department heads to have an annual Quality Improvement Grand Rounds where the “Best of Resident QI” is presented to the Department of Medicine every year. This has helped the residents recognize the importance of the material and the support of the chair. It also gives participating faculty another incentive with increased scholarly activity.

Appendices

A. Literature Review.docx

B. Facilitator Guide.docx

C. Seminar 1 - Introduction to QI & HVC.pptx

D. Seminar 2 - Process Maps & RCA.pptx

E. Seminar 3 - Model for Improvement & Measuring for Quality.pptx

F. Seminar 4 - Change Management.pptx

G. Seminar 5 - Communication and Cost.pptx

H. Seminar 6 - Culture of Safety.pptx

I. Seminar 7 - Human Factors Engineering.pptx

J. Seminar 8 - Health Care Costs, Payment Models, & Equity in Delivery.pptx

K. Seminar 9 - Communication After Adverse Events.pptx

L. Pat Smith Scenarios.docx

M. Resident Handout.docx

All appendices are peer reviewed as integral parts of the Original Publication.