X + Y = Time for QI: Meaningful Engagement of Residents in Quality Improvement During the Ambulatory Block

ABSTRACT

Background 

Meaningful resident engagement in quality improvement (QI) remains challenging. Barriers include a lack of time and of faculty with QI expertise. We leveraged our internal medicine (IM) residency program's adoption of an “X” (inpatient rotations) plus “Y” (ambulatory block) schedule to implement a QI curriculum for all residents during their ambulatory block.

Objective 

We sought to engage residents in interprofessional QI, improve residents' QI confidence and knowledge and application to practice, and create opportunities for QI scholarship.

Methods 

In July 2015, the program provided dedicated time for QI in the ambulatory block. All categorical IM residents and 11 voluntary faculty mentors were divided into 10 teams based on clinic site and “Y” block schedule. Teams participated in resident-led, interprofessional ambulatory QI projects. Resident QI knowledge and confidence were assessed before the curriculum and 11 months after using the Quality Improvement Knowledge Application Tool–Revised (QIKAT-R) and surveys. QI project implementation and scholarship were tracked.

Results 

All categorical residents (N = 81) participated. Residents reported increased confidence in all QI skills, and they demonstrated increased knowledge, with mean QIKAT-R paired scores improving from 15.8 ± 4.6 to 19.1 ± 5.9 (n = 45 pairs, P < .001). A total of 9 of 10 teams implemented process changes, and 6 team project improvements have been sustained.

Conclusions 

This ongoing curriculum engaged IM and IM-psychiatry residents in QI during their ambulatory block using volunteer clinic faculty mentors. Residents demonstrated improved QI confidence and knowledge. The majority of resident projects were sustained and generated scholarship.

What was known and gap

Resident involvement in quality improvement (QI) faces challenges that include lack of time, longitudinal curricula, and faculty expertise.

What is new

Dedicated time for QI during the ambulatory block for internal medicine (IM) residents and IM-psychiatry residents, with faculty mentors and resident-led, interprofessional projects.

Limitations

Assessment tool lacks validity evidence; follow-up survey response rate was 65%.

Bottom line

The curriculum engaged residents in QI during ambulatory experiences, and resulted in improved QI confidence and knowledge, sustained projects, and resident scholarship.

Introduction

The Accreditation Council for Graduate Medical Education (ACGME) mandates that all residents be “integrated and actively participate in interdisciplinary clinical and quality improvement (QI).”¹ In addition, the Next Accreditation System and the Clinical Learning Environment Review hold training programs accountable for providing QI opportunities.^2,3

Despite these requirements, engaging residents in QI remains challenging.^4–6 Barriers include lack of dedicated time, busy resident schedules, and challenges related to working in interprofessional teams. Supervising faculty often lack QI teaching experience.^4,5,7–10 The University of Iowa Health Care (UIHC) IM residency faced similar barriers, and despite monthly workshops dedicated to QI topics, only 63% of residents reported QI participation on the 2014–2015 ACGME annual survey.

In July 2015, the program adopted an “X + Y” schedule, where “X” refers to inpatient rotations and “Y” to a designated ambulatory block.¹¹ We leveraged this opportunity to implement a new QI curriculum into residents' ambulatory block to address these barriers and engage all residents in QI. In this article, we describe the implementation, evaluation, and lessons learned from the first year of this ongoing curriculum.

Methods

In 2015–2016, the UIHC IM residency had 73 categorical IM and 8 IM-psychiatry residents who had continuity clinic at either the institution's outpatient facility or the Iowa City Veterans Affairs Health Care System (VAHC). All 81 residents participated in the curriculum.

Our program adopted a “4 + 1” (“X + Y”) schedule in which residents had their “Y” week every fifth week.¹² The “Y” week included 1 half day for independent QI work and 1 half day for group ambulatory educational curriculum, during which resident teams met every 10 weeks in a 1-hour “QI working session” for brief didactics and teamwork (Figure 1).

Figure 1.

Sample “Y” Week Schedule

Each “Y” week cohort was divided into 2 teams by clinic site. Teams consisted of 7 to 8 residents of all postgraduate levels, 1 to 2 faculty mentors, and volunteer interprofessional stakeholders, such as nurses, medical assistants, schedulers, and pharmacists, who were invited by residents to join the team. Each team selected a resident team leader responsible for coordinating team efforts and communication. Volunteer faculty mentors were physicians who supervised residents in their continuity clinics, and they were not required to have prior QI teaching experience. Mentors were required to attend a 1-hour orientation session or an individual meeting with faculty responsible for the curriculum. Mentors received all assignments, had access to all curricular resources, and were encouraged to attend working sessions and contact their resident team leader every “Y” week.

The curriculum was based on studies that actively engaged residents in QI.^{4,5,7,13–16} Curricular objectives were to (1) engage residents in interprofessional QI; (2) demonstrate residents' improved QI confidence and knowledge; (3) apply QI principles to residents' practices; and (4) create opportunities for resident scholarship in QI. While the curriculum incorporated some didactics, the emphasis was on skill development in the form of resident-led QI projects.^15,17–21 This longitudinal curriculum also emphasized teamwork and interprofessional engagement, and provided a venue for scholarship in QI.²² Table 1 outlines the curricular timeline and content.

Table 1.

Quality Improvement (QI) Curricular Timeline and Content

A flipped classroom model was employed to deliver the curriculum, promote teamwork, and facilitate peer teaching.²³ Specific assignments were sent to teams prior to their “Y” week to be completed during QI half days, before teams met during QI working sessions. SharePoint was used for curricular resources and team product dissemination. Resources included the Institute for Healthcare Improvement modules,²⁴ the textbook Fundamentals of Health Care Improvement: A Guide to Improving Your Patients' Care,²⁵ and articles from the literature.^26–28 Course directors (3 faculty members with QI expertise and VAHC's Chief Resident of Quality and Safety) met biweekly to develop and implement the curriculum, facilitated all QI working sessions, and served as resources to the teams.

QI projects targeted clinic process improvements because program evaluation surveys identified them as a local priority. Prior to curricular implementation, residents met in small groups to identify quality gaps in their clinics, and they created a list of potential improvement targets to aid in project selection. Residents chose targets based on team consensus (Table 2). To make members' tasks clear, teams were provided a worksheet outlining roles and assignments (provided as online supplemental material).

Table 2.

Resident Team Quality Improvement (QI) Project Targets, Interventions, and Impact

All projects were deemed exempt by the University of Iowa or the Iowa City VAHC Institutional Review Board.

Prior to and 11 months after implementation of the curriculum, we assessed residents' QI confidence by administering a survey (provided as online supplemental material). The survey was developed by the authors, based on the literature and curricular objectives, with no added validity testing. Using a 5-point scale (from 1, not confident, to 5, very confident), a rating of 4 or 5 was considered confident for each skill. We grouped skills into 4 levels: basic, intermediate, advanced, and aspirational (Table 3; Figure 2), and used the χ² test to analyze the percentage of residents reporting confidence for each individual QI skill, as well as the 4 skill levels. The postsurvey also asked for feedback regarding curricular strengths, areas for improvement, and team dynamics.

Table 3.

Resident Confidence in Quality Improvement (QI) Skills Before and After Curriculum^a

Figure 2.

Resident Confidence in Quality Improvement (QI) Skills

Note: The figure shows the percentage of residents expressing confidence (rating 4 or 5 on a 5-point scale) in basic, intermediate, advanced, and aspirational QI skills before (n = 75) and after (n = 53) curriculum. Residents reported statistically significant improvement at all skill levels (*P < .001).

With permission, we used the Quality Improvement Knowledge Application Tool–Revised (QIKAT-R)²⁹ to assess knowledge (provided as online supplemental material). Each resident completed 3 cases precurriculum and postcurriculum. The 4 course directors scored the QIKAT-R while blinded to resident and prestatus versus poststatus. We independently graded 8 cases and compared scores. Once consensus was reached, 2 raters independently graded each case. Interrater reliability was calculated using Fleiss-Cohen weights. Differences in QIKAT-R scores precurriculum and postcurriculum were compared using a paired t test. We also tracked project outcomes and QI scholarship.

Results

Prior to launching the QI curriculum, 93% (75 of 81) of residents completed the survey. Most residents lacked confidence in QI skills (Table 3; Figure 2), and only 28% reported prior involvement in a QI project. All categorical residents (N = 81) participated in the curriculum. The ACGME Resident/Fellow Survey showed improvement in resident-reported participation in QI, from 63% in 2014–2015 to 96% in 2015–2016 (n = 78 of 81, P < .001).

A total of 65% (53 of 81) of residents completed the postsurvey. Resident confidence improved significantly for all evaluated QI skills (Table 3; Figure 2). Analysis of 45 paired pre-post QIKAT-R²⁹ items demonstrated improved mean composite scores from 15.8 ± 4.6 to 19.1 ± 5.9 (P < .001, maximum score = 27). Interrater reliability using Fleiss-Cohen weights resulted in a weighted κ of 0.69 (95% confidence interval 0.63–0.75).

A total of 9 of 10 teams implemented process changes within 11 months, and 6 project improvements remain sustained 18 months later (Table 2). All teams presented their projects to clinic stakeholders after 4 months, and to peers 6 months after initiation of the curriculum; 7 teams presented posters at professional meetings.

Residents rated the curriculum an average of 4 on a 5-point scale for having “high educational value.” Recurring feedback on curricular strengths included learning by doing, working in teams, understanding the clinic microsystem, and being empowered to make changes. Common areas for improvement included difficulty maintaining team momentum between “Y” weeks, challenges with project feasibility, and a desire for inpatient-focused projects.

Mentor feedback indicated they felt the curriculum was valuable and that residents and mentors learned QI principles. Mentors also felt they were better able to teach QI and mentor QI teams. All mentors agreed to serve again the following year, except for 1 departing faculty member.

Each resident had 51 hours a year of dedicated QI time: 40 hours during individual QI half days, 5 hours of working sessions, and 6 hours for project presentations. Mentors estimated they spent an average of 2 hours per month (range, 1–3) in their role. Course directors estimated they each spent 2 to 4 hours a week on the curriculum (80–100 hours a year). Costs totaled $500 to purchase the online textbook and print posters.

Discussion

Our QI curriculum engaged categorical IM residents at all training levels in interprofessional QI, with participation rising from 63% to 100%. Residents reported it was educationally valuable, and they demonstrated gains in QI confidence and knowledge. The curriculum provided opportunities for resident scholarship in QI.

To our knowledge, this article is 1 of a few describing a QI curriculum embedded in the “X + Y” schedule.¹⁰ Keys to success were strong support from the residency program's leadership and provision of dedicated QI time for all residents. We addressed the lack of faculty mentors with QI expertise by recruiting and providing essential QI training to faculty who supervised residents in the clinics. The 4 course directors with QI expertise supported the faculty mentors and resident teams by attending all working sessions, monitoring team progress, and offering advice and assistance. This blend of support from team faculty mentors and course directors was important to curriculum success. Feedback from mentors indicated they felt the curriculum was valuable and provided them hands-on faculty development in QI.³⁰ Resident-led projects engaged residents, faculty, and other members of the clinic team in improvements that have the potential to improve morale and decrease burnout.^10,18,20

Strengths of the curriculum were that it was longitudinal, experiential, and allowed residents to apply QI to their own clinical learning environment. It also emphasized interprofessional teamwork and promoted scholarship in QI. It used well-known resources^24–28 and the flipped classroom model with electronic delivery of assignments to allow asynchronous learning and make working sessions more effective.²³ Requiring teams to present projects helped synthesize findings and identify next steps, while also providing peer education and encouraging coordination between teams and nonresident stakeholders.⁹

Challenges included difficulty maintaining momentum and communication between working sessions. Some teams had full contribution by all team members, while others reported unequal sharing of the work. In addition, some teams selected processes that were too complex, such as an intervention to improve the patient check-in process that was hampered by significant system barriers. It was difficult to keep the IM-psychiatry residents engaged, as they did not participate in the ambulatory curriculum during their psychiatry rotations.

Limitations of the study include a limited sample, a lack of validity evidence for the survey to assess QI confidence, and a response rate of 65% for the postcurriculum application.

This curriculum is currently in its third year, and we have encouraged alignment of resident-led projects with institutional and clinic priorities, and we have expanded some projects to include patient outcomes. Continuation of the curriculum has allowed projects with a large scope, such as a staffing model change, to be carried over into the next year. We have recruited 16 mentors, including 9 of the original mentors.

Next steps include assessing resident knowledge and skills after completing 3 years of the curriculum, and surveying alumni on their QI skills and the impact on their practice. We also plan to assess faculty mentors' QI knowledge and skills and confidence to teach and mentor residents. This model could be tested at other institutions and in other ambulatory care specialties.

Conclusion

Our QI curriculum engaged IM residents in QI experiences during their ambulatory block, with volunteer clinic faculty serving as mentors. Residents' QI confidence and knowledge improved. The majority of resident projects were sustained and produced scholarship.