Clinical and Educational Outcomes of an Integrated Inpatient Quality Improvement Curriculum for Internal Medicine Residents

ABSTRACT

Background 

Integrating teaching and hands-on experience in quality improvement (QI) may increase the learning and the impact of resident QI work.

Objective 

We sought to determine the clinical and educational impact of an integrated QI curriculum.

Methods 

This clustered, randomized trial with early and late intervention groups used mixed methods evaluation. For almost 2 years, internal medicine residents from Dartmouth-Hitchcock Medical Center on the inpatient teams at the White River Junction VA participated in the QI curriculum. QI project effectiveness was assessed using statistical process control. Learning outcomes were assessed with the Quality Improvement Knowledge Application Tool–Revised (QIKAT-R) and through self-efficacy, interprofessional care attitudes, and satisfaction of learners. Free text responses by residents and a focus group of nurses who worked with the residents provided information about the acceptability of the intervention.

Results 

The QI projects improved many clinical processes and outcomes, but not all led to improvements. Educational outcome response rates were 65% (68 of 105) at baseline, 50% (18 of 36) for the early intervention group at midpoint, 67% (24 of 36) for the control group at midpoint, and 53% (42 of 80) for the late intervention group. Composite QIKAT-R scores (range, 0–27) increased from 13.3 at baseline to 15.3 at end point (P < .01), as did the self-efficacy composite score (P < .05). Satisfaction with the curriculum was rated highly by all participants.

Conclusions 

Learning and participating in hands-on QI can be integrated into the usual inpatient work of resident physicians.

What was known and gap

Integrating teaching and hands-on experience in quality improvement (QI) is thought to be essential to resident learning.

What is new

A randomized trial assessed the experience of internal medicine residents with integrated QI curriculum and hands-on experience with QI projects.

Limitations

Possible confounders were not controlled, and including only completed surveys may have biased the study in favor of high performers.

Bottom line

QI teaching and learning and hands-on experiences can be integrated into the usual workflow of residents' inpatient rotations.

Introduction

Opportunities for resident physicians to learn quality improvement (QI) often exist in “protected” experiences that are removed from their everyday clinical duties.^1–4 The new accreditation system builds on the 6 competencies by requiring sponsoring institutions to provide opportunities for resident engagement in QI and patient safety, as a required element of the common standards and a key component of the Clinical Learning Environment Review program.^5,6

Ideally, QI should be part of the usual workflow, allowing physicians to care for individual patients and improve the system of care delivery simultaneously.⁷ The clinical microsystem and the exemplary care and learning site are 2 models to operationalize this integration. A microsystem is defined as a small group of individuals (health care professionals, patients, and families) who work together in a particular setting to deliver care for a defined population of patients.⁸ This framework has been used to improve delivery of care and to teach residents about QI.⁹ The exemplary care and learning site model—pilot tested in 6 sites in the United States and Sweden—uses the clinical microsystem at its core, and incorporates 5 elements (health care professionals, teachers, learners, data, and patients) thought necessary for combining learning about and doing QI in clinical settings.^10–12 The 2 frameworks provide the foundation to test an integrated curriculum in which residents deliver care, improve the health care delivery system, and learn about both.

Previously, we identified key factors in the site, learner, and teacher domains for engaging residents in inpatient QI work.¹³ This article describes the clinical and educational outcomes from the implementation of an integrated inpatient QI curriculum.

Methods

This study was conducted at the White River Junction VA (WRJVA) Medical Center in Vermont. This rural 60-bed hospital has a primary academic affiliation with the Geisel School of Medicine at Dartmouth College and Dartmouth-Hitchcock Medical Center. Each inpatient medicine team consists of an attending physician, a supervising resident (second-year or third-year), 1 first-year resident, and 1 or 2 medical students. The residents rotate on the inpatient team for 4 weeks at a time, usually once or twice per academic year.

The study employed a clustered randomized trial with early and late intervention groups. Two of 4 internal medicine inpatient teams were randomly assigned to participate in the integrated QI curriculum (provided as online supplemental material) from April through December 2011 (early intervention group), with the nonintervention teams (no QI curriculum) serving as early controls (table 1). From January through early December 2012, all 4 teams received the educational intervention and constituted the late intervention group. All residents in the program were invited to complete the baseline assessment in March 2011, and interns new to the program were invited to complete the baseline assessment in July 2011 and 2012. Participation in the evaluation was optional.

Table 1.

Participant Exposure to Quality Improvement Curriculum by Time Point

This study was approved by the White River Junction Veterans Affairs Medical Center Research and Development Committee and the Dartmouth College Committee for the Protection of Human Subjects.

Resident QI work was evaluated by the focus, duration, and outcomes of the project. Outcomes were monitored using statistical process control.¹⁴ Educational outcomes were assessed at baseline, midpoint, and end point with the Quality Improvement Knowledge Application Tool–Revised (QIKAT-R),¹⁵ self-reported efficacy, and attitudes about interprofessional care.¹⁶ Resident satisfaction was assessed at midpoint for early intervention residents and at end point for late intervention residents. A focus group of nurses provided feedback at the end point about their experiences with resident teams and the QI work. Based on prior work,^17,18 we projected 80% statistical power with at least 36 residents per group to detect a 15% difference between baseline and end point and between early intervention and control groups.

At each time point, respondents provided free text answers to 3 QIKAT-R scenarios. QIKAT-R subsection scores (aim, measures, change) range from 0 to 3, and case scores from 0 to 9. Scores were summed, with the composite for each completed QIKAT-R instrument (3 cases) ranging from 0 to 27.¹⁵ Six faculty members from institutions in the United States and Canada who had experience with the instrument scored the QIKAT-R, with each scorer randomly assigned to approximately one-third of the completed instruments. Scorers were blinded to the assessment time point, and whether respondents had participated in the QI curriculum (intervention) or not (control).

We compared the composite scores between pairs of scorers for each time point by calculating intraclass correlation coefficients. Self-efficacy ratings are presented as a composite score, as well as subsection scores for aim, measures, changes, and interprofessional care. Mean QIKAT-R and self-efficacy scores assessed differences from baseline and between early intervention and control using the teams as clusters.

Two independent readers (G.O. and L.D.) reviewed and categorized the resident free text responses about satisfaction with the QI curriculum and identified common themes. The nurse focus group was digitally recorded (B.H.) and transcribed. Qualitative data were collapsed and reduced until major themes were identified (G.O. and L.D., separately, then in collaboration). Analytic differences were resolved by consensus.

Results

Response rates, based on the number of participants who fully completed the instrument, were 65% (68 of 105) at baseline, 50% (18 of 36) for the early intervention group at midpoint, 67% (24 of 36) for the control group at midpoint, and 53% (42 of 80) for the late intervention group (table 1). Twelve individuals contributed only baseline data and did not rotate to the VA during the study period. Five late intervention residents had been part of the early intervention group, and received a second 4-week exposure to the intervention during the study.

Clinical Outcomes

Resident teams initiated and completed 11 QI projects during the study period (table 2). Data were updated as appropriate for the project (eg, weekly for venous thromboembolism prophylaxis, monthly for time to initiate unfractionated heparin).¹³ Initial projects focused on the resident teams working on their own, while subsequent projects included interprofessional health care personnel from nursing, pharmacy, physical therapy, and administration. Some projects were discontinued due to infrequent exposure to the clinical condition (eg, antibiotics for pneumonia); baseline performance already at goal (eg, Foley catheter duration); or a scope that was too large (eg, falls prevention) for a resident team.

Table 2.

Description and Outcomes of Resident Teams' Quality Improvement Projects Initiated During Study Period (March 2011–December 2012)

Educational Outcomes

The composite QIKAT-R score and the “measure” subsection score increased from baseline to end point; other subsection scores also increased, although not significantly (table 3). The measure subsection increased significantly in the early intervention group from baseline to midpoint. However, there was no significant difference when compared directly to the control group at midpoint. Pairs of QIKAT-R scorers showed fair to very good agreement (0.36, 0.60, and 0.65 intraclass correlation coefficients).

Table 3.

Mean (SD) Quality Improvement Knowledge Application Tool–Revised¹⁵ (QIKAT-R)^a

The composite self-efficacy score and the measure subsection improved significantly from baseline to end point (table 4). Other self-efficacy subsections increased as well, but not significantly. The “aim” subsection self-efficacy score improved significantly in the control and the early intervention residents from baseline to midpoint, but at end point it was not different from baseline. When compared directly, there was no difference between the self-efficacy composite or subsection scores for control and early intervention residents at midpoint.

Table 4.

Mean Self-Efficacy Scores Indicating Level of Confidence in Quality Improvement Skills^a

At the final time point, more than 85% of residents (36 of 42) considered QI to be an essential or very essential component of future professional work. Residents in the intervention groups reported strong satisfaction with the curriculum, with more than 87% (35 of 40) agreeing that the integrated QI curriculum was valuable and should be continued. Attitudes toward interprofessional care were strongly positive and stable during the assessed time points. Only 1 of 20 interprofessional items showed a statistically significant change over time. At baseline, 55% (35 of 64) of residents expressed that time may be wasted in interprofessional care translating profession-specific jargon. This remained unchanged at midpoint (57%, 24 of 42) but improved to 29% (12 of 42) at the end point (P < .05).

Resident and Nursing Impressions of the Curriculum

Resident written comments at baseline showed misunderstanding of the basic structure of QI (“QI [is] really just clinical research, isn't it?”). Early intervention residents expressed their appreciation of the QI curriculum (“[it] is an interesting component of working at the VA”). Late intervention residents agreed the curriculum should continue, with comments such as “this will be an incredibly important element of physician duties going forward.” A minority did not favor the curriculum, with the most common concern being that QI may interfere with workflow on a busy service.

The nursing focus group revealed 2 main themes. First, nurses felt that the QI projects facilitated good communication with the residents. One stated that the QI projects “opened channels of communication between me, the team, and then on to the nursing staff.” Second, the projects provided meaningful opportunities for collaboration between the residents and the nurses. A nurse stated that “when you work collaboratively between physician[s] and nursing it just really strengthens the relationship in general.”

Discussion

In this study of QI education integrated with VA inpatient clinical care, internal medicine residents contributed to meaningful improvements in health care services, and they demonstrated small improvements in learning core concepts of QI and self-reported confidence with QI skills. The “learning while doing” QI experience was well accepted by residents and nursing staff working in collaborative teams.

This is an important educational model^1,3,4,7 because relying on “protected time” for QI dissociates care delivery from the improvement of care. The curriculum described here allows “the resident”—not the individual resident, but the figurative “resident”—to be a valuable member of QI efforts while on service (provided as online supplemental material). QI is sometimes perceived similarly to research that must be “owned” by a primary investigator; however, this ownership model limits the impact of QI to only when that individual is available and engaged at the QI site. This is a significant limitation for residents whose work location often changes with each rotation. In contrast, the residents in this study learned QI and interprofessional skills by engaging in meaningful work focused on common system-level goals with other health care professionals. The QI work was part of each inpatient team, not owned by one or a few residents.

Our educational outcomes showed modest benefit, with a 15% increase in the composite QIKAT-R scores (13.2 baseline to 15.2 end point), consistent with our hypothesized effect size. All 3 subsections increased, although only the increase for the “measure” subsection was statistically significant. There is likely a “dose response” to the curriculum, but the sample size provided insufficient power for a subgroup analysis, as only 5 residents received the intervention twice during the study period. In addition, for most residents, applying QI knowledge and skills during the delivery of care was a new experience they had not encountered during medical school or earlier in residency.

Our curriculum focuses heavily on finding and using the right data at the right level of the system, and the increased measure subsection scores for QIKAT-R and self-reported efficacy metrics reflect this. Despite their stated concerns about time, more than 87% of residents agreed that the integrated QI curriculum was valuable and should continue at the WRJVA. After the conclusion of this formal study, the curriculum continues as a key part of inpatient rotations. Faculty were initially supported by grant funding, and currently WRJVA leadership provides support for the curriculum through faculty time (0.1 full-time equivalent), data management, and a VA chief resident in quality and safety who serves as junior faculty.

Our study approach has limitations. Emphasizing didactic teaching might have strengthened the educational effect. However, a study priority was integrating meaningful QI work to impact health care services rather than simply teaching relevant QI content. Possible confounders for outcomes, such as other curricular content and societal trends in residencies, are challenging to control. We attempted to do so with early and late intervention groups, but the study was underpowered to detect a difference. In addition, a single “dose” of the intervention may not be sufficient for a meaningful impact on educational outcomes. By only analyzing fully completed instruments, we may have limited our selection to higher-performing residents. Finally, the interrater agreement for the QIKAT-R scoring was variable, an established accepted limitation of an instrument that relies on subjective assessment of free text responses.

Conclusion

Integrating meaningful QI work in inpatient medicine is 1 way to teach core competencies that improve health care delivery and allow residents to learn QI knowledge and skills concurrently. The model described is 1 of a range of effective approaches that will emerge as the focus on QI in program and institutional accreditation continues to evolve.