A Didactic and Experiential Quality Improvement Curriculum for Psychiatry Residents

Abstract

Background

Quality improvement (QI) education in residency training is important and necessary for accreditation. Although the literature on this topic has been growing, some specialties, in particular psychiatry, have been underrepresented.

Methods

We developed a didactic and experiential QI curriculum within a US psychiatry residency program that included a seminar series and development of QI projects. Evaluation included resident knowledge using the Quality Improvement Knowledge Application Tool, implementation of resident QI projects, and qualitative and quantitative satisfaction with the curriculum.

Results

Our curriculum significantly improved QI knowledge in 2 cohorts of residents (N  =  16) as measured by the Quality Improvement Knowledge Application Tool. All resident QI projects (100%) in the first cohort were implemented. Residents and faculty reported satisfaction with the curriculum.

Conclusions

Our curriculum incorporated QI education through didactic and experiential learning in a moderately sized US psychiatry residency program. Important factors included a longitudinal experience with protected time for residents to develop QI projects and a process for developing faculty competence in QI. Further studies should use a control group of residents and examine interprofessional QI curricula.

Background

Educating residents about quality improvement (QI) is important for many reasons. QI spans 2 of the Accreditation Council for Graduate Medical Education (ACGME) 6 competencies—practice-based learning and improvement and systems-based practice. The ACGME requirements state that residents must be “integrated and actively participate in interdisciplinary clinical quality improvement and patient safety programs.”¹ Performing QI activities will soon be linked to maintenance of certification, maintenance of licensure, and even reimbursement by Medicare.²

Reports of QI education within residency training are increasingly common, with primary care representing most of this work.³ A 2009 review found 28 reports documenting residency QI curricula, 9 of which described QI didactic instruction, followed by engagement in QI activities.⁴ Curricula that combine didactic and experiential learning appear most effective.^5,6 Most QI curricula are time-limited, often consisting of only a 4-week rotation, which 5 explain the finding that few resident QI projects are implemented.⁴ Another barrier is the shortage of faculty educated in QI.⁷ Moreover, some specialties, including psychiatry, lag behind others in development of performance measures.⁸

A more-recent review showed that most published studies on QI curricula had small sample sizes and low response rates.³ Acquisition of QI knowledge has typically been evaluated via subjective self-assessments as opposed to established assessment tools.^3,9 However, some reports use the Quality Improvement Knowledge Application Tool (QIKAT), which assesses ability to apply core concepts of improvement to cases.^6,10,11

Our review of the literature found only one published psychiatry QI curriculum, from a Canadian residency program.¹² Third-year (postgraduate year-3 [PGY-3]) residents participated in QI didactics and projects; outcomes consisted of focus groups to examine attitudes toward QI training following the curriculum. To address the paucity of psychiatry QI education in the literature, we developed and assessed a didactic and experiential QI curriculum for US psychiatry residents. We describe our curriculum, outcomes, and recommendations for future directions.

Methods

Curricular Development

To develop our curriculum, we used literature review, internal-medicine residency leaders at our university who had already developed an innovative QI curriculum,¹³ and the Institute for Healthcare Improvement website.^14,15

Our educational objectives were focused on the practice-based learning and improvement and systems-based practice competencies (table 1). Our didactics consisted of 10 seminars, 1-hour each (table 2). For resident cohort 1, the QI rotation was 3 months long with 50% protected time (table 3). Each resident was assigned to a psychiatry unit, where he or she developed a QI project, had a QI project supervisor, and was given an internally developed QI workbook. At the end of the rotation, residents presented their projects at departmental conferences.

TABLE 1.

Competency-Based Objectives for Psychiatry Quality Improvement Rotation

TABLE 2.

Seminar Topics for Psychiatry Quality Improvement Didactics

TABLE 3.

Psychiatry Quality Improvement Curriculum for Cohort 1 Versus Cohort 2

After cohort 1 completed the curriculum, we reviewed the assessments and met with project supervisors. This led to a change in the rotation to 9 months at 15% protected time. Consistent with literature reports, residents and supervisors felt that a longer experience would be helpful in implementing projects. Additionally, we changed the format so that QI projects would be completed in teams of 2 to 3 residents. The University of Wisconsin-Madison Health Sciences Institutional Review Board determined the educational study to be exempt from review, and the QI activities did not require formal Institutional Review Board review.

Faculty Development

To address faculty development in QI, we created a department-wide “QI Month” during which we held weekly grand rounds on QI topics. The month concluded with a 1-day Educational Retreat entitled “Quality Improvement: What's in it for Me? consisting of interactive breakout sessions for faculty to learn QI basics.

Evaluation

Assessment of our QI didactic curriculum included predidactic and postdidactic administration of the QIKAT, which was given to PGY-3 residents during their seminar time. The QIKAT contains 2 scenarios, with each scored from 0 (low) to 5 (high), with the highest score being 10. The scenarios were written specifically for psychiatry residents. Responses were blinded and scored by 2 raters using the standard QIKAT scoring rubric.¹⁰ Discrepancies were reconciled between the scorers. Pretest and posttest scores for each resident cohort were compared using paired t tests. We tabulated separate scores for each cohort because of the change in curriculum that occurred. We assessed attitudes toward QI learning through qualitative evaluations of the seminars via free-text responses to standard questions and quantitative evaluations of the experiential rotation via Likert scales. Finally, we qualitatively assessed resident QI projects for content and implementation.

Results

In cohort 1, 7 of 8 residents completed both predidactic and postdidactic QIKAT assessments. The QIKAT scores improved from a mean of 2.4 (SD 0.9) to 7.1 (SD 1.3) (P < .001). In cohort 2, 5 of 8 residents completed both pre-QIKAT and post-QIKAT assessments. One resident in cohort 1, and 3 in cohort 2, did not complete the posttest because they were on leave or vacation. Scores again improved, this time from a mean of 4.6 (SD 2.3) to 8.1 (SD 1.9) (P < .05; figure).

FIGURE.

Residents' Knowledge of QI Principles Pre-QI and Post-QI Didactics, as Measured by Mean QIKAT Scores (10  =  Best)

Abbreviations: QI, quality improvement; QIKAT, Quality Improvement Knowledge Application Tool.

Resident evaluations of the QI seminars were positive. Comments included “inspiring” and “very useful for the QI rotation.” Resident evaluations of the QI experiential rotation have also been positive, showing a mean score of 5 out of 5 (5 best) for “overall satisfaction with rotation,” and 4.3 out of 5 for “impact on learning.”

Four resident QI projects thus far have been completed. The first was the development of a psychiatry intern how-to manual to standardize the process for learning inpatient procedures. The second was designed to increase resident/faculty attendance at department-wide seminars via the development of online streaming of these seminars to remote sites. Attendance improved 25% over 3 months. The third was development of a process to minimize the time psychiatric patients spend in the emergency department, the result of which was that waiting time from consultation to disposition decreased from 3.5 to 2.05 hours over 3 months. Finally, a resident developed a system ensuring that all psychiatry consults placed in our hospital are systematically transmitted to the psychiatry team.

Discussion

In developing our curriculum, we addressed problems that have been described as commonplace in QI curricula. As the second report of a psychiatry residency QI curriculum, we built on the first by incorporating quantitative assessment of our curriculum and by examining the feasibility of a longitudinal QI curriculum within a more modestly sized program. It is difficult to know precisely which ingredients were necessary for the success of our curriculum. However, many reports have highlighted the need for greater faculty development to achieve sufficient numbers of teachers of QI.³ Our QI Month and QI Retreat addressed this for our department. Likewise, we met frequently with interprofessional stakeholders who would be involved in implementing QI projects and chose QI project mentors who were able to facilitate system change.

Factors that contributed to the success of the projects included providing relatively long durations for the rotation, requiring residents to present projects at department-wide conferences, informing residents of other opportunities for scholarly presentations of their work, and providing protected time. Additionally, one resident enrolled in a QI elective and another on the department QI committee was more engaged because of her training in the core QI curriculum.

Our study limitations include small sample size and a single-site intervention. We relied on 7 faculty members knowledgeable in QI, 3 of whom each contributed 18 hours per academic year and 4 of whom contributed less than 2 hours each. This amount of time and expertise might be unavailable in some settings, limiting generalizability. We did not include a control group and thus cannot rule out the possibility that changes occurred because of other trends in training, although this seems unlikely. We did not track the direct costs, but, apart from faculty time, these were minimal.

Future QI curriculum initiatives could involve system-wide QI curricula, cutting across multiple education programs at a given hospital. If programs are teaching systems-based practice via QI curricula, then it makes sense to teach across specialties and professions with nursing, social work, and administration. 5o Clinic has instituted a QI curriculum across specialties, but the effects on learner knowledge, skills, and attitudes are not yet known.¹⁶

Conclusions

We have provided evidence that a didactic and experiential QI curriculum within a US psychiatry residency program can increase QI knowledge and lead to implemented QI projects. Important factors might include faculty development and a longitudinal curriculum with protected resident time. Further studies should examine curricula that use a control group and cross-specialty QI curricula.