Abstract
Background
Despite a mandate to teach quality improvement (QI) to residents, many training programs lack faculty capacity to deliver a QI curriculum.
Objective
We piloted a co-learning curriculum in QI to train residents while simultaneously developing QI teachers. We evaluated the curriculum's acceptability and feasibility and its effect on faculty engagement in doing and teaching QI.
Methods
The curriculum involved 2 half-day, interactive sessions, a team-based QI project, and end-of-year project presentations. Key curriculum design principles included (1) residents and faculty co-attend all interactive sessions, (2) residents and faculty work together on team-based QI projects, and (3) QI projects align with divisional QI priorities. Using the Kirkpatrick framework for learner outcomes, we focused our program evaluation on Level 1 (satisfaction) and Level 2 (knowledge and skills acquisition) outcomes using year-end curriculum evaluations.
Results
Our study included 14 residents (70%) and 6 faculty members (30%). With respect to satisfaction (Kirkpatrick Level 1 outcome), 93% (13 of 14) of residents and 100% (6 of 6) of faculty participants rated the overall curriculum as “above average” or “outstanding.” Regarding faculty knowledge and skills acquisition (Kirkpatrick Level 2 outcomes), faculty self-rated their QI knowledge and interest in QI higher than their intent to incorporate QI into future practice and their comfort in teaching or supervising QI projects. All 5 faculty respondents (100%) rated the co-learning model for faculty development in QI as “above average” or “outstanding.”
Conclusions
Teaching QI to residents and faculty as co-learners is feasible and acceptable and offers a promising model for programs to teach QI to residents while concurrently building faculty capacity.
Editor's Note: The online version of this article contains resident and faculty learning objectives, and the curriculum evaluation form for residents and faculty (47.5KB, doc) .
Introduction
The Accreditation Council for Graduate Medical Education (ACGME) program requirements mandate that residency programs formally teach quality improvement (QI) concepts and skills as part of the practice-based learning and improvement core competency.1 Yet reports from the field suggest that 1 of the major barriers to teaching QI is the lack of faculty with sufficient expertise in QI to teach those concepts to others.2–6
Advanced training opportunities in QI exist for practicing physicians,7,8 but those courses are not designed to develop faculty to teach QI to learners. Faculty development approaches in this area, such as train-the-trainer programs, are underdeveloped4,9 and have not been consistently shown to translate to establishing effective training interventions.10 Furthermore, relying on those programs to develop a group of faculty QI teachers and educators would likely take considerable time, delaying the launch of formal QI training for residents. That is unacceptable given the urgent need to impart future physicians with the necessary QI knowledge and skills to address growing quality concerns that plague our system.
Myers and Jaeger4 recently advocated for a co-learning model to train faculty and residents together; however no concrete descriptions exist, to our knowledge, to guide the use of that approach for faculty development in doing and teaching QI. The approach has the potential of accelerating the development of a cadre of QI teachers while simultaneously training the next generation of physicians. We recently piloted a co-learning curriculum in QI targeting program directors, faculty leads, and senior residents in 3 subspecialty training programs. We describe our experience designing and implementing this curriculum and present the findings of our initial evaluation, which aimed at assessing acceptability and feasibility of a co-learning model for teaching QI to residents and faculty.
Methods
Educational Context
The University of Toronto Department of Medicine consists of 19 subspecialty divisions, each of which delivers a postgraduate training program across 5 major training sites to a total of 110 residents and subspecialty fellows in each year of training. We piloted our curriculum in the 2011–2012 academic year in 3 of these subspecialty divisions (endocrinology, medical oncology, and nephrology).
Co-Learning Curriculum in QI
We conducted a needs assessment to inform the design and delivery of our co-learning curriculum in QI by reviewing the literature for teaching QI to residents5,6,11 and interviewing QI curricular leads in the United States and Canada to establish the most successful and feasible approaches that have led to successful implementation of a QI curriculum.12
Our key curricular design principles were as follows: (1) residents and faculty are co-learners and will attend the formal teaching sessions together to participate in experiential learning activities to develop knowledge and skills in QI; (2) residents from a given subspecialty division work in teams with a faculty lead in their division to plan and carry out their QI project, which allows for individual team members to take a more active role during lighter clinical rotations, while handing the project off to other team members while on busier rotations; and (3) wherever possible, QI projects should align with departmental quality priorities and ongoing initiatives.
Resident and faculty learning objectives are provided as supplemental material. For residents, the learning objectives focused primarily on developing knowledge and skills in QI and working in teams to apply new learning to a QI project. For faculty, the learning objectives were similar to those for residents (ie, development and application of QI knowledge and skills) but also included the broadening of teaching and project facilitation skills in QI. For faculty who were skilled teachers but lacked experience in QI, the objectives relating to building QI knowledge and skills would be more relevant, whereas for faculty with more QI experience, the focus would be on the “teaching QI” learning objectives.
The curriculum consisted of an interactive, half-day workshop in September to introduce core QI knowledge and skills that were then immediately applied to team-based QI project over the ensuing 3 months. In January, there was a second workshop that introduced additional QI knowledge and skills that were again immediately used to carry out the next phase of the team-based QI project. At the end of the year, there was a formal day during which all the groups met and presented their projects. table 1 provides a more detailed description of the QI content covered in each of the workshops.
TABLE 1.
Co-Learning Curriculum in Quality Improvement (QI) Outline
Resident and faculty QI teams met monthly, in between workshops, to discuss project progress and worked during and outside their clinical rotations to carry out the actual QI project work. Key resources included a 0.2 full-time equivalent dedicated program assistant, web-based cloud storage to house curriculum materials, space for the face-to-face meetings, and modest costs for food at the final project presentation day.
Program Evaluation
Using the Kirkpatrick framework for learner outcomes,13 we focused our program evaluation on Level 1 (satisfaction and acceptability) and Level 2 (knowledge and skills acquisition) outcomes using year-end curriculum evaluations. To assess acceptability and feasibility of the curriculum, as well as the self-assessed changes in knowledge, skills, and attitudes, all participating residents and faculty were asked to complete end-of-workshop evaluations as well as an evaluation of the curriculum as a whole at the end of year. The evaluations included items rated on a 5-point Likert scale, plus open-ended questions to allow for additional comments from respondents (provided as online supplemental material).
We summarized program evaluation data descriptively, using mean and standard deviation for continuous data and counts and percentages for categoric data, and reviewed participant open-ended comments to identify key recurring themes. The University of Toronto Research Ethics Board approved this study.
Results
Fourteen residents (5 from endocrinology [36%], 5 from medical oncology [36%], and 4 from nephrology [29%]) and 6 faculty members participated in the curriculum. Two faculty members were assistant professors (33%) and 4 were associate professors (66%). Three were clinician teachers/educators (50%) and 3 were clinician-investigators (50%). Three faculty members (50%) attended all 3 sessions, whereas the other 3 faculty (50%) attended 2 of the 3 sessions.
Twelve of 14 eligible residents (86%) and 5 of 6 eligible faculty members (83%) completed end-of-curriculum evaluations (table 2). Residents and faculty rated the overall co-learning curriculum highly. In general, self-rated improvements in QI knowledge and interest were higher than self-rated skills to apply QI methods into future practice for both residents and faculty. Faculty also rated their preparedness to teach and supervise QI projects lower than their self-ratings of improvements in QI knowledge and interest levels.
TABLE 2.
Resident Evaluations of the Co-Learning Curriculum
Discussion
We successfully piloted a co-learning curriculum in QI for faculty and resident learners. The curriculum was highly rated, suggesting that this model was acceptable to both residents and faculty for learning about QI. We are encouraged by the findings of our initial pilot, particularly with respect to the feasibility and acceptability of a co-learning approach to teaching QI. In less than 2 years, this model has significantly expanded the number of residents and faculty engaging in QI through our curriculum (we have since expanded to 9 subspecialty training programs, involving 20 faculty members and 67 residents for the 2012–2013 year). This ability to scale up quickly is particularly appealing because it allows programs with limited faculty capacity in QI to deliver QI training to a relatively large number of trainees.
Self-rated improvements in knowledge and interest in QI were higher than intent to engage in or teach QI to others among faculty participants. We had hoped that by involving faculty as co-learners we would have been more successful in developing faculty QI teaching skills. However, our findings suggest that a greater emphasis on preparing the faculty for their role in teaching the curriculum is needed. Despite faculty support for this model, our experience clearly highlights the challenge that besets any efforts aimed at building faculty capacity in QI and suggests that “one-off” or self-limited faculty development programs in QI without ongoing support are likely to be unsuccessful.
Our study has several limitations. This was a pilot curriculum and early program evaluation based on a small number of faculty and residents at a single center. Ongoing program evaluation with a recruitment of additional faculty participants from the next year of our curriculum is needed to confirm our findings. We developed the participant surveys de novo and did not validate them. Thus, participants may not have interpreted the questions as we intended and may have interpreted the questions differently from each other. Finally, our preliminary program evaluation relied on self-reported assessment of knowledge and skills and did not evaluate the effect of resident and faculty participation in our curriculum on clinical processes or patient outcomes.
Conclusion
We have been able to use a co-learning approach to rapidly scale up our educational efforts in QI to reach a broad audience of senior trainees, faculty leads, and program directors. Given the early findings, co-learning is an acceptable and feasible model for programs to teach QI to residents while concurrently building faculty capacity.
Footnotes
Brian M. Wong, MD, FRCPC, is Assistant Professor of Medicine and Associate Director of the Centre for Quality Improvement and Patient Safety at the University of Toronto, Ontario, Canada, and Staff Physician of General Internal Medicine at Sunnybrook Health Sciences Centre, Toronto; Jeannette Goguen, MD, FRCPC, MEd, is Associate Professor of Medicine and Director of Postgraduate Programs in Medicine at the University of Toronto and Staff Physician of Endocrinology at St Michael's Hospital, Toronto; and Kaveh G. Shojania, MD, FRCPC, is Associate Professor of Medicine and Director of the Centre for Quality Improvement and Patient Safety at the University of Toronto and Staff Physician of General Internal Medicine at Sunnybrook Health Sciences Centre.
The authors would like to thank Elisa Hollenberg, MSW, for her assistance with data gathering (including survey administration and participant interviews), entry and analysis, and revision of an earlier draft of this manuscript.
Funding: The authors report no external funding source for this study.
References
- 1.Accreditation Council for Graduate Medical Education. ACGME common program requirements. http://www.acgme.org/acgmeweb/Portals/0/dh_dutyhoursCommonPR07012007.pdf. Revised July 1, 2011. Accessed January 17, 2013. [Google Scholar]
- 2.Lucian Leape Institute. Unmet Needs: Teaching Physicians to Provide Safe Patient Care. Boston, MA: National Patient Safety Foundation; 2010. [Google Scholar]
- 3.Headrick LA, Baron RB, Pingleton SK, Skeff KM, Sklar DP, Varkey P, et al. Teaching for Quality: Integrating Quality Improvement and Patient Safety Across the Continuum of Medical Education—Report of an Expert Panel. Washington, DC: Association of American Medical Colleges; 2013. [Google Scholar]
- 4.Myers JS, Jaeger J. Faculty development in quality improvement: crossing the educational chasm. Am J Med Qual. 2012;27(2):96–97. doi: 10.1177/1062860611433640. [DOI] [PubMed] [Google Scholar]
- 5.Wong B, Etchells E, Kuper A, Levinson W, Shojania K. Teaching quality improvement and patient safety to trainees: a systematic review. Acad Med. 2010;85(9):1425–1439. doi: 10.1097/ACM.0b013e3181e2d0c6. [DOI] [PubMed] [Google Scholar]
- 6.Wong BM, Levinson W, Shojania KG. Quality improvement in medical education: current state and future directions. Med Educ. 2012;46(1):107–119. doi: 10.1111/j.1365-2923.2011.04154.x. [DOI] [PubMed] [Google Scholar]
- 7.Splaine ME, Ogrinc G, Gilman SC, Aron DC, Estrada CA, Rosenthal GE, et al. The Department of Veterans Affairs National Quality Scholars Fellowship Program: experience from 10 years of training quality scholars. Acad Med. 2009;84(12):1741–1748. doi: 10.1097/ACM.0b013e3181bfdcef. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Stille CJ, Savageau JA, McBride J, Alper EJ. Quality improvement “201”: context-relevant quality improvement leadership training for the busy clinician-educator. Am J Med Qual. 2012;27(2):98–105. doi: 10.1177/1062860611414404. [DOI] [PubMed] [Google Scholar]
- 9.Emanuel L, Walton M, Hatlie M, Lau D, Shaw T, Shalowitz J, et al. The patient safety education project: an international collaboration. In: Henriksen K, Battles JB, Keyes MA, Grady ML, editors. Advances in Patient Safety: New Directions and Alternative Approaches (Vol 2: Culture and Redesign) Rockville, MD: Agency for Healthcare Research and Quality (US); 2008. pp. 1–15. [PubMed] [Google Scholar]
- 10.Pearce J, Mann MK, Jones C, van Buschbach S, Olff M, Bisson JI. The most effective way of delivering a train-the-trainers program: a systematic review. J Contin Educ Health Prof. 2012;32(3):215–226. doi: 10.1002/chp.21148. [DOI] [PubMed] [Google Scholar]
- 11.Ogrinc G, Headrick L, Mutha S, Coleman M, O'Donnell J, Miles P. A framework for teaching medical students and residents about practice-based learning and improvement, synthesized from a literature review. Acad Med. 2003;78(7):748–756. doi: 10.1097/00001888-200307000-00019. [DOI] [PubMed] [Google Scholar]
- 12.Wong BM, Kuper A, Hollenberg E, Etchells EE, Levinson W, Shojania KG. Sustaining Quality Improvement and Patient Safety Training in Graduate Medical Education: Lessons From Social Theory. Acad Med. 2013;88(8):1149–1156. doi: 10.1097/ACM.0b013e31829a0fec. [DOI] [PubMed] [Google Scholar]
- 13.Kirkpatrick D. Evaluation of training. In: Craig R, Mittel I, editors. Training and Development Handbook. New York, NY: McGraw-Hill; 1967. pp. 87–112. [Google Scholar]