Abstract
Background
Although many residency programs are instituting quality improvement (QI) curricula in response to both institutional and external mandates, there are few reports of successful integration of resident initiated projects into these QI curricula with documented impact on health care processes and measures.
Intervention
We introduced a multifaceted curriculum into an Obstetrics-Gynecology continuity clinic. Following a needs assessment, we developed a didactic session to introduce residents to QI tools and the how to of a mentored resident-initiated project. Resident projects were presented to peers and faculty and were evaluated. A postgraduation survey assessed residents' satisfaction with the curriculum and preparedness for involvement in QI initiatives after residency. We also assessed whether this resulted in sustained improvement in health care measures.
Results
The curriculum was presented to 7 classes of residents (n = 25) and 17 resident initiated projects have been completed. Twenty-one residents (84%) completed the preintervention survey and 12 of 17 (71%) residents who completed the entire curriculum completed the postintervention survey. Sustained change in surrogate health measures was documented for 4 projects focused on improving clinical measures, and improvement in clinical systems was sustained in 9 of the remaining 13 projects (69%). Most of the respondents (75%, n = 9) agreed or strongly agreed that the projects done in residency provided a helpful foundation to their current QI efforts.
Conclusion
This project successfully demonstrates that a multifaceted program in QI education can be implemented in a busy Obstetrics-Gynecology residency program, resulting in sustained improvement in surrogate health measures and in clinical systems. A longitudinal model for resident projects results in an opportunity for reflection, project revision, and a maintenance plan for continued clinical impact.
Editor's Note: The online version of this article contains the survey instrument (79KB, doc) used in this study and a summary of quality improvement projects (42KB, doc) completed by residents following the intervention.
What was known
Residents are expected to learn about and participate in quality improvement (QI), yet few studies have described QI curricula for residents or assessed their impact of residents' QI involvement after graduation.
What is new
A multifaceted quality-improvement (QI) curriculum in an Obstetrics-Gynecology residency produced gains in residents' QI knowledge, resulted in sustained improvement in health care measures, and increased graduates' preparedness for involvement in QI initiatives after residency.
Limitations
Small sample, single-site study, with lack of a control group, and validated evaluation tools.
Bottom line
Introduction of a multifaceted QI curriculum in an Obstetrics-Gynecology residency resulted in sustained improvement in surrogate health and in clinical systems outcomes.
Introduction
Following the publication of the Institute of Medicine reports, To Err Is Human and Crossing the Quality Chasm, health care quality has become a critical principle and an important marker of performance for consumers and payers.1–3 Credentialing boards link maintenance of board certification directly to practice-based assessment and improvement metrics.4 Furthermore, the Accreditation Council for Graduate Medical Education mandates that trainees demonstrate competency in practice-based learning and improvement.5
Despite the importance of resident involvement in quality initiatives, few publications describe resident initiated projects and even fewer describe improved patient outcomes as a direct result of resident QI education.6–11 We are unaware of any publication describing a similar intervention in obstetrics and gynecology (Ob-Gyn) training programs. The primary goal of this study was to assess whether introduction of a multifaceted QI curriculum in an Ob-Gyn residency could result in the completion of resident selected QI projects that incorporated the required knowledge, attitude, and skills within a busy Ob-Gyn residency continuity clinic. A secondary goal was to determine whether improvement in systems or surrogate health markers could be achieved because of the resident-driven projects.
Intervention
Educational Setting
We implemented our QI curriculum in academic year 2006–2007 at an Accreditation Council for Graduate Medical Education (ACGME)-accredited Ob-Gyn residency program, recently increased from 12 to 16 residents. Resident projects were completed in the Ob-Gyn resident continuity clinic within Maine Medical Center, an academic medical center using an electronic medical record. Three dedicated attending physicians supervised all of the approximately 10 000 Ob-Gyn clinic visits per year. Two of these physicians supported all resident quality initiatives.
Curriculum Design and Delivery
Objectives for resident projects included improving residents' ability to analyze and use population information and practice experience, perform practice-based improvement, and teach peers principles learned.
The curriculum was designed on the principles of adult learning and consisted of an initial didactic session, a self-paced web-based tutorial, and an applied project.12 The intervention used the Plan-Do-Study-Act (PDSA) cycle within individual projects. The Miller13,14 Model for Improvement of “knows, knows how, shows, does” was applied in the educational intervention design.
After a thorough review of QI education literature, a needs-assessment questionnaire was developed by the authors to obtain a baseline understanding of resident knowledge of basic QI concepts and residents' comfort with their own knowledge, attitudes, and perceived skills in performing QI. The survey contained both free text and Likert-type scale responses (appendix 1).
After completing the questionnaire, residents received a didactic lecture entitled “What, Why, and How of Quality Improvement,” written by one of the authors (D.S.), designed to teach concepts covered in the questionnaire. The lecture reviewed reasons for quality in health care and introduced basic QI tools, such as the PDSA, described by Walter Shewhart.15,16 Each resident received a “QI Project Worksheet” (appendix 2), detailing educational objectives, expectations for the project, and a timeline for project completion. Residents also received a copy of the final evaluation form (appendix 3; appendixes 1 through 3 are provided as online supplemental material). Following the lecture, residents were expected to independently complete 2 self-paced tutorials to reinforce the principles introduced in the didactic session: the Institute of Healthcare Improvement's Model for Improvement and Measuring for Improvement.16
After developing preliminary ideas for a quality project, each resident met individually with the faculty mentors to verify that the project was of sufficient scope to demonstrate the principles of QI and to improve quality of care in the continuity clinic. The timeline was reaffirmed with each resident, and required resources were identified. Residents verbally updated their QI project progress during resident evaluations with the program director (K.V.).
Methods for Evaluation
Graduating residents presented their quality projects to their peers and the 2 attending physicians, all of whom evaluated their presentations and projects using the Resident and Attending QI Project Evaluation Form. Residents also completed the evaluation form as a self-assessment. An anonymous follow-up survey was sent to all graduated resident participants using Survey Monkey (Palo Alto, CA), to investigate the QI teaching and experience from a postresidency perspective, with emphasis on self-perception of QI skills. We assessed the sustainability of projects to evaluate long term effects of all completed initiatives. When applicable, specific process or outcome measures related to individual QI projects were repeated to determine whether care improved because of the initiative. The project design was submitted to the Maine Medical Center Internal Review Board and was determined to be an educational innovation that did not require review or exemption status.
Data Collection
The intervention was structured with an action research design in which an innovative curriculum was introduced and data were prospectively collected to inform the development of the project.17 Simple descriptive statistics were performed using Microsoft Excel (Redmond, WA).
Results
The QI didactic session was presented to 7 classes of residents (n = 25), starting in July 2006. Five classes of residents received the didactic at the end of the postgraduate year-3 (PGY-3). The remaining 2 classes of residents (n = 8) received the QI curriculum at the beginning of the PGY-3 (n = 4) and PGY-2 (n = 4), in response to graduated residents' suggestion to provide the curriculum earlier in residency training to allow for follow-up implementation study during the PGY-4.
All residents (n = 21) completed the preintervention survey developed in 2007, except for the first cohort of residents in 2006 (n = 4). Results of the residents' responses on the questions investigating knowledge, attitude, and skills regarding QI are shown in table 1.
TABLE 1.
Results of the Preintervention Survey Using a 5-Point, Likert-Type Scale
Responses to the free-text questions were analyzed using the Constant Comparison/Grounded Theory Method.18 All residents were able to answer “What is QI” and to provide a response indicating that they understood why QI was important. Residents were less able to provide an example of a quality indicator, with 33.3% (7/21) either providing an incorrect answer or no answer. Residents provided incorrect responses to baseline questions investigating knowledge of specific quality-assessment tools, with only one resident providing a correct answer to “What does ‘PDSA’ in the PDSA cycle of quality improvement stand for?” No resident could provide a correct answer to, “What is a QI Infrastructure?” Most residents (76%, 16/21) could provide at least one example of a method to maintain change after implementation of a QI project.
Five classes of residents have completed the entire curriculum, including presentation of their project (17 residents). Two classes of residents (n = 8) who received the curriculum earlier in residency are at varying stages of their projects at the time of this writing. A summary of completed quality projects is presented in the appendix provided as online supplemental material. All of the resident projects that focused on improving clinical outcomes (n = 4) have had a sustained positive impact on clinical care. For example, the QI project that addressed the question, “What percentage of eligible patients is offered the human papillomavirus vaccine at their colposcopy visit?” noted a baseline rate of vaccine discussion in 47% of patients presenting with an abnormal Papanicolaou test. A follow-up review 3 years after implementation of a multidisciplinary education program and change in clinic workflow revealed a 97% rate of offering human papillomavirus vaccine to eligible patients. Sustained system or process changes have resulted in 9 of the remaining 13 (69%) QI projects that focused on patient education or systems-based projects.
The postproject survey was sent to the 17 graduated residents who completed the entire QI curriculum. Twelve graduates (71%) returned the survey. More than half of graduated residents (58%; 7/12) reported active involvement in quality initiatives. One graduate commented, “This was a valuable component to our residency program and offers preparation and skill to tackle QI/QA projects expected of us in our practice, especially as a hospital employed physician. Thank you!” Most respondents (75%, 9/12) agreed or strongly agreed that QI projects done in residency had provided a helpful foundation to their current quality efforts. Although 58% (7/12) of graduates agreed that their project had improved patient care, only one-half of graduates were confident that their project made a lasting difference in the continuity clinic. One-half of graduates felt that their QI project was a meaningful part of their educational experience as a resident. The responses to the question, “How well did the Quality Curriculum you received in residency prepare you to perform the following tasks after residency?” are shown in table 2.
TABLE 2.
Results of the Postintervention Survey Completed by Graduated Residents Using a 3-Point, Likert-Type Scale
Discussion
This study is one of few studies published with residents directing QI projects as part of a QI curriculum.7–11 The strength of this project is that it represents a longitudinal practicum model based in adult learning theory, practicable for the graduate medical education office, and easily replicable by other programs and institutions.
The initial didactic session was designed to meet adult learners' perceived educational needs by emphasizing the critical nature of QI initiatives for efficient systems and quality patient care in future practice, and by addressing questions missed on the preproject needs-assessment questionnaire. Individual identification of quality gaps, with self-designed and implemented QI projects, enhanced motivation by linking learning with self-identified critical outcomes. Projects provided for transfer of learning with concrete application of concepts. As residents presented their projects to attending physicians and colleagues, they received acknowledgement for their expertise, with opportunity for self-evaluation and reflection on their experience.
Published quality initiatives involving residents with a practicum component have used various approaches, including interdisciplinary teams, teams composed solely of residents, and resident-driven initiatives with staff involvement.6–11,19,20 Unlike other studies, which assigned quality topics to residents, residents in this project identified quality gaps in their own practice and selected initiatives for improvement, a critical skill within the PDSA cycle.21 This educational intervention demonstrates that resident-designed projects can affect process measures and surrogate measures of health care, such as improved rates of appropriate immunization counseling. It also demonstrates that a significant percentage of graduates from the program remain involved in quality initiatives. A controlled trial would be required to determine whether this represented true behavior change because of the curriculum or was a reflection of the current, quality-driven practice environment.
Resources of time and financial investment for resident projects in published literature vary widely.19 No separate financial or time resource was allocated; faculty assumed oversight for the projects as a portion of job assignment expectations for quality improvement. Estimates of time required to develop, mentor, and complete quality projects are presented in table 3. Despite minimal investment of dedicated resources, all residents completed projects and 75% (9/12) of graduated residents felt that teaching and practice in QI had been “excellent” or “sufficient.” The minimal investment of resources needed in our intervention makes it likely that the project can be successfully adapted by training programs smaller than those in a previously published study.22
TABLE 3.
Estimate of Total Faculty and Resident Time Dedicated to Quality Improvement (QI) Intervention in the First 5 Years of the QI Initiative With 4 Residents/Year
Most published projects offer didactics and project learning, but few extend them to a 1-year time frame.6,8,19 In recognition of the need for an expanded longitudinal framework to complete a PDSA cycle within each individual project, and in response to resident input, we expanded the period to 3 years, initiating the project at the beginning of PGY-2. This allows time to fully assess and adjust each intervention as necessary and to design a sustainability plan for the project.20,23 We expect that this expanded period will enhance the effects of changes long term with improved resident confidence in the benefit of their projects. To increase visibility of these initiatives, presentation of QI projects will now be scheduled for departmental grand rounds, and residents will be encouraged to present their projects at hospital-wide QI fairs.
Limitations of this initiative are the small sample sizes and the lack of controls and validation of the evaluation methods.6,19
Conclusion
This project demonstrates that a multifaceted program in QI education requiring minimal resources can be implemented in a busy Ob-Gyn residency. This program resulted in sustained changes in both systems and processes and in surrogate health measures. Further studies are required to obtain objective measures of knowledge improvement in QI and to correlate increased knowledge and confidence with improved clinical outcomes. Studies with larger sample sizes, objective assessments of knowledge and outcomes, and the ability to show sustained improvement would also be beneficial. Further studies are also needed to correlate increased resident knowledge and confidence in QI to improved clinical outcomes.
Footnotes
All authors are at Maine Medical Center. Debra Sepulveda, MD, MPH, is Director of Ambulatory Obstetrics and Gynecology and the Medical Student Clerkship Director of Obstetrics and Gynecology; and Kalli Varaklis, MD, MMed, is Residency Program Director of Obstetrics and Gynecology.
Funding: The authors report no external funding source for this study.
References
- 1.Committee on Quality of Health Care in America, Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academies Press; 2001. [Google Scholar]
- 2.Committee on Quality of Health Care in America, Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, DC: National Academies Press; 2000. [PubMed] [Google Scholar]
- 3.Groman R. Linking Physician Payments to Quality Care. Philadelphia, PA: American College of Physicians; 2005. [Google Scholar]
- 4.American Congress of Obstetricians and Gynecologists. ACOG website. http://www.acog.org. Accessed June 12, 2011. [Google Scholar]
- 5.Accreditation Council on Graduate Medical Education. Outcome project website. http://www.acgme.org/acwebsite/RRC_280/280_corecomp.asp. Accessed April 9, 2012. [DOI] [PubMed] [Google Scholar]
- 6.Patow C, Karpovich K, Riesenberg L, Jaeger J, Rosenfeld J, Wittenbreer M, et al. Residents' engagement in quality improvement: a systematic review of the literature. Acad Med. 2009;84(12):1757–1764. doi: 10.1097/ACM.0b013e3181bf53ab. [DOI] [PubMed] [Google Scholar]
- 7.Vinci LM, Oyler J, Johnson JK, Arora VM. Effect of a quality improvement curriculum on resident knowledge and skills in improvement. Qual Saf Health Care. 2010;19(4):351–354. doi: 10.1136/qshc.2009.033829. [DOI] [PubMed] [Google Scholar]
- 8.Laiteerapong N, Keh CE, Naylor KB, Yang VL, Vinci LM, Oyler JL, et al. A resident-led quality improvement initiative to improve obesity screening. Am J Med Qual. 2011;26(4):315–322. doi: 10.1177/1062860610395930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Canal DF, Torbeck L, Djuricich AM. Practice-based learning and improvement: a curriculum in continuous quality improvement for surgery residents. Arch Surg. 2007;142(5):479–482. doi: 10.1001/archsurg.142.5.479. [DOI] [PubMed] [Google Scholar]
- 10.Djuricich AM, Ciccarelli M, Swigonski NL. A continuous quality improvement curriculum for residents: addressing core competency, improving systems. Acad Med. 2004;79(10)(suppl):S65–S67. doi: 10.1097/00001888-200410001-00020. [DOI] [PubMed] [Google Scholar]
- 11.Weingart S, Tess A, Driver J, Aronson M, Sands K. Creating a quality improvement elective for medical house officers. J Gen Intern Med. 2004;19(8):861–867. doi: 10.1111/j.1525-1497.2004.30127.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Knowles MS. The Modern Practice of Adult Education: Andragogy Versus Pedagogy. Englewood Cliffs, CA: Prentice Hall/Cambridge; 1970. [Google Scholar]
- 13.Tague NR. The Quality Toolbox. 2nd ed. Milwaukee, WI: Quality Press; 2004. [Google Scholar]
- 14.Miller GE. The assessment of clinical skills/competence/performance. Acad Med. 1990;65(9)(suppl):S63–S67. doi: 10.1097/00001888-199009000-00045. [DOI] [PubMed] [Google Scholar]
- 15.Shewhart WA. Statistical Method From the Viewpoint of Quality Control. Vol. 1939. New York, NY: Dover; p. 45. [Google Scholar]
- 16.Institute of Healthcare Improvement. Model for Improvement and Measuring for Improvement. http://courses.ihi.org/coursecatalogue. Accessed June 12, 2011. [Google Scholar]
- 17.O'Brien R. Theory and Practice of Action Research [in Portuguese] João Pessoa, Brazil: Universidade Federal da Paraiba; 2001. An Overview of the Methodological Approach of Action Research [in Portuguese] English version available at: http://www.web.ca/∼robrien/papers/arfinal.html. Accessed April 9, 2012. [Google Scholar]
- 18.Strauss AC, Corbin JM. Basics of Qualitative Research: Grounded Theory Procedures and Techniques. Newbury Park, CA: Sage Publications; 1990. [Google Scholar]
- 19.Boonyasai RT, Windish MD, Chakraborti C, Feldman LS, Rubin HR, Bass EB. Effectiveness of teaching quality improvement to clinicians: a systematic review. JAMA. 2007;298(9):1023–1037. doi: 10.1001/jama.298.9.1023. [DOI] [PubMed] [Google Scholar]
- 20.Ogrinc G, Headrick LA, Mutha S, Coleman M, O'Donnell J, Miles P. A framework for teaching medical students and residents about practice-based learning and improvement: synthesized from a literature review. Acad Med. 2003;78(7):748–756. doi: 10.1097/00001888-200307000-00019. [DOI] [PubMed] [Google Scholar]
- 21.Patterson BR, Kimball KJ, Walsh-Covarrubias JB, Kilgore LC. Effecting the sixth core competency: a project-based curriculum. Am J Obstet Gynecol. 2008;199(5):e561–e566. doi: 10.1016/j.ajog.2008.05.027. [DOI] [PubMed] [Google Scholar]
- 22.Sockalingham S, Stergiopoulos V, Maggi J, Zaretsky A. Quality education: a pilot quality improvement curriculum for psychiatry residents. Med Teach. 2010;32(5):e221–e226. doi: 10.3109/01421591003690346. [DOI] [PubMed] [Google Scholar]
- 23.Speck M. Best practice in professional development for sustained educational change. ERS Spectrum. 1996;14(2):33–41. [Google Scholar]