Education in Quality Improvement for Practice in Primary Care During Residency Training and Subsequent Activities in Practice

Abstract

Background

Quality improvement (QI) is an integral aspect of graduate medical education and an important competence for physicians.

Objective

We examined the QI activities of recent family medicine residency graduates and whether a standardized curriculum in QI during residency resulted in greater self-reported participation in QI activities in practice after graduation.

Methods

The family medicine residency programs affiliated with the South Carolina Area Health Education Consortium (N  =  7) were invited to participate in this study. Following completion of introductory educational activities, each site implemented regularly occurring (at least monthly) educational and patient care activities using QI principles and tools. Semiannually, representatives from each participating site met to review project aims and to provide updates regarding the QI activities in their program. To examine the impact of this project on QI activities, we surveyed graduates from participating programs from the year prior to and 2 years after the implementation of the curriculum.

Results

Graduates in the preimplementation and postimplementation cohorts reported participating in periodic patient care data review, patient care registries, QI projects, and disease-specific activities (57%–71% and 54%–63%, respectively). There were no significant differences in QI activities between the 2 groups except in activities associated with status of their practice as a patient-centered medical home.

Conclusions

Most but not all family medicine graduates reported they were actively involved in QI activities within their practices, independent of their exposure to a QI curriculum during training.

What was known

Quality improvement (QI) is an important competency for physicians.

What is new

Data on whether recent family medicine graduates who participated in a QI intervention during training are more likely to conduct QI in practice.

Limitations

Single-specialty focus limits generalizability; survey instrument is not validated.

Bottom line

Recent family medicine residency graduates' likelihood of conducting QI activities in practice is not associated with their exposure to QI activities during training.

Introduction

Because of concerns regarding patient safety and quality of medical care within the US health care system, quality improvement (QI) training has become an integral aspect of graduate medical education, with the assumption that residents will continue to participate in such activities once they finish formal training and enter practice. The Accreditation Council for Graduate Medical Education (ACGME) Common Program Requirements state that “residents must demonstrate the ability to investigate and evaluate their care of patients, to appraise and assimilate scientific evidence, and to continuously improve patient care based on constant self-evaluation and life-long learning.”¹ Residents also are expected to develop the skills necessary to “systematically analyze practice using QI methods and implement changes with the goal of practice improvement.”¹

Limited literature describes specific curricular activities or experiences that incorporate QI principles and methodologies into graduate medical education.^2–19 Whereas several of these studies have shown an improvement in patient care, as measured by specific quality indicators and medication reconciliation, there is a lack of data on the impact of the educational interventions on the practice of graduates after completing training. Studies of the effect of QI interventions had limited information regarding patient care. Notably, these studies did not specifically examine QI activities and experiences after graduation.

The purpose of our study was to examine the QI activities and experiences of recent family medicine residency graduates and whether a guided curriculum in QI during residency resulted in greater participation in QI activities in their practices after graduation.

Methods

Settings and Participants

Representatives of the family medicine residency programs affiliated with the South Carolina Area Health Education Consortium (SCAHEC; N  =  7) were invited to participate in this study. For their participation, each site received funding for a lead faculty member and a part-time quality coordinator. The skills and qualifications of the quality coordinator were to be determined by each program.

Guided QI Curriculum

Whereas approximately half of the programs indicated that they had an existing QI curriculum, all programs agreed to participate in biannual group meetings and to incorporate the following educational activities into their required curriculum: (1) a guided didactic session that served as an introduction to the principles and tools of QI, and included QI discussion in terms of managing human performance using a systemic, data-driven, and integrated approach, and the importance of measurement, benchmarking, and guidelines. This session needed to be conducted at each residency program in August or September; and (2) completion of both the “Quality Improvement and Beyond: Achieving Excellence in Healthcare” course (a website-based interactive activity introducing the principles and tools of QI [http://musc.remote-learner.net/myqi]) and the Collaborative Institutional Training Initiative (Miami, FL) course in the Protection of Human Research Subjects. Each resident of the participating program would be required to complete the above-cited activities.

Following completion of the introductory educational activities, each site was required to implement regularly (ie, at least monthly) occurring educational and patient care activities involving QI. Activities included but were not limited to regular group meetings of residents, faculty, and staff to assess educational and patient care needs of the program regarding quality of care being provided, development and implementation of patient care registries, and utilization of a systematic approach for QI activities addressing the quality of care for specific diseases. As part of the project, the patient care registries were required to include individuals with diabetes mellitus and/or heart disease and stroke. Beyond these requirements, each site was allowed to determine the most appropriate specific interventions and activities for their residents and patients.

Semiannually, representatives from each participating site met to review the specific aims of the project and to provide updates regarding the activities being conducted at their program. These meetings would also provide an opportunity for programs to share activities considered to be best practices in both teaching and conducting practice-based QI and to discuss strategies for overcoming barriers (ie, software, institutional policies, personnel) to implementing the activities associated with this project.

Survey Design and Distribution

We designed a 20-item questionnaire that was reviewed by the authors and several other family medicine physicians for readability and understanding. The survey gathered information on graduate characteristics, characteristics of the graduates' current practice and patient population, the practices' QI activities and programs, and the graduates' assessment regarding the usefulness of QI training they received during their residency.

We surveyed graduates from participating programs from the year prior to and 2 years after the implementation of the curriculum. The questionnaire was mailed to each graduate with a cover letter explaining the purpose of the study and inviting participation. Completed anonymous questionnaires were returned in an accompanying self-addressed stamped envelope. A second mailing was sent to nonresponders 6 weeks after the initial mailing with the same cover letter.

The study was reviewed and considered exempt by the Institutional Review Board of the Medical University of South Carolina.

Analysis

Descriptive statistics were used to characterize and summarize the data obtained. Analysis was completed using a χ² test for noncontinuous variables and a t test for continuous variables. Significance was defined as a P value < .05. For adequate power to reflect a 50% difference between the 2 groups (10% margin of error, P < .05), a response rate of 34% for each group was required. All statistical analyses were conducted with Statistical Analysis Software (SAS Institute Inc, Cary, NC).

Results

Five of 7 affiliated family medicine residency programs participated in the study. Two sites were unable to participate due to local institutional and administrative barriers. Surveys were mailed to the 132 residents who graduated from participating programs during the survey period. Sixteen surveys were undeliverable, and our survey response rate was 55% (64 of 116). Twenty-one surveys (33%) were from the preintervention (2009) cohort, and 43 (67%) were from the cohort trained with enhanced exposure to QI (2010 or 2011). The average age of respondents was 33.6 (± 5.5) years and a majority (78%) identified themselves as non-Hispanic White. No significant differences in sex, age, race, practice type, or setting were present between the 2 groups.

Most respondents in both cohorts participated in periodic patient care data review, patient care registries, QI projects, and disease-specific activities. No significant differences in participation in QI activities in practice were found between the 2 pre-QI and post-QI implementation cohorts, except in activities associated with National Committee for Quality Assurance recognition as a patient-centered medical home (PCMH; table). Significantly fewer graduates who participated in the enhanced QI curriculum had activity in this area in their practice. Most graduates reported being able to integrate QI knowledge into their current practice, and more than half had implemented PCMH standards, although there were no significant differences between the 2 groups of graduates.

TABLE.

Quality Improvement Activities of Graduates by Year of Graduation

Discussion

We hypothesized that, following the incorporation of a specific and standardized QI curriculum into family medicine residency training, graduates would have an increased level of QI activities in their practices. Most graduates of the SCAHEC family medicine residency programs reported being actively involved in QI activities in their practices a few years after graduation. However, those graduates who experienced the enhanced, guided QI curriculum were no more likely than those who had a less intensive or no specific QI curriculum to participate in periodic patient data review, patient care registries, specific QI projects, and disease-specific activities. A high percentage of graduates (87% [52 of 60]) reported they were able to integrate the knowledge and skills regarding QI learned during residency into their current practice activities.

Our findings raise questions regarding the impact of residency curriculum on the practice patterns of family medicine physicians following residency training and why some graduates do not participate in commonly occurring QI activities in their current practice. Additionally, as residents are required to participate in activities associated with improving the care of their patients as part of the practice-based learning and improvement core competency, the ability to operationalize this education into practice appears to be difficult for many of these graduates.

Reasons for the lack of QI activities in practice reported by respondents require further study. In a previous study, the practice patterns of family medicine physicians were found to be similar despite the type of residency program from which they graduated.²⁰ The actual practice and surrounding community that residents enter following completion of formal training may have more significant impact on their practice than the curriculum or other educational variables associated with their residency program. For instance, the trend in residency training is to provide more education regarding the PCMH; however, we found that PCMH activities were actually less likely in the practices of more recent graduates. Finally, QI activities are common licensing and board requirements and should be an additional incentive and requirement for these graduates to participate in such activities. Presumably, these incentives may not be affecting the QI activities of these physicians due to their recent completion of training, initial licensing, and board certification. The impact of these requirements on specific QI activities requires further study.

While the practice environment may be influencing recent graduates, other factors likely are present. The cohort of graduates who participated in a formalized QI curriculum in residency was out of training for a shorter period of time and, therefore, newer in their practice setting than the control group. These graduates would have had the initial focus of their practice activities spent in learning new systems and building a patient panel and may not have yet engaged in QI activities that would naturally come later, such as preparation for maintenance of certification.

This study has several limitations. The overall response rate, the response rate of the graduates by program, and unknown program infrastructure changes (ie, faculty, curriculum, or resources) may have affected the results. Several of the programs had participated in a QI collaborative prior to this project, and residents who completed training before the start of this program had been previously exposed to practice-based QI in their program. In addition, the study included graduates from a relatively small number of programs and only programs located in a single state, limiting generalizability. This study is also limited by self-report bias. Reliability and validity of the survey instrument is not known. Some findings may relate to the shorter period in practice for the more recent graduates with exposure to the enhanced QI curriculum. Finally, a significant impact of the curriculum may be noted if a follow-up occurred over a longer period of time.

Conclusion

Many family medicine graduates are actively involved in QI activities within their practices; however, it appears that this involvement is not associated with receiving formalized training in QI during residency. Furthermore, some recent graduates are not participating in QI activities despite a curriculum present during residency training and licensing and board requirements.