Abstract
Background and Objectives:
This study was conducted by the West Virginia Practice-Based Research Network Learning Collaborative to assess research activity, confidence, and attitudes toward residency programs’ research and quality improvement requirements and inform the integration of the state-wide practice-based research network (PBRN) as mentors to support practice transformation implementation initiatives across various resident training sites in West Virginia.
Methods:
This pilot study assessed residents’ attitudes regarding (1) research activity, (2) confidence and (3) requirements of their program in research and quality improvement training by using an anonymous survey, administered during regular residency meetings.
Results:
Of the 68 residents,representing four DO and MD Family Medicine residency programs in West Virginia, 40 (58.8%) responded to the survey. About 64 percent of residents had worked on a quality improvement project, and more than half of residents (52.5%) submitted a research project for a competitive presentation within the most recent year. Sixty-five percent felt satisfied with the residency program’s research and quality improvement curriculum. However, only 55 percent felt confident to perform a project and 52 percent submitted a project to a competitive forum.
Conclusion:
Only half of the residents demonstrate activity and confidence in research and quality improvement. This shows an opportunity to assess current curriculums and provide new strategies to enhance their ability to conduct practice transformation initiatives.
Introduction
The healthcare climate in the US is changing dramatically. Practice transformation initiatives require clinics to undergo systems changes and additional support to improve health outcomes, enhance access and quality, and reduce healthcare costs.1,2 As many providers are adjusting to these changes, it can be difficult for the programs to reflect the necessary educational experiences. In order for these significant systems changes to be adequately integrated into residency training and for residents to provide quality-based care, the residents will need to obtain new skills.3,4
Research and quality improvement capabilities play an important role in practice transformation.5 Successful transformation is often led by data-driven decisions and exploration of population health data.6 Primary care residency programs vary in the levels of research activities required to complete the program. American Osteopathic Association residencies require all residents to take part in scholarship activity. However, this requirement may be completed through any of the following activities: a department scholarly project, a research project through the institution or state, a manuscript, or authoring a grant.7 The American College of Graduate Medical Education (ACGME) guidelines state that scholarly activity must be “shared with peers and subject to peer review”.8 This variability produces difficulty in analyzing the preparedness of the residents as they begin to practice in communities.9 Providers are now asked to promote healthy behaviors, adhere to best practices, and achieve high quality patient-level and population-level outcomes.2 These changes often involve incorporating research and/or quality improvement techniques to attain a higher quality of care. Each residency program has unique ways to address and support research training. As a result, residents may not be exposed to many of the skills essential during their training to the transition out of residency and into the role of a practicing provider.4,10
Practice-based research networks (PBRNs) are established across the country to provide research support to providers in conducting clinical research and quality improvement projects. These networks tend to vary in the amount of experience each provider possesses, making collaborative research endeavors challenging in some circumstances. However, the collaborative groups that assemble in PBRNs have allowed providers to build research capacity, gain experience together, and learn from each other.11–13 Mentoring, promoting research projects, and providing forums to present projects have been shown to increase the research and quality improvement productivity of the program leading to more success in clinical practice.9,14
Collaborative learning groups can be created within PBRNs and have a positive impact on quality improvement performance and knowledge among residents.15 Many studies find that, while residents have a positive attitude about practice transformation, they feel that they were lacking the knowledge and skills of practicing, teaching, and leading these efforts.9,11,16,17 Often residents take a passive role in these initiatives during their training and thus are not optimally prepared.18 Many of the studies conducted regarding this topic confine the approach to a single institution, rather than a larger approach, when addressing population health issues.9,16,17,19
There are advantages to creating a collaborative learning group for residents. Through these learning groups, residents are often shown how to examine their own data and to summarize what the data shows. They are given the opportunity to compare results to collectively discuss the weaknesses and strengths represented. Another advantage is to create a learning environment that gives residents the confidence to make real-life systems changes based on their discoveries.16 This active approach to improving health system procedures contributes to the resident’s career development by elevating their interest and active participation in clinically relevant research and quality improvement work, and this has been shown to continue throughout their careers.9,16
The West Virginia Practice-Based Research Network (WVPBRN) Resident Learning Collaborative team surveyed current residents on their perspective, knowledge, and level of activity toward research and quality improvement pursuits. The overall objective of this pilot study was to assess research activity and confidence, as well as attitudes toward current residency programs’ research and quality improvement requirements in preparation for practice transformation implementation. The goal is to use the information collected to identify opportunities for PBRNs to engage residents in clinical research early and enhance their skills so that they can thrive in this changing healthcare environment. The results will be presented and discussed with residency program faculty and advisors to find ways to incorporate a network approach to research and quality improvement learning across residency programs in West Virginia.
Methods
This pilot study was conducted over a three-month period and was approved by the West Virginia University Institutional Review Board. The study population surveyed included medical residents training at four Family Medicine residency programs that are members of the WVU and Mountain State Osteopathic Postdoctoral Training Institutions (MSOPTI) residency programs in West Virginia. Residency faculty and advisors came together to discuss needs of assessing current residency programs’ research and quality improvement training. A questionnaire developed by Fischer et al.20 to assess research activity, and requirements in research and quality improvement initiatives, was modified for this specific population, and we added concepts of confidence and understanding of usefulness.
The survey was given during a regularly scheduled residency meeting and was returned anonymously. This convenience sampling included 100 percent of the residents who attended the program meetings. However, residents on off-site rotations were not included in this study. Residents were not involved in the development or analysis of this study.
A Likert scale was used to assess satisfaction, benefits, and confidence in research and quality improvement endeavors and curriculum requirements. We converted the Likert scales to dichotomous responses by combining the last two choices as positive response and the other three choices as negative response (example, combining “moderately satisfied” and “very satisfied” as “satisfied” and combining “very dissatisfied”, “somewhat dissatisfied” and “neutral” as “unsatisfied”). A yes-no scale was used to determine whether each resident had participated in a research or quality improvement project within the last year.
Results
Forty residents completed the survey for a response rate of 100 percent of resident meeting attendees, but 58.8 percent of the overall programs’ residents. The respondents included 14 post graduate year (PGY)-one, 11 PGY-two, and 15 PGY-three residents. Most residents (64.1%) worked on a quality improvement project during the year. More than half of residents (52.5%) submitted a research project for a competitive presentation, however only six of those projects were selected for presentation (Table 1). A majority of residents (87.5%) believed their program had a reasonable research requirement. Overall the residents had a positive response to their level of satisfaction with the programs’ current research and quality improvement curriculum (65%), ability to meet research and quality improvement needs (80%), and research and quality improvement requirement (60%). Additionally, 70 percent thought they should be given additional time for research duties in order to meet submission requirements.
Table 1.
Demographics and research participation of resident participants (N=40).
| Residency Program | n (%) |
| Harpers Ferry Family Medicine (WVU) | 11 (27.5) |
| WVU Family Medicine | 11 (27.5) |
| United Hospital Center Family Medicine (MSOPTI/WVU) | 9 (22.5) |
| Greenbrier Valley Medical Center (MSOPTI) | 9 (22.5) |
| Post Graduate Year (PGY) | |
| PGY 1 | 14 (35.0) |
| PGY 2 | 11 (27.5) |
| PGY 3 | 15 (37.5) |
| Question | n (%) positive responses |
| Did you submit a research project to a competitive forum this academic year? | 21 (52.5) |
| Do you think submitting a research project to a competitive forum as a resident is beneficial to you? | 17 (42.5) |
| Do you think quality improvement projects are beneficial to clinical practice? | 30 (75.0) |
| Did you have a project competitively selected for presentation during this academic year? | 6 (15.0) |
| Did you perform or work on a quality improvement project during this academic year?* | 25 (64.1) |
| Do you think the current medicine residency programs’ research submission requirement is reasonable? | 35 (87.5) |
| What is your level of satisfaction with the medicine residency program’s research/quality improvement submission requirement? | 24 (60.0) |
| What is your level of satisfaction with the medicine residency program’s ability to meet your research/quality improvement needs? | 32 (80.0) |
| What is your level of satisfaction with the medicine residency program’s current research/quality improvement curriculum? | 26 (65.0) |
| How confident are you to perform a quality improvement project? | 22 (55.0) |
| Do you think residents should be given extra time to work on research projects outside of their usual resident duties? | 28 (70.0) |
Missing responses to the questions.
Surveyed residents who submitted to a competitive forum were more likely to feel that going through the submission process was beneficial to them than those who did not submit to a competitive forum (Table 2). However, the submission process did not affect how the resident viewed their satisfaction with their program’s residency requirements, ability of the program to meet the resident’s needs, or confidence in their ability to complete quality improvement projects.
Table 2.
Association of residents’ views on benefits, confidence, and satisfaction with submission to and selection by a competitive forum.
| Residents’ viewsa on benefits, confidence, and satisfaction | n (%)a | Submitted to competitive forum | Selected by competitive forum | ||||
|---|---|---|---|---|---|---|---|
| Yes n (%) | No n (%) | p-valueb | Yes n (%) | No n (%) | p-valueb | ||
| Submitting a research project to a competitive forum as a resident is beneficial to you | 17 (42.5) | 12 (57.1) | 5 (26.3) | 0.04 | 1 (16.7) | 16 (47.1) | 0.22 |
| Quality improvement projects are beneficial to clinical practice | 30 (75.0) | 14 (66.7) | 16 (84.2) | 0.28 | 2 (33.3) | 28 (82.4) | 0.03 |
| Satisfied with the medicine residency program’s research/quality improvement submission requirement | 24 (60.0) | 13 (61.9) | 11 (57.9) | 0.80 | 1 (16.7) | 23 (67.7) | 0.03 |
| Satisfied with the medicine residency program’s ability to meet your research/ quality improvement needs | 32 (80.0) | 18 (85.7) | 14 (73.7) | 0.44 | 3 (50.0) | 29 (85.3) | 0.08 |
| Satisfied with the medicine residency program’s current research/quality improvement curriculum | 26 (65.0) | 14 (66.7) | 12 (63.2) | 0.82 | 1 (16.7) | 25 (73.5) | 0.01 |
| Confidence to perform a quality improvement project within the residency program | 22 (55.0) | 12 (57.1) | 10 (52.6) | 0.77 | 3 (50.0) | 19 (55.9) | 0.79 |
Number and percentage of positive answers to each research question on residents’ views of benefits, confidence, and satisfaction.
p-value of chi-square analysis or fisher’s exact test.
Selection of a resident’s project for a competitive forum did not change the resident’s views on the benefits of the submission process and the resident’s confidence level to complete a quality improvement project (Table 2). However, residents who were not selected by a competitive forum still saw the benefit in conducting quality improvement projects and felt satisfied with the residency program’s research curriculum and the requirements for research/quality improvement submission (see Table 2a).
Discussion
With the changing healthcare environment in recent years, it is critical for residency programs to keep pace with the new demands on practicing providers. Integrated with the patient care process are now increased expectation of quality of care improvement and the shift toward value-based care reimbursement. Internal research support systems and teams have been developed in residencies to help meet research requirements.21–23 However, there has been limited extension of residency research, learning, and support outside of their institution. Residents often have different needs, experiences, and time demands that can be better met with a customized support system. Practice-based research networks have developed around the nation to provide additional support for providers in research and quality improvement efforts that can generate clinical impact.
The results of this survey give insight into the current curriculum and opportunities in the various residency programs. While residents acknowledge that quality improvement is beneficial to patient care and health outcomes, only 64 percent of residents in our study have participated in quality improvement activities in their programs. This percentage is lower than expected as programs are making an effort to integrate quality improvement into their programs in preparation for an increased focus of scholarly activities. This affords opportunity to build and tailor support for the residents from the PBRN as that gap is closed. Additionally, only 55 percent of the residents felt only “somewhat confident” or “very confident” in performing research and quality improvement projects. Increased activity and support to residency partners can help bolster self-efficacy.24
The WVPBRN aims to partner with the residency programs to augment research training in residency programs. This network facilitates programs to collaborate with one another and provide the opportunity to present customized training that will enhance the skills needed for quality improvement projects. To begin, network members who work with residency programs have developed a series of research and quality improvement training modules using didactic formats that can be utilized to guide discussions among the residency groups. These sessions will be presented by network members and staff and will be recorded for future viewing. The goal of the training is to connect residents with PBRN resources available and provide information to help guide their research and quality improvement endeavors, thereby increasing experience and confidence to continue using these skills to improve their practices. The next generation of providers can have the opportunity to continue to utilize the support and take a lead role in practice-based research networks.
The survey had a response rate of 58.8 percent which could have introduced bias into the data. However, the primary limitations of this study relate to the survey questions. There is often overlap in research and quality improvement activities. Quality improvement is implementing clinic or process change to create improvement, and research is generating new generalizable knowledge. An overlap exists if quality improvement is disseminated for generalized knowledge and it is also considered research. Residency programs tend to use one term synonymously with the other, though some programs may make more specific distinctions between research and quality improvement. Other questions referring to “competitive forums” may be ambiguous. Residents may not be counting internal competitive forums. The result of the comparison in residents’ views between selected and not selected by a competitive forum might not be reliable since there were only 6 out of 40 projects selected by a competitive forum. Additionally, some responding participants were residents within clinics that already have support from the WVPBRN. Research and quality improvement project opportunities may vary among facilities, leading to some programs needing more support.
Our survey results spurred additional discussion and planning around the need for not only specialized training and support for the resident community, but also peer-to-peer learning sessions whereby residents lead those discussions and research planning processes. We anticipate ample opportunity for the formation and growth of a dedicated residency track within the WVPBRN to help prepare the next generation of healthcare providers.
Acknowledgments
Financial support:
The project described was supported by the National Institute of General Medical Sciences, U54GM104942. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH. West Virginia University’s IRB acknowledgement of this study is on file.
Presentations:
Practice-based research networks: training the next generation of residents in practice-based research” presented to the North American Primary Care Research Network PBRN Conference, Hyatt Regency, Bethesda, MD June 22, 2017.
Contributor Information
Treah Haggerty, West Virginia University School of Medicine, Department of Family Medicine & West Virginia Clinical and Translational Science Institute; .
William Lewis, West Virginia University Eastern Division; Harpers Ferry Family Medicine & West Virginia Clinical and Translational Science Institute; .
Christine Plaugher, West Virginia School of Osteopathic Medicine & West Virginia Clinical and Translational Science Institute; .
Jun Xiang, West Virginia University School of Medicine, Department of Family Medicine; .
Eric Radcliffe, West Virginia University; United Hospital Center; .
Stacey Whanger, West Virginia Clinical and Translational Science Institute; .
Adam Baus, West Virginia University Office of Health Services Research; .
References
- 1.Audet AM, Doty MM, Shamasdin J, Schoenbaum SC. Measure, learn, and improve: physicians’ involvement in quality improvement. Health Aff (Millwood). 2005;24(3):843–853. [DOI] [PubMed] [Google Scholar]
- 2.Graff L, Stevens C, Spaite D, Foody J. Measuring and improving quality in emergency medicine. Acad Emerg Med. 2002;9(11):1091–1107. [DOI] [PubMed] [Google Scholar]
- 3.Walshe K Understanding what works- -and why--in quality improvement: the need for theory-driven evaluation. Int J Qual Health Care. 2007;19(2):57–59. [DOI] [PubMed] [Google Scholar]
- 4.Diaz VA, Carek PJ, Johnson SP. Impact of quality improvement training during residency on current practice. Fam Med. 2012;44(8):569–573. [PubMed] [Google Scholar]
- 5.Mold JW, Peterson KA. Primary care practice-based research networks: working at the interface between research and quality improvement. Ann Fam Med. 2005;3 Suppl 1:S12–20. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Donahue KE, Newton WP, Lefebvre A, Plescia M. Natural history of practice transformation: development and initial testing of an outcomes-based model. Ann Fam Med. 2013;11(3):212–219. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Basic Standards for Residency Training in Osteopathic Family Medicine and Manipulative Treatment. In: Association AO, ed2017:1–22. [Google Scholar]
- 8.Review Committee for Family Medicine. Scholarly Activity Guidelines. In: Education ACfGM, ed2017:1–4. [Google Scholar]
- 9.Dixon JL, Papaconstantinou HT, Erwin JP 3rd, McAllister RK, Berry T, Wehbe- Janek H. House Staff Quality Council: One Institution’s Experience to Integrate Resident Involvement in Patient Care Improvement Initiatives. Ochsner J. 2013;13(3):394–399. [PMC free article] [PubMed] [Google Scholar]
- 10.Quinn DC, Bingham JW, Garriss GW, Dozier EA. Residents Learn to Improve Care Using the ACGME Core Competencies and Institute of Medicine Aims for Improvement: the Health Care Matrix. J Grad Med Educ. 2009;1(1):119–126. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11.Hickner J, Green LA. Practice-based Research Networks (PBRNs) in the United States: Growing and Still Going After All These Years. J Am Board Fam Med. 2015;28(5):541–545. [DOI] [PubMed] [Google Scholar]
- 12.DeVoe JE, Likumahuwa-Ackman S, Shannon J, Steiner Hayward E. Creating 21st-Century Laboratories and Classrooms for Improving Population Health: A Call to Action for Academic Medical Centers. Acad Med. 2017;92(4):475–482. [DOI] [PubMed] [Google Scholar]
- 13.Varkey P, Karlapudi S, Rose S, Nelson R, Warner M. A systems approach for implementing practice-based learning and improvement and systems-based practice in graduate medical education. Acad Med. 2009;84(3):335–339. [DOI] [PubMed] [Google Scholar]
- 14.Seehusen DA, Weaver SP. Resident research in family medicine: where are we now? Fam Med. 2009;41(9):663–668. [PubMed] [Google Scholar]
- 15.O’Mahony S, Mazur E, Charney P, Wang Y, Fine J. Use of multidisciplinary rounds to simultaneously improve quality outcomes, enhance resident education, and shorten length of stay. J Gen Intern Med. 2007;22(8):1073–1079. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Voss JD, May NB, Schorling JB, et al. Changing conversations: teaching safety and quality in residency training. Acad Med. 2008;83(11):1080–1087. [DOI] [PubMed] [Google Scholar]
- 17.Punzalan CS RS, Lerner A, Kouvatsos T, Thakkar S. Quality Improvement of Diabetic Care at a Resident Clinic. The Medicine Forum. 2012;13(1):53–57. [Google Scholar]
- 18.Mladenovic J, Shea JA, Duffy FD, Lynn LA, Holmboe ES, Lipner RS. Variation in internal medicine residency clinic practices: assessing practice environments and quality of care. J Gen Intern Med. 2008;23(7):914–920. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Laiteerapong N, Keh CE, Naylor KB, et al. A resident-led quality improvement initiative to improve obesity screening. Am J Med Qual. 2011;26(4):315–322. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Fischer JL, Cation LJ. Impact of a residency research program on research activity, faculty involvement, and institutional cost. Teach Learn Med. 2005;17(2):159–165. [DOI] [PubMed] [Google Scholar]
- 21.Rothberg MB, Kleppel R, Friderici JL, Hinchey K. Implementing a resident research program to overcome barriers to resident research. Acad Med. 2014;89(8):1133–1139. [DOI] [PubMed] [Google Scholar]
- 22.Byrnes AB, McCormack FX, Diers T, Jazieh AR. The resident scholar program: a research training opportunity for internal medicine house staff. J Cancer Educ. 2007;22(1):47–49. [DOI] [PubMed] [Google Scholar]
- 23.Fancher TL, Wun T, Hotz CS, Henderson MC. Jumpstarting academic careers with a novel intern research rotation: the AIMS rotation. Am J Med. 2009;122(11):1061–1066. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Levine RB, Hebert RS, Wright SM. Resident research and scholarly activity in internal medicine residency training programs. J Gen Intern Med. 2005;20(2):155–159. [DOI] [PMC free article] [PubMed] [Google Scholar]