Engagement of Groups in Family Medicine Board Maintenance of Certification

Abstract

Purpose

The American Board of Medical Specialties’ performance in practice, ‘Part IV’, portion of Maintenance of Certification (MOC) requirement provides an opportunity for practicing physicians to demonstrate quality improvement (QI) competence. However, specialty boards’ certification of one physician at a time does not tap into the potential of collective effort. This paper shares learning from a project to help family physicians work in groups to meet their Part IV MOC requirement.

Methods

A year-long implementation and evaluation project was conducted. Initially, 348 members of a regional family physician organization were invited to participate. A second path for enrollment was established through three healthcare systems and a county-wide learning collaborative. The participants were offered: a) a basic and straightforward introduction to quality improvement (QI) methods, b) the option of an alternative Part IV MOC module using a patient experience survey to guide QI efforts, c) practice-level improvement coaching, d) support for intra- and inter-practice collaboration and co-learning, and e) provision of QI resources.

Results

More physicians participated through group (66) than individual (12) recruitment, for a total of 78 physicians in 20 practices. Participation occurred at three levels: individual, intra-practice, and inter-practice. Within the one-year time frame, intra-practice collaboration occurred most frequently. Inter-practice and system-level collaboration has begun and continues to evolve. Physicians felt that they benefited from access to a practice coach, group process, and a subset of participants also reported that their QI competency had improved with participation.

Conclusions

Practice-level collaboration, access to a practice coach, flexibility in choosing and focusing improvement projects, tailored support, and involvement with professional affiliations, can enhance the Part IV MOC process. Specialty boards are likely to discover productive opportunities from working with practices, professional organizations, and healthcare systems to support intra- and inter-practice collaborative QI work that uses Part IV MOC requirements to motivate practice improvement.

Introduction

Despite the fact that demonstration of competence in Practice-Based Learning and Improvement (PBLI) has been embraced across the medical education and medical practice continuum for more than a decade,¹ development of quality competence and improvement in quality itself continues to lag.² The requirement for recertification, now embraced by all 24 American Board of Medical Specialties (ABMS) member boards,³ presents an opportunity to engage physicians in improving the quality of medical care.⁴ A recent study showed high rates of family physician participation in the Maintenance of Certification (MOC) process.⁵

The ‘Part IV’ portion of MOC “performance in practice” requirement engages physicians in assessing their quality of care, participating in a quality improvement project, and reassessing their individual contribution toward the effort. Part IV MOC provides a strategically important opportunity for quality improvement (QI) and for the development and assessment of PBLI competence. However, whereas most QI involves engaging teams within a practice and support across systems and multiple practices,^6–8 medical specialty boards certify single physicians. Although there is an emerging use of a small number of institutionally-based QI programs that meet Part IV MOC requirements, for the vast majority of physicians, the Part IV MOC opportunity is limited by its one physician at a time approach.

Therefore, with support from the American Board of Family Medicine Foundation, we undertook a learning initiative to support physicians in working in groups to meet their Part IV MOC requirements. Our purpose was to provide on the ground experience with the process of engaging multiple physicians and their practice, and when possible their health care system and community partners, in working together to improve practice while meeting Part IV MOC requirements, and to identify opportunities to expand group approaches to QI through Part IV MOC.

Methods

Design

This initiative was deemed exempt by the Case Western Reserve University Institutional Review Board (IRB) as an educational and quality improvement project. A team consisting of members of a practice-based research and development support unit and an educational and quality improvement consultant worked with a local family physician professional organization, local health care organizations, and individual physicians and practices to conduct and learn from shared Part IV MOC practice improvement projects.

Participant sampling procedure

Figure 1 illustrates the two-phased participant engagement process. The first enrollment path focused on individual physicians by mailing, and then emailing, an introductory letter and interest/needs survey to the 348 members of a regional family physician professional organization. Several weeks later, a targeted email was sent by the American Board of Family Medicine (ABFM) to their Diplomates within the regional catchment area who were required to complete a Part IV by the end of years 2011 and 2012 in order to stay on the Maintenance of Certification-Family Physician (MC-FP) pathway.

Figure 1.

Participant engagement

*One practice divided into two working groups; a PES working group and a Diabetes PPM working group

The second enrollment path involved recruiting participants at the group level. Though the majority of the physicians among these groups were contacted in the initial mailing, efforts to enroll physicians through their participation in professional groups focused on 1) a county-wide data sharing collaborative, 2) a large integrated health system, 3) a large independent health system, and 4) a group of rural hospital-affiliated family physicians. Most of the family physicians affiliated with these groups also would have received the individual mailings, but the exact number is not known. Contact with these institutional participants was achieved through attending and presenting the project at already scheduled organizational meetings and learning summits. Simultaneously, several meetings were held with administrative leaders to begin to link the MC-FP Part IV project with ongoing quality improvement initiatives in the systems.

Opportunities for Participants

A facilitation process was established by the project team and followed for each participating physician and practice. An initial consultation was held between physician(s) who expressed interest in the MOC QI Collaborative and a practice coach, an experienced clinical research facilitator cross-trained in an 11-month Primary Care Practice Coach Training Program developed and facilitated by the Institute for Healthcare Improvement (IHI). At this meeting the physicians’ and practice’s priorities for improvement were discussed. In addition, an alternative Patient Experience Survey (PES) module was introduced, and interest in using this alternative module versus a standard module was assessed. Quality improvement needs and various ways of beginning to meet those needs also were discussed. The QI education framework was adapted from the Health Improvement Skills Center (HISC) and the Plan, Do, Study, Act (PDSA) cycle adapted from the IHI “How to Improve” webpage.⁹ This framework allowed for a basic and fundamental approach to learning quality improvement methods.

For those using the PES alternate module the practice coaches facilitated survey distribution, before and after the improvement project, and provided peer-comparison feedback reports. In subsequent improvement project consultations, practice coaches supported the physician(s) and practice colleagues in examining the results of the baseline patient experience survey or the data collected using a standard ABFM module. They discussed possible QI projects, selected one to pursue, and planned a timeline for completion.

Typically, during this same meeting, coaches supported detailed project planning with an eight question “Quality Improvement Project Plan” (Appendix A). In working with the physicians/practices, the practice coach sought to identify and assess the QI skills and knowledge that physician(s) and their colleagues already possessed. This helped the practice improvement team identify QI assistance needs, and helped the participants to identify learning opportunities. Types of assistance needs included help with defining the problem or area for improvement, designing the intervention, determining outcome measures for their chosen intervention, and interpreting data generated by the intervention. This support, and practice and meeting structure and processes were documented in project field notes collected by all team members.

Practice members also were encouraged to access online resources and tools¹⁰ during the implementation phase of the QI effort. The practice coach was available by telephone, email and face-to-face meetings should additional support be required.

Practices “check-ins” usually were conducted 3 weeks after QI project implementation, when participants had had an opportunity to test their intervention. After collection of follow-up patient experience surveys, or chart audits for a standard module, participants were asked to complete a “QI Project Assessment” (Appendix B). This document allowed physician(s) to critically assess the QI project both from the perspective of the team’s and their individual work. The final consultation meeting between the practice coach and QI project team was used to discuss the results of the follow-up data, the success of the QI intervention, lessons learned, and next steps in sustaining or altering their efforts.

Data collection

Multiple data collection mechanisms were built into the project: an interest/needs survey during recruitment, a participant-tracking document, detailed field notes from the consultations, aggregate and individual scores from the patient experience survey, and a QI project planning and report forms. The project team met routinely to share meeting notes and to discuss common themes and observations throughout implementation of the collaborative. The subset of physicians participating in the alternative PES module was asked to complete an evaluation form that assessed the utility of the alternative module.

Analysis

For each physician and practice participating in the PES module, anonymous summaries were created of patient responses. To identify themes of what worked to engage participants in working together to meet Part IV MOC requirements, the authors periodically reviewed and reflected on field notes and identified common themes. These themes were discussed and refined throughout the project. Level of collaboration was documented throughout the project and results were tabulated. Those physicians participating in the PES alternative module were surveyed and responses were tabulated and summary statistics created.

We also used the participant-tracking document data to create a matrix of interactions among participants, and then used UCINET (v 6.354) software¹¹ to map inter-practice collaborations within the MOC IV Collaborative for the purpose of illustrating the various levels of collaboration.

Results

The individual recruitment path yielded 48 physician responses; of these 12 participated. The group recruitment path yielded 66 additional participating physicians, totaling 78 participants in 20 practices. Forty-one physicians opted for the alternative Patient Experience Survey (PES) module, while the remaining 37 physicians completed a standard Part IV MOC module. In one practice, half the participants used the alternative patient experience and half used a standard module. All participants completed the project (0% attrition).

As shown in Table 1, QI projects motivated by the PES and standard modules included topics surrounding health maintenance and preventive screening, chronic disease self-management and decision support, improving access, workflow and efficiency, care coordination, chronic disease management, and improvement of the patient/practice experience. Physicians and/or practices made contact with a practice coach an average of 10.3 times either by telephone, email, or in face-to-face meetings.

Table 1.

Quality Improvement Projects and Alternate Modules Used

Number of sites	Quality Improvement Project	Part IV Module
3	Improve preventive health maintenance screening	Both
1	Update patient education (PE) handouts	Patient Experience
1	Building knowledge of patient’s personal information	Patient Experience
2	Increase self-management decision support	Patient Experience
2	Improve access	Patient Experience
3	Improve care coordination	Patient Experience
1	Educate patients on newly established ePrescribing	Patient Experience
1	Update lab follow-up procedure	Patient Experience
1	Improve care for HTN patients and at-risk patients	Standard Module
2	Assess Diabetes care through coordination, support staff, and tracking outcomes	Standard Module
2	Evaluate and improve In Basket management	Standard Module
1	Increase the rates of retinal eye exams for patients with diabetes	Standard Module
2	Redesign telephone systems for prescription refills and lab results	Patient Experience

Within the one-year project time frame, intra-practice collaboration (group work) occurred in all participating multi-physician practices. Intra-practice collaboration was considered to occur when more than one physician in a particular practice worked together on a Part IV QI project. Practice teams ranged from an individual to 8-members, and frequently included nurses, medical assistants, office managers, and clerical staff.

Inter-practice collaboration was said to occur when two or more practices met to share their experience with Part IV quality improvement interventions. A smaller amount of inter-practice and system-level collaboration took place during the project year, and this continues to evolve. For example, several of the health care systems and a county-wide data sharing and learning collaborative^{12, 13} continue to plan ways to integrate Part IV MOC with their institutional QI efforts. Also, a rural hospital primary care institute is using the PES alternate module as a framework for collecting discharge information on patients, and working with physicians to improve the patient post-discharge experience and reduce re-hospitalization. Additionally, eight practices continue to meet once per month to support and share their QI learning.

Figure 2 depicts collaborations within the MOC IV Collaborative. This figure shows that the practice coach was centrally involved, and shows that the organizations from which we recruited and several physicians within large, integrated health care systems were also central to the process.

Figure 2.

Map of Inter-Practice Collaborations Around Part IV MOC

Upon completion of the PES alternate module, this subset of physicians were asked to complete a self-assessment survey reflecting on their experience with this practice-coach assisted, group approach to QI (Table 2). This survey yielded an 85% response rate. Of the 18 respondents, 78% rated access to a practice coach highly valuable; 47% responded that the role of the practice coach was what they liked most. 100% of respondents felt that their QI competency had improved with participation and 89% believe that they have enough confidence to complete another QI project.

Table 2.

Patient Experience Module Participant Evaluation Responses

n=18 (85% response rate)

Quantitative	% Positive response
Overall opinion of Part IV activity	88.9
Did this activity encourage you to work with other physicians?	88.9
How valuable did you find working with other physicians to be?	72.2
How much value did access to a Practice Coach add to your Part IV experience?	88.9
How confident are you in doing another quality improvement project?	88.9
Do you feel your competence in quality improvement has improved?	100.0
Open-ended Questions:	Most Frequent Response (%)
The content was:	Just Right (100.0)
What did you like most?	Facilitator/Practice Coach (47.4)
What did you like least?	Extra Work (16.7)

Table 3 summarizes the observations important to facilitating collaborative group participation in Part IV MOC. In particular, the opportunity to work collaboratively with others on QI interventions was widely appreciated; and the practice coach was highly valued.

Table 3.

Observations from Facilitating the MOC QI Collaborative

Practices found that the opportunity to work on meeting the Part IV QI requirement in groups was quite helpful and enhanced their improvement (QI) efforts. Quality Improvement (QI) coaching (facilitation) appears to be an essential and highly appreciated, previously missing element in the MOC Part IV process. Individualizing response QI intervention to each practice’s priorities is also essential Immediate engagement with practice priorities is important Employing a Patient Experience Survey (PES) can be particularly helpful in fostering individualization, creativity and motivation. Using a QI template or planning form helps individuals and practices to begin work on projects, rapidly, and to keep on task and on track. Learning about QI is generated and sustained most naturally and effectively as team members work on specific, personally meaningful improvement projects. A warm welcome to systematic improvement enhances facilitation. That is, an approach that: Helps team members to identify and utilize existing QI skills and knowledge Encourages and helps team members to identify the assistance needed with the effort: ▪ Regarding the kind of improvement being attempted ▪ Regarding the technical parts of QI, such as determining outcome measures and interpreting data on a small scale Helps team members to identify next steps Provides gentle, consistent encouragement as teams encounter problems or obstacles Conveys a sense of play and creativity when approaching these problems or obstacles Helps teams mobilize and/or connect with any needed assistance, such as a particular kind of practice coaching Provides the option of online resources as well as human sources of support and learning Cited concerns of time and staff availability need to be taken quite seriously QI interventions can sometimes uncover interpersonal or practice dynamics that might require additional resources

Discussion

This project is one of the first to describe the utility of supporting work in groups to complete medical specialty board maintenance of certification (MOC) practice improvement (Part IV) projects. Working together to meet Part IV MOC requirements, and having the opportunity to engage in a practice-individualized QI approach based on both patient experience data and on group practice, input appeared to make the Part IV MOC experience particularly meaningful to participants.

Practice coaches were able to foster intra-practice collaboration by using peer-comparison feedback reports of Patient Experience Survey (PES) results from the alternative module and/or similar data from standard modules, and by supporting the assessment, QI, and collaborative processes. When participants closely examined data, particularly the PES results, this reflective process facilitated critical thinking and led to improvement projects that were tailored to local patient-identified needs. The process of reflecting on data established a space for open communication and shared learning where coaches were routinely able to help physicians identify what they already know about PBLI and help team members to use, share, enhance and add to this knowledge. All points of communication, especially face-to-face meetings with the physicians and practice members, were critical in providing a supportive environment.

In early practice meetings, physicians seemed eager to quickly complete a Part IV project in order to “check it off of their to-do list.” This sentiment became less prevalent as physicians learned that they were able to take part in projects that were meaningful to them and that they were able to apply skills they already had to their QI work. Employing the PES was particularly helpful in fostering individualization of QI projects that were personally meaningful. Tying change efforts to personal meaning has been shown to be important to motivating engagement of busy primary care physicians in QI, ¹⁴ and a locally-tailored approach is an essential characteristic of successful practice change facilitation^15,14,16

Physicians in practices that engaged other practice staff members as well as fellow physicians were more likely to develop a team approach to the intervention and to begin to establish a culture of practice-based learning. This may have contributed to many physicians’ self-assessment that they felt confident about doing future QI projects. In many practices, the initial emphasis on fulfilling the Part IV requirement became secondary to achieving practice priorities and seeing an improvement cycle through to completion.

In the early project planning phase of their work, teams narrowed choices for possible topics for improvement through discussion and feedback from members as they shared ways in which they individually addressed particular issues or concerns. Involving physicians at a group level in this way established a platform for sharing best practices, which has been shown to be an effective QI approach.^{13, 17} One group developed and refined shared goals for improvement, and crafted shared plans for capacity building and task assignment. Practices that established multidisciplinary teams early on exhibited the most success in their efforts and were more likely to communicate with the practice coach on a regular basis, consistent with the emerging literature on the benefits of diverse teams.^18–23

While we encouraged the use of supplemental QI tools and learning activities by way of web-based resources, uptake on these suggestions was low. Future efforts might be better advised to require, rather than just recommend, completion of a minimum number of QI learning activities on a phased, just in time basis. This would provide a more robust, shared mental map of QI principles and methods to team members, enhancing the efficiency, effectiveness and generalizability of their efforts, and better advance the QI capability development agenda of the MOC Performance in Practice (Part IV) requirement.

Initially we had anticipated supporting inter-practice collaboration while simultaneously supporting group work on QI projects within individual practices. What emerged was that practice team members needed to develop their ability to work with each other first. Efforts to support a greater degree of inter-practice exchange and collaboration are ongoing. Collaboration across groups of practices often involves negotiating institutional-level agreements, data use and sharing agreements, and use of information technology and institutional QI resources. Supported by our Practice-Based Research & Development Shared Resource²⁴ (a core facility of our NIH-supported Clinical & Translational Science Collaborative and Comprehensive Cancer Center) this inter-practice Part IV MOC QI work continues to develop, and appears to have great potential to institutionalize collaborative QI work. Our ongoing efforts in this realm particularly focus on engaging family medicine practices that serve underserved patient populations and that have been shown to be under-represented in the MOC process.⁵

A number of physicians have expressed trepidation with MOC, referring to it as both a “burdensome” and “questionable” process.²⁵ Citing time as a barrier to completing MOC requirements, some physicians have described the MOC process as unfair and unneccessary.^26–28 Actively encouraging and supporting QI work in groups and providing a practice coach seems to ease this trepidation by helping physicians to see and experience the MOC Part IV requirement as a realistic and approachable objective that can meet larger practice improvement aims.

Strengths and Limitations

Coaching was a key part of the intervention. The availability of a practice coach appeared to be directly related to the success of getting groups of physicians to work together to meet Part IV requirements. Two emerging infrastructures that could provide the facilitation¹⁶ necessary to advance use of Part IV MOC requirements to stimulate group QI are: involvement of practice-based research and best practices networks,¹⁷ and adoption of a Primary Care Health Cooperative Extension Service, as advocated by Grunmbach and Mold,²⁹ included but unfunded in the Affordable Care Act, and currently being tested in multiple states.³⁰

The facilitation process happened organically, which contributed to the increased communication and relationship-building, however this sometimes revealed human dynamics that required a deeper level of coaching. This level of complexity should be considered when coaches are introduced to the practice and they should be equipped with skills necessary to address these complexities.

We were limited by the one-year time frame to fully develop inter-practice level collaboration, but development of inter-practice collaboration continues. Eight practices continue to meet once per month are concurrently developing QI projects with their large, integrated regional health system, and a rural hospital primary care institute continues to advance use of hospital-collected patient experience data to support post-discharge care and reduce re-admissions among the independent practices that admit to the hospital. The data sharing and learning collaborative continues to evolve Part IV MOC support among its learning collaborative across family medicine and soon internal medicine practices. While other health systems have shown interest in working across practices, it will take ongoing work to make this happen. Inter-practice level collaboration benefits from sponsorship at the administrative level, as well as the intra-practice level in order to facilitate widespread support of practice-based learning.^{14, 15} It is also important to note that participants in this project were voluntary and not necessarily representative of the entire population of family physicians.

Implications/Next steps

Timely, competent coaching of the work of improvement teams, particularly when practices are widely dispersed, is labor intensive. Therefore, for this approach to be replicated widely, and, ideally, incorporated into medical specialty board MOC systems, it will be necessary to develop and test a variety of efficient and cost effective strategies for preparing QI coaches and for scaling up the availability and provision of coaching for those engaged in MOC related improvement projects. A web-based interface serving as a professional network site for participants might encourage the development of regional partnerships and a culture of shared learning. Attention to independent, solo, rural and inner city practices⁵ will be particularly important to avoid creating a QI-divide between practices with access to QI support resources and more isolated practices.

Conclusions

This project demonstrates the potential and the process for group approaches to Part IV MOC to engage physicians and their practice, system, and organizational partners in working together to improve practice. Further work to refine and support group approaches to Part IV MOC appears warranted.

Appendix A: Quality Improvement Project Plan

Quality Improvement Project Plan
Clinician:	Coach:
Project title:	Date:

INSTRUCTIONS

Please fill out the following Improvement Project Plan. Then revise it as you go, incorporating the ideas that emerge from the work of the improvement project team you assemble for this effort.

1. In your current work situation, do you have any measures of the quality of your (and your team’s) performance that could serve as a basis for selecting an improvement target? If yes, please describe.

2. What is your target for improvement? Please be as specific as possible even if you do not have measures of current performance.

3. List several options for changes you might make that could help you achieve your target for improvement.

4. Of the options listed in your response to #3, which specific change in work or activity will you test in an attempt to achieve your target for improvement?

5. Whom might you need to involve in attempting the change? (Also, who are the other members of your improvement team?)

6. What might impede or block your intervention?

7. What additional resources might you need in attempting the change?

8. What measurement will you perform that will enable you to determine, if change occurs, whether it is an improvement, given your target? Please be specific as to how you will measure your progress.

9. Describe how this plan uses quality improvement principles and methods.

Copyright © 2011, Improvement Learning, all rights reserved. Reproduction for non-commercial purposes encouraged.

Appendix B: Quality Improvement Assessment Form

Quality Improvement Project Assessment
Clinician:	Coach:
Project title:	Date:

INSTRUCTIONS

Please fill out the following Improvement Project Report.

1. Did you reach your target? Describe the evidence you used to make this judgment

2. Describe in more detail what you and your team did to achieve these results. Include specific information about your personal role.

3. What modifications, if any, did you make in your original change idea as you put it into practice?

4. Describe any issues or challenges you experienced in measuring the impact of the change idea on your (or your team’s) performance?

5. What did you think you did particularly well in making this change?

6. What might you do differently the next time?

7. If your change resulted in an improvement in performance, what are you planning to do to sustain this improvement?

8. If the change did not result in an improvement, what will you try next?

Copyright © 2011, Improvement Learning, all rights reserved.