Continuing Medical Education and Maintenance of Certification: Essential Links

Background

Few question the need for physicians to engage in continuous learning throughout their professional career. Knowledge and technological advances are expanding at a breathtaking pace, as is the evolution of new skills and attitudes required for the physician to just “keep up.” For example, major breakthroughs in our understanding of genetics are already beginning to change how physicians make therapeutic decisions with individual patients.¹ The introduction of quality improvement science since the 1980s is changing how physicians manage groups of patients.^2,3 More importantly, quality improvement requires physicians to systematically examine their practice performance.

To assist physicians with lifelong learning, institutions and organizations have created continuing medical education (CME) programs. Traditionally, such programs involve didactic learning experiences such as grand rounds and lectures by experts at either the local institution or at a regional or national conference. Whereas physician satisfaction with this traditional form of CME is often high, research has repeatedly demonstrated these more “passive” learning activities are ineffective in helping physicians to change their practice.⁴ Traditional CME seldom involved reflection by the physician on their real learning needs.

Against the backdrop of this changing landscape is the recognition that many physicians struggle to keep current and engage in meaningful lifelong learning. First, many physicians are no longer active in their local hospital's educational programs and committee activities.⁵ Second, Choudhry and colleagues reported in their systematic review that, on average, physicians' knowledge and skill declines over time.⁶ Third, many physicians are not terribly accurate in self-assessment of their knowledge and skills.⁷ The inability to accurately self-assess creates substantial challenges for physicians to determine what their true learning needs are.⁷

This is the “perfect storm”: the predominant form of CME, the didactic-based experience, is ineffective in changing behavior; on average, physicians' knowledge and skills decline over time; and physicians' ability to perform self-assessment accurately is suspect. Where does this leave the profession? What does effective CME actually look like?

Effective CME

At a minimum, planners of CME activities need to understand and embrace adult learning principles.⁸ Two basic principles are: 1) The activity must have high relevance to what the physician actually does. Although there is nothing inherently wrong with learning about an interesting subject for the simple joy of learning, physicians should recognize that such activities often do not translate into meaningful changes in their clinical practice. 2) The CME activity needs to be interactive, not passive. This means the participating physician must have the opportunity to work and manipulate the subject material. For example, a recent systematic review by the Cochrane Collaboration concluded that workshops using interactive formats can lead to moderately large changes in physician practice.⁹ However, workshops require a significant amount of planning and logistical support, are hard to perform more than a few times during the course of a year, and often reach only limited numbers of participants.

… many physicians are not terribly accurate in self-assessment of their knowledge and skills.⁷

As a result, there is substantial interest in developing meaningful self-directed learning and assessment activities physicians can do on their own time and that incorporate quality improvement and change. This suggests that health care organizations should look for methods to facilitate ongoing CME activities that can be embedded into the work of busy practicing physicians. Certification boards are one group of organizations that have embraced the importance of flexible, real time interactive self-assessment for more effective CME.

The certification boards recognized that their maintenance of certification (MOC) program could be an effective approach for helping physicians “keep up” and improve their practices. The American Board of Internal Medicine's (ABIM) MOC program has evolved rapidly over the last six years. A substantial portion of the MOC program involves interactive self-assessment that promotes professional development and provides CME credit hours. We'll start with a description of the ABIM MOC program, followed by results of early research about the impact of the program. We'll end by describing how current CME activities in the Kaiser Permanente (KP) system can interface with MOC.

The ABIM MOC Program

The MOC program consists of four components and is specifically designed to provide assessment for all six competencies of the American Board of Medical Specialties (ABMS)—Accreditation Council for Graduate Medical Education (ACGME) (Table 1). First, physicians must possess an unrestricted license. Second, physicians must pass a secure exam of knowledge once every ten years. The other two components are called self-assessment for lifelong learning, focused on medical knowledge, and evaluation of performance in practice.

Table 1.

The general competencies

Lifelong Learning in Practice

The lifelong learning component can be accomplished through several pathways. The ABIM provides a number of Web-based modules in specific content areas that help physicians self-assess medical knowledge. These modules consist of 25 to 60 multiple-choice questions based on a clinical vignette that are designed to help physicians learn new knowledge and skills. These are “open-book” modules; physicians are encouraged to use educational resources in completing the module. All ABIM modules have CME credits available upon completion. Over the course of the next two years, the ABIM is developing annual update modules consisting of 25 questions for general medicine and all subspecialties. ABIM diplomates can also use a number of approved, society-developed knowledge modules to meet this requirement, such as the American College of Physicians Medical Knowledge Self Assessment Program (MKKSAP) program and a number of subspecialty society medical-knowledge products.

A new project involves the competencies of medical knowledge and practice-based learning and improvement; seeking answers to clinical questions that arise in the context of patient care. Previous studies have demonstrated that approximately two thirds of the clinical questions a physician encounters, and for which they do not know the answer but that could potentially impact the patient's care, go unanswered.^10,11 The ABIM is developing a new type of Web-based module that facilitates the systematic collection of clinical questions, using the framework of evidence-based practice to help the physician structure the question and subsequently search efficiently for an answer. This module builds on the work of others using a Web-based diary to capture clinical questions for self-directed learning.^12–14

Evaluation of Performance in Practice: Practice Improvement Modules

The practice improvement modules (PIMs) are Web-based tools designed with important adult learning principles in mind. PIMs are highly interactive, involving the physician in active review and reflection about their practice. Completion of a PIM involves collecting data from medical record audit, patient surveys and a questionnaire of the practice's microsystem, reflecting on the performance measures in a practice quality report, developing and implementing an improvement plan, and finally reporting the impact of a rapid cycle test of change. Figure 1 shows the PIM framework. This framework is based on the cycle of change methodology for quality improvement.² The available PIMs and their make-up are shown in Table 2.

Figure 1.

Practice Improvement Module (PIM) framework.

Table 2.

American Board of Internal Medicine available practice improvement modules (PIM)

Because the PIMs are all about the physician's own practice, PIMs are highly relevant to what the physician actually does in their daily work. However, interactivity and relevance do not necessarily ensure the experience has value to the physician or facilitates change in their practice. Recognizing the need to investigate the impact of the PIMs, the ABIM has embarked on an operational research effort to study the effectiveness of the PIMs in physician practices.

Early Research with the PIMs

One of the first studies involved a small sample of sixteen practicing physicians in Connecticut who volunteered to use the Diabetes PIM to assess and implement a quality improvement intervention in their practice.¹⁵ The physicians identified areas for improvement in multiple processes of care through the medical record audit and patient surveys. A significant proportion of the physicians found the patient surveys provided valuable feedback that patients wanted more information about their medications and diabetes. The physicians also valued the comprehensive, high-quality performance data from the audit.

A recent analysis of the first 179 completers of the Preventive Cardiology PIM (PC-PIM) confirmed several findings found in the original diabetes pilot study. First, the PC-PIM performed well as a self-administered tool for assessment of the quality of practice using performance measures and obtaining feedback from patients. Second, the act of self-assessment and performing practice quality improvement was novel for most physicians; most had not previously had a personal experience in quality improvement. Third, the patient survey provided new insights for the physicians on how their practice communicated with patients.

However, both of these early studies highlighted several challenges in helping physicians change practice through a Web-based tool. First, most of the physicians did not involve other members of the office. Too often physicians used the “work-harder” approach. Second, physicians often struggled on how to redesign office work processes necessary for effective and sustainable quality improvement. Third, the majority of physicians had little prior experience with implementing change before using the PIM.

The results of these two early PIM studies demonstrate the clear need to continue to redesign CME activities to help physicians acquire the knowledge and skills needed to collect evidence of performance in practice, to learn from it, and to use it to improve practice. The ABIM is continually learning from these early experiences with the Web-based PIMs to improve the educational and self-assessment process. Future work will focus on the ABIM's Self-Directed PIM that allow physicians to use external data, similar to that provided by KP to its physicians.

The challenge ahead for all physicians in the United States is to understand the denominator of the patients they are serving … this is population-based medicine …

MOC and CME at Kaiser Permanente

Permanente physicians have a unique advantage in their approach to CME and in its links to MOC. As members of an integrated multispecialty group, Permanente physicians have access to CME programs that are organized by the Permanente Medical Groups. In these, they can bring up-to-date methods to physicians based not only on specialty society topics, but on clinical conditions that might affect patients cared for by multiple specialists within the group. Since Kaiser Permanente (KP) offers many CME activities internally, physicians may not have to travel as extensively and lose time away from home and work in order to keep up to date. Now that MOC programs for all 24 ABMS specialties cite reference to ABMS, this activity also gains one credit towards maintaining board certification in one's specialty. Specialty certification requires active and unrestricted licensure, demonstrated involvement in self-evaluation of medical knowledge and performance in practice, and passing a secure examination. The self-evaluation of both knowledge and practice performance are enhanced for Permanente physicians. First, they are enhanced because KP is an organization in which data about performance, attention to population-based care, and allowance for physicians to participate in continuing education have always been core to the organization's identity. Components, such as the Care Management Institute, which examines evidenced-based approaches to current guidelines for care in a wide range of conditions, contribute to the rich and high-quality resources that KP physicians can draw on. In addition, with KP HealthConnect, Permanente physicians now have the ability to get feedback about the quality of clinical care that they provide in a wide range of conditions. This kind of data is directly applicable to MOC for internists through the ABIM Practice Improvement Modules (PIMs). Data for common conditions such as Diabetes, Asthma, and HIV can be directly entered into the ABIM Web-based template. Other data related to other conditions can be entered through the “Self-Directed PIM” and credit obtained for MOC. Both Northern California and Southern California are participating in evaluation of an established quality program initiative by the American Board of Internal Medicine in which a program that engages in measurement and active engagement of individual physicians in improvement plans awards credit on the basis of attestation from leaders that the physician has been engaged in such activities.

The challenge ahead for all physicians in the United States is to understand the denominator of the patients they are serving. To Permanente physicians this is population-based medicine, a very familiar concept. Physicians must then use these denominator statistics to calculate measures, the rates at which appropriate tests and examinations and treatments were ordered; and ultimately, patient-specific outcomes are achieved. The Permanente physician has a tremendous advantage over community-based physicians and especially physicians in solo or small practices in rural areas where, with multiple payers and no electronic medical records, determination of denominators and calculations of these measures is dependent only on information from claims-based data from health plans. For this reason, it is multidisciplinary group practices such as KP that will set the standard for the rest of the nation in quality of physician practice. An additional advantage for Permanente physicians is the reduction of the burden of redundant measurement as they are able to get credit for these activities towards MOC.