The effectiveness of continuing medical education for specialist recertification

Abstract

Evolving professional, social and political pressures highlight the importance of lifelong learning for clinicians. Continuing medical education (CME) facilitates lifelong learning and is a fundamental factor in the maintenance of certification. The type of CME differs between surgical and non-surgical specialties. CME methods of teaching include lectures, workshops, conferences and simulation training. Interventions involving several modalities, instructional techniques and multiple exposures are more effective. The beneficial effects of CME can be maintained in the long term and can improve clinical outcome. However, quantitative evidence on validity, reliability, efficacy and cost-effectiveness of various methods is lacking. This is especially evident in urology. The effectiveness of CME interventions on maintenance of certification is also unknown. Currently, many specialists fulfil mandatory CME credit requirements opportunistically, therefore erroneously equating number of hours accumulated with competence. New CME interventions must emphasize actual performance and should correlate with clinical outcomes. Improved CME practice must in turn lead to continuing critical reflection, practice modification and implementation with a focus towards excellent patient care.

Introduction

Maintaining lifelong knowledge and skills is essential for safe clinical practice.¹ Continuing medical education (CME) is an established method that can facilitate lifelong learning. It focuses on maintaining or developing knowledge, skills and relationships to ensure competent practice.² A decade ago, the To Err Is Human report and the Bristol Inquiry turned the spotlight on patient safety.^3,4 These incidents recorded in these reports, in addition to the gradually increasing demand by the healthcare professionals, contributed to the development of a professional regulation system for clinicians.⁵ CME is an essential component of this regulatory system; it is called “Maintenance of certification” in Canada and in the United States or “Specialist Revalidation” in the United Kingdom.^6–8

The goal of the current CME system for specialists is to improve knowledge through conferences, formal courses, workshops and symposiums. Specialist practice has, however, recently undergone several changes. These changes include increasing innovation in therapeutic and diagnostic modalities, increasing inter-speciality collaboration and transforming traditional, non-interventional specialities into interventional disciplines. Many of the traditional CME methods designed to address these changes have not yet been validated. Therefore, current CME program may not be effective at narrowing the gap between optimal evidence-based practice and actual clinical practice.⁹

CME is considered a core component of continuous professional development (CPD). The term CPD addresses a wide range of skills, including education, training, audit, management, team building and communication.¹⁰ The terms CME and CPD are often used interchangeably. CME contributes to CPD, whereas Maintenance of Certification or Revalidation certifies the CPD of specialists.^11,12

Current CME curricula are being influenced by the requirements of urologists who are undergoing recertification. These curricula are therefore focused on providing up-to-date, patient-centred and evidence-based care. Recently, the emphasis of CME has shifted from simply improving knowledge to improving skills, performance and patient outcome through altering clinician practice behaviours.¹³ Therefore, clinicians are increasingly engaging in learning activities that provide specialist teaching beyond didactic lectures.¹⁴ The CPD activities of practicing clinicians are being integrated with the core competencies of practice-based learning and improvement (PBLI).¹⁵ Consequently, newer CME methods are being designed to encompass the concept of PBLI. These CME methods use multimedia, multi-technique and multiple exposures to maximize compliance with PBLI (Table 1). Professional healthcare regulatory bodies have suggested using these recent CME techniques to enhance the effectiveness and application of continuing education. However, their effectiveness for use in recertification remains to be established. We conducted a literature search (Medline, Psychinfo and Google Scholar) to include studies investigating the effectiveness of CME on knowledge, skills, performance and outcome in surgical skills (specialties with interventional procedures) and non-surgical disciplines at specialist level.

Table 1.

An overview of methods for Continuing Medical Education

Educational methods	Definition
Academic detailing	Service-oriented outreach education provided by an institution (medical governing bodies or industry) or hospital.
Audience response systems	Type of interaction associated with the use of audience response systems. It addresses knowledge objectives (used in combination with live lectures or discussion groups).
Case-based learning	An instructional design model that is a case-oriented learning. It addresses high order knowledge and skill objectives (actual or authored clinical cases are created to highlight learning objectives).
Clinical experiences	Clinical experiences address skill, knowledge, decision making and attitudinal objectives (Preceptorship or observership with an expert to gain experience).
Demonstration	Involves teaching or explaining by showing how to do or use something. It addresses skill and or knowledge objectives (live or video or audio media).
Discussion group	Addresses knowledge, especially application or higher order knowledge (readings, or another experience).
Feedback	Addresses knowledge and decision-making (the provision of information about an individual’s performance to learners).
Lecture	Lectures address knowledge content (live, video, audio).
Mentor or preceptor	Personal skills developmental relationship in which an experienced clinician helps a less experienced clinician. It addresses higher order cognitive and technical skills. Also used to teach new set of technical skills.
Point of care	It addresses knowledge and higher order cognitive objectives (decision-making). Information that is provided at the time of clinical need, integrated into chart or electronic medical record.
Problem-based learning or team-based learning	PBL is a clinician-centred instructional strategy in which clinicians collaboratively solve problems and reflect on their experiences. It addresses higher order knowledge objectives, meta-cognition, and some skill (group work) objectives (Clinical scenario/discussion).
Programmed learning	It aims to manage clinician learning under controlled conditions. Addresses knowledge objectives (delivery of contents in sequential steps).
Readings	Reading addresses knowledge content or background for attitudinal objectives (journals, newsletters, searching online).
Role play	Addresses skill, knowledge, and affective objectives.
Simulation	Addresses knowledge, team working, decision making and technical skill objectives (full simulation; partial task simulation; computer simulation; virtual reality; standardized patient; role play).
Standardized patient	Addresses skill and some knowledge and affective objectives. Usually used for communication and physical examination skills training and assessment.
Writing and authoring	Addresses knowledge and affective objectives. Usually used for assessment purposes.

Adapted from Marinopoulos et al.¹⁶

We provide an overview of: (1) instructional methods for CME at the specialist level; (2) effectiveness of each method in view of surgical and non-surgical specialties, and (3) the practice of CME across various regions (including its providers). We also address current limitations and future recommendations to provide the best available evidence for the link between CME and recertification.

Instructional methods for CME

CME can take on various forms (Table 1). The media methods for provision of CME include:^9,16 (1) live in person; (2) Internet/computer-based learning (through the Internet or CD-ROMs); (3) video; (4) audio; (5) handheld materials; and (6) educational printed materials.

Effectiveness of CME

An ideal system of CME must ensure that a clinician’s knowledge, psychomotor skills, performance and clinical outcomes are up-to-date for safe medical practice. These factors contribute to clinical competence and performance.

Clinician knowledge

Knowledge is directly related to clinical outcomes and psychomotor skills.^16,17 and is inversely correlated to the number of years since certification.¹⁸ Surgeons certified through knowledge testing by the American College of Surgeons reported lower morbidity and mortality rates following segmental colon resection and repair of ruptured abdominal aortic aneurysms compared to surgeons who did not undergo certification.^19,20 Similarly, patients managed by certified cardiologists have a 19% lower rate of mortality following myocardial infarction compared to patients who are managed by cardiologists who lack certification.²¹ The provision of up-to-date knowledge (whichever the medium) is an essential component of CME.²²

We analyzed 28 studies by the Johns Hopkins Evidence-based Practice Centre assessing the knowledge gained by specialists through CME. Of these studies, 22 (79%) demonstrated an improvement in knowledge, 4 (14%) had no difference and 2 (7%) had mixed results.⁹ Fifteen out of 22 studies with follow-up (68%) all demonstrated long-term knowledge retention.⁹ Regarding multimedia, 4 of the 28 studies compared single medium versus multimedia CME interventions. Three of these 28 studies favoured multimedia interventions. Among the 28, 5 studies compared single technique versus multiple instructional techniques of CME. Two of these 5 studies showed that knowledge increased with multiple learning methods. Another group of authors also suggested that single method CME interventions have little impact.²² Instead, multiple modality activities that require the application of knowledge and problem-solving have been reported to be good indicators of competency.

CME has been demonstrated to provide both short- and long-term advantages.⁹ However, studies investigating multimedia and multiple instructional CME techniques can be confounded by repetition in certain areas of knowledge leading to inaccurate results. Novel teaching media can be used to negate these repetition effects. As a result, it has been recommended that CME teaching should encompass multimedia, multiple instructional techniques and multiple exposures whenever possible.²² The effectiveness of CME knowledge modules has not yet been established in relation to the process of recertification. Moreover, in the American certification is an optional process and it is possible that a self-selection bias may occur and influence the results of various available studies.

Clinician skills

The degree of competence in an individual’s clinical skills represents the extent of their clinical knowledge. Clinical skills can be categorized into cognitive (ability to apply knowledge) and psychomotor (procedural or physical examination techniques).¹⁶ CME has been shown to be effective at improving clinical skills. Leopold and colleagues taught primary care providers how to perform a knee injection with various CME instructional methods.²³ After attending a 15-minute lecture on the relevant anatomy and injection techniques, 93 participants were randomized into three groups. The first group received written instructions, the second group watched a videotape of the procedure and the third group underwent hands-on training with feedback. All groups significantly improved at assessment (simulated knee injection on model) with no significant differences between the groups.²³

Similarly, Hergenroeder and colleagues taught pediatricians how to physically examine the ankle and knee using either written instructions and videotape alone or written instructions, videotape and a hands-on skills workshop.²⁴ When assessed 5 months later, both groups showed significant improvement in examination skills, with the greatest change demonstrated in the hands-on skills group.

For psychomotor skills training, only simple out-patient procedures have been studied.^23–25 The participants in these studies were primary care clinicians. Training was provided by using multiple instructional techniques and multiple media. Due to a lack of evidence, no conclusions can be drawn regarding effective CME technique, media, exposure or frequency in improving procedural skills. There are no studies addressing the training effectiveness of any CME methods at a specialist level within tertiary care centres.

In surgical specialties, simulation has emerged as a novel method for skills training.²⁶ Haque and Srinivasan reported that virtual reality (VR) simulation is highly effective at demonstrating both construct validity (ability to differentiate junior versus experienced clinicians) and skill transfer from the training to operating room environments.²⁷ However, the reliability and validity of VR or any other type of simulation at the specialist level has not been established.⁶ This is due to a lack of the complexity in the models and the level of interaction required for training this group of clinicians.

The available evidence demonstrates that CME methods can be effective at teaching clinical skills to primary care clinicians; however, there is currently no evidence for the use of applying CME methods to train hospital specialists.

Clinician performance

Clinical performance involves the activities of physicians and surgeons in their professional practice. The effect of CME on the performance of clinicians is debatable. The gap between ideal and actual performance in the practice setting is of great interest to policy makers, managers and accreditation councils.

Davis and Galbraith identified 105 studies evaluating the effect of CME on short- and long-term practice performance.¹³ Practice objectives ranged from guideline adherence, screening, prescribing and diet to giving smoking cessation advice. Sixty-one studies (58%) met practice objectives. Participants in 50 studies (48%) met several objectives, all of which were maintained long term (30 days to 1 year).¹³ Single live media or multimedia CME methods have been shown to be effective at improving practice performance in the short and long term, while single print media failed to meet practice objectives. Similarly, it was demonstrated that multiple exposure and multiple instructional techniques should be employed whenever possible.¹³

In contrast to primary care and certain other medical specialities where there was a positive correlation between CME and performance, this has not been demonstrated in various surgical specialities, including surgery, orthopaedics and cardiology.⁶

Clinical outcome

Clinical outcomes include patient health status, patient attitudes and the healthcare attitudes of professionals.¹⁶ CME aims to improve clinical outcomes that are considered to be the foremost measure of clinical performance. Outcomes can directly indicate whether an individual has improved health and whether a healthcare system is achieving its purpose.²⁸

Different specialties rely on diverse clinical outcomes to identify healthcare effectiveness. These outcomes range from cholesterol levels in patients recently started on statin therapy or an allergic reaction from a single dose of antibiotic to complications after surgery. Surgical specialties are frequently concerned with morbidity and mortality data. It has been shown that interactive CME sessions involving skills training can effect change in professional practice and healthcare outcomes.¹

Primary care specialists who underwent a single interactive CME seminar significantly reduced subsequent emergency department admission rates for children with asthma from a low-income family versus the control group (annual rate 0.208 vs. 1.441).²⁹ Similarly, specialists who underwent a web-based education program significantly improved dosing accuracy, while reducing dosing time compared to controls.³⁰

In a systematic review, Marinopoulos and colleagues investigated whether CME influenced clinical practice outcomes and to see if this effect persisted for more than 30 days.⁹ Thirty-three studies measured the effect of CME on long-term clinical outcomes and 13 of these 33 studies (39%) demonstrated the beneficial effect of CME. The data were inconclusive regarding single versus multiple-exposure CME in improving outcome.⁹ This systematic review demonstrates that the multiple, media and educational techniques benefit long-term clinical outcomes.

This early evidence maintains that CME exercises should be used to improve clinical outcomes. However, studies are still needed to correlate CME to the recertification process.

CME improves the success rates of passing recertification examinations. Butterworth and Reppert discovered that CME significantly reduces the age-related decline in diagnostic performance of general practitioners.³¹ Recently, the results of U.S. recertification examinations showed significantly higher failure rates for participants who were more than 30 years out of training than those who had completed their original training recently.³² Several factors, such as working in an isolated practice, inadequate up-to-date knowledge or limited provision of CME-type exercises, can result in poor outcomes.³² Another possible explanation for poor results could be that clinical practice requires a specific focus and the recertification exam may not test for the appropriate knowledge to indicate individual competence.

CME providers

Although the ultimate aim of CME is focused at adequate patient care, CME practices and activities differ widely across various regions (http://journals.sfu.ca/cuaj/index.php/journal/article/view/378/1275).^33–39

CME is provided by several organizations, including scientific societies, medical institutions, professional bodies, academic centres and private companies.⁴⁰ These are accredited by medical associations or public authorities (Table 2). In the U.S. all CME providers must be accredited by the Accreditations Council for Continuing Medical Education (ACCME) or an ACCME-approved state medical society.² Within the continent Europe, the European Accreditation Council for CME (EACCME) assures the quality for CME program.⁴¹ In the U.K., the management and accreditation of CME programs is currently undergoing review.¹² CME activities offered by each individual provider differ widely in content due to speciality requirements, regional demands, organizational management and funding pressures. As a result the current global provision of CME can be diverse and varied, even within individual specialties. This will require internationally endorsed measures to unify and substantiate CME provision. Garratini and colleagues recommend accrediting individual CME events rather than the providers themselves.⁴⁰

The pharmaceutical industry is by far the largest sponsor of CME activities worldwide. In certain European countries this accounts for about 75% of all CME provision. For instance, two-thirds of all 159 accredited CME providers in France declare a conflict of interest due to industry-related funding.⁴² In an effort to limit this commercial influence, organizers have to respect the scientific independence of CME, clearly delineating what is scientific versus what is promotional.⁴⁰ Despite these restrictions, CME provision has presented itself as a very active market for the pharmaceutical industry.

Many countries are in the process of reforming their CME systems. The Institute of Medicine (IOM) in the U.S. recently published an independent review of CME in health care.⁴³ Various recommendations were delivered to help institutions explore evidence, objectivity and responsiveness behind existing CME methods and practice. Recommendations of the IOM on the current challenges in CME and on using the CPD platform to address deficiencies in the system should pave the way for other international organizations to take similar steps. The definitions and mechanisms of ensuring competence and performance vary considerably across countries. Therefore, regional initiatives need to be taken to address issues related to the specialist education.

Limitations and future considerations for CME

Current CME practice has expanded significantly since the late 1990s. Since that time in the U.S., there have been 10% more certified CME providers, 40% more CME activities, 10% more hours of instruction, and 40% more clinician participants.⁴⁴ Due to this steep growth, the CME enterprise is steadily establishing itself as an important component of healthcare quality improvement. This section outlines the factors that are likely to influence the future of CME and suggests strategies for CME professionals to ensure that their work can meet professional expectations.

Most CME systems employ an hour-related credit system, where 1 hour of CME activity is equivalent to 1 credit. The required number of credits varies from 50 to 200 per year.⁴⁵ This system has been widely debated, as quantity of hours does not necessarily equal a change in quality of performance. Monitoring a change in outcome is more valid, but its objective measurement is challenging. Moreover, many CME providers have close relationships with biomedical and pharmaceutical companies. This potential for conflict of interest in the CME process could undermine the validity and reliability of CME through commercial bias.⁴⁶ If CME is to be a reliable medium of lifelong learning, guidelines need to be introduced to prevent the influence of commercial bias. Several structural and organizational questions must be answered before the establishment of acceptable and feasible CME tools. First, do methods exist to assess a specialist learner’s needs? Second, are there any program designed in line with the requirements for recertification, that meet the learners’ requirements in both surgical and non-surgical disciplines? Third, is there any evidence that a particular method or combination of CME delivery methods are more effective in providing knowledge and skills to clinicians to correspond with changing attitudes, clinical practice and outcome requirements? Fourth, what is the evidence for the use of simulation in CME? Finally, what do specialists think about the requirements, structure and frequency of the CME process?

The optimal method of CME and its enforcement remain to be decided. At the specialist level, CME activities must be designed to address learners’ needs (Fig. 1). The effectiveness of most CME components for the process of recertification remains vague.

Fig. 1.

Recommendations for the development of a Continuing Medical Education (CME) program. NIH: National Institutes of Health; NICE: National Institute for Health and Clinical Excellence.

Funding remains one of the largest obstacles in implementing CME. Traditionally, many regions and organizations turn to the private sector, especially the pharmaceutical industry to fund even the most basic CME activities. This can result in the pharmaceutical industry influencing the content of CME activities without independent regulation. This may limit the content of CME for clinicians. As a result, there is a need for healthcare authorities to introduce safeguards and guidelines to regulate the contents of CME materials for each specialty.

An intricate CME system without established reliability and validity will not last and might ultimately restrict the freedom of medical professionals.^6,47 The system needs to be acceptable to clinicians and healthcare authorities. At present the effectiveness of the current CME system in association with the recertification process is not established (Fig. 2). The current outcomes of CME associated with the recertification practice will not be known until the completion of CME cycles that have been recently introduced across different regions.

Fig. 2.

An overview of challenges and recommendations for Continuing Medical Education in relation to recertification.

Specialist educational activity should be based on the professional activity of doctors. As previously mentioned, a combination of different CME methods is more effective than a single method. Guidelines need to be established by the accreditation organizations regarding CME methods in various surgical and non-surgical specialties. A combination of learning modes can improve the effectiveness of lifelong learning.⁴⁸ Learning in the craft disciplines will require further measures, including the incorporation of technical skills programs in line with diagnostic and therapeutic innovations.

Conclusion

Successful CME must be truly continuous and not opportunistic, erratic or casual. Despite limited evidence, CME appears to be useful at the acquisition and retention of knowledge, attitudes, skills, behaviours and clinical outcomes. Improved CME practice must in turn lead to continuing critical reflection, practice modification and implementation with a focus towards the excellence of patient care.

Systematic, centralized and collaborative research is needed to better assess the unique and combined contributions of the various aspects of CME. Clinicians must identify their educational needs, develop activities to target those needs and as result set objective criteria of performance to achieve these targets. If developed appropriately, CME can offer the opportunity to fulfil many of the objectives of specialist recertification and revalidation.