Maintenance of Certification: Update on Attitudes of Members of the American Society of Neuroradiology

Abstract

BACKGROUND AND PURPOSE:

Neuroradiology is in the fifth year of subspecialty recertification. New requirements for the MOC process include a PQI initiative. The purpose of this report was to survey compliance of ASNR members with the MOC process.

MATERIALS AND METHODS:

A 12-question Internet-based survey was sent to the 2864 members of the ASNR through e-mail addresses. The survey asked questions regarding compliance with cognitive testing, SAMs, CME credits, and PQI. Multiple reminders were sent and open comments graded for attitudes toward the MOC process.

RESULTS:

While most respondents were engaged in the MOC process (878/1074 [81.8%]) and were aware of CME and SAMs, (736/1067 [69%]) and cognitive testing requirements of the recertification, the PQI initiative had limited participation and many questions about the component. Of the 1057 respondents, 687 (490 not started, 87 registered but not started, 110 unaware of PQI) or 65% had not started a PQI project at the time of the survey. The ASNR was asked to help with the cognitive examination by 25 (4.4%), CME offerings by 53 (9.4%), SAM modules by 88 (14.0%), and PQI projects by 205 (36.2%) respondents. Open comments were generally unfavorable toward the MOC-PQI process by a 3:1 ratio.

CONCLUSIONS:

Compliance with the MOC process has improved since a prior survey 3 years ago; however, confusion over the PQI process remains the primary concern of ASNR members.

In 1998, the ABMS chartered a task force on competency, driven by a number of outside forces, including consumer concern about physician competency and quality of care and increased health care costs. The 4 components of MOC were adopted by all ABMS member boards in 2000, and by 2006 all member boards had received approval for their MOC programs.

The ABR MOC process has been in effect for more than a decade with the implementation of 3 of the 4 requirements: maintenance of professional standing (eg, licensure), lifelong learning and self-assessment (CME and SAMs), and cognitive expertise (examination).

In 2008, the fourth component of practice performance assessment was added, dubbed the PQI initiative. The concept of quality improvement is seen as a key component of MOC, demonstrating that certified physicians are continuously improving care, reducing risk, increasing patient safety, and providing better cost-benefit ratios. Radiologists engaging in the MOC process are required to participate in projects of practice enhancement, which entail data collection, analysis, implementation of a course of action, recollection of data, reanalysis, and assessment for improvement. As with any large program that applies to diverse and widely scattered physician groups, the promotion and implementation of this newest MOC component has had variable recognition, acceptance, and application.

In late 2006 and early 2007, the ASNR membership was surveyed to assess acceptance of the MOC process and to gauge the attitudes toward the components. At that time the compliance rates for the CME and SAM portions of the MOC ranged from 10% to 30%.¹ Most open-ended comments regarding the MOC process were neutral (46.6%) or favorable (6.1%), but there were a number of detractors of the process. With the new requirement of PQI projects, we decided to resurvey the membership to assess for changes in attitudes and compliance with the MOC process.

Materials and Methods

Between July 2009 and September 2009, an on-line survey consisting of 12 questions, including 2 opportunities for open-ended comments, was sent to 2864 members of the ASNR. Of these, 65% or 1862 were senior members of the ASNR (definition at www.asnr.org/asnr/brochure/apps/mem_info.pdf). The text of the questions can be found in the On-line Table. Each member was reminded on 4 separate occasions between July 7 and August 31 to answer the survey. By electronic feedback means, each person could complete the survey only once and after completing the survey that person was no longer included on subsequent e-mail requests/reminders.

The results in the open-ended commentary portion of the survey were categorized as favorable, unfavorable, or neutral toward the MOC process. One of the questions queried how the ASNR might better serve its membership through the MOC process and those answers were separated into CME, SAM, cognitive examination, PQI, and administrative categories.

The responses were compared with the previous AJNR article on the topic published 2 years ago.¹

Results

The complete results of the survey may be found at http://members.asnr.org/misc/surveyresults/MOC2009.pdf.

One thousand seventy-four (37.5%) survey responses were obtained from 2864 e-mail surveys sent, but some multiple choice questions were omitted by respondents such that the maximum number of responses was 1074 (for question 1) and the minimum was 361 for question 7. There were 453 answers to the open-ended questions that dealt with the ASNR's role in helping with the MOC process and 254 other comments about the process.

Cognitive Expertise: Examination

For respondents, 878 of 1074 (81.8%) were registered with the ABR for the MOC process. Five hundred forty-three of 1073 answering (50.6%) had taken the MOC recertification examination, up from 41.1% in the previous survey. Most of the individuals taking the examination did so between 2005 and 2006 (287 of 545, or 52.7%), with fewer than 100 taking the examination in 2007, 2008, and 2009 (although the survey was sent before the November 2009 examination opportunity). Of those not taking the examination and answering the question (n = 558), 322 (57.7%) were not due for the examination. However, 58 (10.4%) of these respondents indicated that they did not support the MOC process, up from 8% from 2006–2007.

Lifelong Learning and Self-Assessment: CME and SAMs

SAMs.

Most of respondents, that is, 736/1067 (69%), stated that they had completed SAMs as part of the MOC process. Previously, only 21.4% (217/1012) had completed SAMs. While 125 of 749 (16.7%) had completed >8 modules, most were in the 3–8 range (526/746, or 70.5%). This compares with most respondents (78.7%) having <2 SAM credits at the time of the previous survey. As to the reasons why they had not completed SAMs, 124 of 361 (34.3%) plan to take the modules in the future, and 71 of 361 (19.7%) do not support the SAM component of the MOC process, with the latter being up from previous results (7.3%). A number of respondents remained unaware that they had to complete SAMs (45/361, or 12.5%), and several (33/361, or 9.1%) did not know where they were offered. Still, this is an improvement over the prior survey, in which 21.6% were unaware of SAMs and 12.4% did not know where to complete them.

CME.

CME offerings at the ASNR annual meeting (226/976, or 23.2%), other society meetings including the RSNA (298/976, or 30.5%), dedicated CME courses (240/976, or 24.6%), and Web-based CME content (212/976, or 21.7%) account for the sources of CME credits according to those responding. The ASNR meeting as a source of CME credits has declined by 6.5% since the previous survey when 295 of 993 (29.7%) said it would be their primary source for CME credits. Many survey respondents referred to the Neuroimaging Clinics of North America and the ACR Case in Point as sources of CME credits.

Practice Performance Assessment: Practice Quality Improvement

Of the 1057 respondents to this question, 687 (490 not started, 87 registered but not started, 110 unaware of PQI) or 65% had not started a PQI project at the time of the survey. Eighty-three (7.9%) had completed a project. There were 287 (27.2%) individuals actively engaged in a PQI project in addition to the 83 who had completed the PQI project. Still, 110 (10.4%) respondents indicated that they were unaware of [the existence of] PQI projects.

ASNR Role and Comments About MOC

Of the respondents, 628 of 1017 (61.8%) would like the assistance of the ASNR with the MOC process. When asked how the ASNR could help, the 453 answers were categorized into those that asked for help with the cognitive examination (25, or 4.4%), CME offerings (53, or 9.4%), SAM (88, or 14.0%), PQI projects (205, or 36.2%), general administrative information on the process (125, or 22.1%), and other (70, or 12.4%). Thirty-five people requested that the ASNR work to revoke the MOC process, particularly targeting the PQI requirement (30/35, or 83.3%).

There were 254 surveyed members who provided open comments about the MOC process. These were subdivided into unfavorable (192, or 75.6%), neutral (38, or 14.9%), and favorable (25, or 9.8%). Of note, some of the comments contained both favorable and unfavorable statements. In the previous survey, 46.6% of the 247 comments were unfavorable, 47.4% were neutral, and 6.1% were favorable. Among the unfavorable comments in the current survey, 64 cited the PQI component as the source of their negativism, 49 cited the cost, 6 were unhappy with the examination, 3 with the SAM requirement, and 2 with the CME requirement.

Discussion

The MOC is a multifaceted process that includes 1) passing a cognitive test of medical knowledge, 2) completing 20 SAMs in a 10-year period, 3) documenting 250 category 1 CME hours over a 10-year period with at most 80% in specialty-specific or related areas, and 4) engaging in 3 PQI measures over 10 years. Additionally, maintaining current and unrestricted state licensure is deemed necessary for maintaining certification as part of the “professional standing” requirement. Throughout the maintenance of certification process, diplomates must hold unrestricted licenses to practice medicine in at least 1 jurisdiction in the United States, its territories, or Canada. In neuroradiology, by 2008, 2700 diplomates were in their first 10-year cycle of MOC and 1300 were in their second 10-year cycle.

Certification and MOC programs for neuroradiologists are unique to the United States in the world. Neuroradiology subspecialty certification is offered in only 8 countries: Belgium, Brazil, Canada, Germany, Mexico, Romania, Thailand, and the United States.² Renewal of training/proficiency certification for neuroradiology is required in Brazil, Canada, Germany, Mexico, Romania, and the United States, with all of the other countries besides the United States requiring renewal at 5-year intervals. Most of the other countries engaged in recertification programs have limited requirements based on CME credits.

The SAM and the CME requirements should be predominantly in areas that reflect actual practice activities. SAMs must be approved by the ABR and consist of instructional content followed by multiple-choice questions, with immediate feedback to the respondent. Currently, only 4 SAMs of the 20 required can be credited in any 1 year. Twenty percent of SAMs (4 over 10 years) must be of general knowledge categories (eg, consent, contrast administration, reactions, medical ethics, patient safety, radiation physics). The concept of self-assessment is one that the ABR is committed to so as to encourage a culture of reflection and evaluation and then intervention for self-improvement.³ This personalized plan for education based on self-evaluation is designed to target each person's weak areas rather than prescribing a blanket educational program for all radiologists.³ Ideally the SAMs provide further reading guides with references and/or links to CME modules that can address any self-determined gaps in knowledge. Currently on the ABR Web site (as of November 15, 2009) there are 29 neuroradiology SAMS available and 28 general noninterpretive skills modules available for maintaining certification (http://theabr.org/moc/moc_neuro/moc_neuro_sam.html). Of the 29 neuroradiology SAMs, all but 7 have been created through the ASNR and its subspecialty societies. Currently on the ASNR Web site 22 SAM modules are available (http://members.asnr.org/ecme/). Additionally, there are >150 CME credit hours also available through the ASNR Web site.

The ABMS has endorsed the ABR process as a means of evaluating physician competence and efforts to improve patient care and outcomes. Practice improvement is 1 of the components of all MOC programs of every ABMS member board. This has led to the practice quality improvement initiative fashioned by the ABR.

The PQI component of the MOC was initiated in 2007–2008. Each physician must demonstrate a commitment to improving practice quality and enhancing competence.⁴ According to the ABR Web site, 5 categories of PQI projects have been defined: 1) patient safety, 2) accuracy of interpretation, 3) report turnaround time, 4) practice guidelines and technical standards, and 5) referring physician surveys. Although projects can be developed by an individual, the MOC process allows group practices, academic departments or divisions, or national societies to engage in PQI projects together. The defined steps necessary for the PQI project are: 1) learn about PQI process during year 1; 2) select project and metrics; 3) collect baseline data; 4) analyze the data; 5) create improvement plan; 6) collect data, compare with initial data, and summarize results; 7) modify plan yet again as needed; 8) repeat cycles 3–6; 9) prepare final report.

By the end of the first year of the PQI cycle, the diplomate must complete an attestation statement on his or her personal data base, indicating that he or she has learned about the concepts and tools of quality improvement.

The ASNR has numerous links to PQI information and a template for 1 PQI project involving spine disk terminology nomenclature (www.asnr.org/pqi/survey.shtml#example). Additional projects can be found at the ABR Web site (www.theabr.org/moc/moc_neuro/moc_neuro_pqi_projects.html). Although what is assembled there applies to multiple specialties, projects pertaining specifically to neuroradiologists are 1) universal protocol/procedural pause, 2) intravenous iodinated contrast extravasation during CT, 3) fluoroscopy dose recording, 4) prospective analysis for radiation dose reduction, 5) patient safety improvement program, 6) RADPEER, and 7) Image Gently: PQI in safety for children undergoing CT scan.

A personal data base to collate one's CME, SAM, licensure, and PQI data is offered through the ASNR, ABR, or the CME gateway. The ABR central repository (www.abronline.org/MOC/DR/home.cfm) is useful in maintaining the raw material for completion of MOC requirements in the 10-year cycle. Through the site one can pay the fees, register for the cognitive test, update personal information, and store credits as needed.⁴

The ABR has made the case repeatedly that self-regulation is in the best interest of the field of radiology rather than having the government or an outside agency legislate the process for radiology.^5,6 There is ample evidence that if the program of self-regulation through the MOC “fails” (and failure can be defined by the public or the government in a variety of ways), the field of radiology may be subject to government regulations that will be more onerous, more expensive, and less valuable than those proposed by the ABR and ABMS. A prime example of outside regulations imposed on medicine is government intervention in resident physician duty hours and the upheaval that this caused. Hattery and Dunnick attempted to rally the field of radiology to the MOC process in 2006 with a review urging radiologists to “seize the opportunity to explore adult learning and the process of MOC,” adding that “Shaping the future is a noble task requiring leadership, vision, and patience throughout change.”⁶

In the 2006–2007 survey, there were some common criticisms, particularly cost, which was $2800 for the 10-year process.¹ Physicians also complained about the limited locales for taking the recertification cognitive test. These complaints were echoed by an editorial written by David Hackney in 2006, in which he stated that the ABR should 1) establish better communication and more transparent goals, 2) reduce the cost of the MOC, 3) make test centers widely available, and 4) make the content more closely related to modern clinical study interpretation and consultation.⁷ However, despite the above directives, cost and availability remain problematic. The fee has not changed since the 2006–2007 survey, despite the promise that it “will be adjusted as actual expenses are defined more clearly.”⁸ To its credit the ABR has been transparent in documenting its costs and the justification of the fees (see www.theabr.org/moc/moc_feeinfo.html). Additionally, while the examination is being offered at the annual meetings of the ASNR and the RSNA, as well as at limited testing sites across the country, the hope of expanding sites to “national testing centers” as suggested in a commentary in February 2008 has not materialized.⁸ In fact, according to the ABR Web site, aside from the annual society meetings, the examination will only be offered in Tucson, Arizona, in 2010 and 2011. Travel expenses thus potentially add to the economic burden of recertification.

The results of the current survey reflect these issues, indicating that a growing number of ASNR members are unhappy with the MOC process. Complaints about cost and accessibility were echoed in the current survey, and the practice quality improvement component has quickly become the biggest point of contention, with 64 of 192 (33.3%) of the open-ended negative comments directed at the PQI requirement. Many found the PQI component “worthless,” “time-consuming,” and “not beneficial” to their practice. The expansion of the PQI requirement from 1 project in 10 years to 3 projects in 10 years was another source of unhappiness. A number of respondents commented on the positive value of the cognitive test, SAMs, and CME at the same time that they challenged the value of the PQI. Revoking this portion of the MOC process was advocated by a substantial number of people providing open comments. The expense of the MOC process was also criticized but was less cited in the current survey compared with the 2006–2007 survey,¹ as dissatisfaction with the PQI process gained new fervor.

Additionally, the survey indicates that a significant number of ASNR members remain unaware of the existence of the core components of MOC, in particular SAMs and PQI, the latter of which is viewed as a cornerstone for improvement in quality of patient care, which was the impetus for MOC in the first place.⁹ Many of the open-ended comments complained that the requirements for the PQI are vague and confusing, which could lead some radiologists to neglect the process entirely.

Recent changes proposed by the ABMS include adding surveys of patients and referring physicians, and/or evidence of medical safety, as part of the professional standing requirement. The ABMS has announced that it will soon be requiring 8 hours of SAMs per year, leading to a total of 80 SAM hours in 10 years. Currently, most SAM sessions are 90 minutes, which would mean increasing from 2 SAMS to 5–6 SAMS a year. A computerized test for the initial neuroradiology subspecialty examination is projected for 2012, replacing the oral examination.

Arl Van Moore, Jr summarized the importance of the MOC process in 2007 with his statement, “We as a profession need to embrace MOC as a verifiable process by which we can demonstrate to patients that we are best qualified to deliver potentially lifesaving examinations and procedures and that quality patient care is what is most important to us and our profession …. As we engage in this process, we should not view it as a burdensome claim on our time; this is a rewarding opportunity to shine a positive light on the care that we provide.”¹⁰ Our survey suggests that the admirable goal of patient care improvement through the MOC requires additional education and support as we move forward. The members of the ASNR surveyed provided the commentary that they are looking for their society to provide the resources necessary to painlessly meet the requirements of the MOC. In so doing the ASNR can bring the goals of the individual members, the ASNR, the ABR, and the ABMS into concert and thereby improve health care.

Abbreviations

ABMS

American Board of Medical Specialties

ABR

American Board of Radiology

ASNR

American Society of Neuroradiology

CAQ

Certification of Added Qualification

CME

continuing medical education

MOC

maintenance of certification

PQI

practice quality improvement

RSNA

Radiological Society of North America

SAM

self-assessment module