Costs and Benefits of Initial Certification for Emergency Medicine Residency Graduates

Abstract

Background

Graduates of emergency medicine residency programs can seek certification from the American Board of Emergency Medicine (ABEM), yet the costs and perceived value by residents is not clear.

Objective

This report sought to better define the value of board certification by asking physicians taking the ABEM Oral Certification Examination (OCE) to describe its costs (eg, time, money) and perceived benefits.

Methods

A descriptive, cross-sectional, voluntary, anonymous survey was administered to physicians taking the 2018 spring and fall ABEM OCEs. Response frequencies were used to report response rates.

Results

There were 2016 physicians who participated in the 2018 OCEs, of whom 1565 (78%) completed a survey. With respect to preparation, 38% (599 of 1565 responses) spent more than 30 hours preparing for the examination. Regarding the expense of preparing for the examination, 21% (328) spent nothing, 50% (776) spent less than $1,000, and 2% (38) spent more than $3,000. Most physicians (80%, 1254) reported a learning benefit to preparing for and taking the OCE. There were 49% (765) of respondents who reported that preparing for the examination reinforced their knowledge of emergency medicine; 20% (311) reported no learning benefit. Most physicians (92%, 1442) reported that ABEM certification provided a career benefit, the most common of which was more career opportunities (69%, 1076).

Conclusions

Initial certification requires a considerable investment of time and money. Physicians seeking initial ABEM certification found both learning and professional benefits, with the most frequently reported being reinforcement of medical knowledge and more career opportunities.

What was known and gap

Most graduates of emergency medicine residency programs become certified by the American Board of Emergency Medicine (ABEM), yet it is not known how the value and costs of certification are perceived by residents.

What is new

A cross-sectional, voluntary, anonymous survey was administered to physicians taking the 2018 spring and fall ABEM Oral Certification Examinations.

Limitations

Self-reporting of costs and administering the survey during the examination could have led to social desirability bias; survey validity evidence was limited.

Bottom line

Physicians do invest considerable time and money into initial certification, but they report receiving learning and career benefits.

Introduction

To become certified by the American Board of Emergency Medicine (ABEM), physicians undergo rigorous residency training followed by secure multiple-choice-question and oral examinations. Though the value of initial certification is generally not contested, some physicians in other specialties have questioned the value of maintaining or renewing certification.^1,2 Board-certified physicians have been shown to deliver improved quality of care and have fewer medical malpractice complaints.^3–13 Prior studies^14,15 have looked at preparation for the ABEM recertification examination and the career and learning benefits of staying certified. However, the value of initially achieving certification in most specialties, specifically emergency medicine, has been poorly defined.

This study sought to understand the perceived value of board certification by describing examination preparation costs and the benefits of becoming board certified.

Methods

In this descriptive, cross-sectional study, a survey was administered to physicians taking the 2018 spring and fall ABEM Oral Certification Examinations (OCEs). The spring OCE was administered May 5–7, 2018, and the fall OCE was administered October 13–15, 2018. The anonymous, voluntary survey was available to physicians during the OCE administration in paper-and-pencil format, which physicians dropped into an on-site sealed box. The de-identified survey responses were manually loaded into a secure database at ABEM headquarters. Response frequencies were calculated (Table).

Table.

American Board of Emergency Medicine Oral Certification Examination 2018 Survey Responses

Item	Response Option	No. of Responses	Response Frequency, %
Survey respondents	N/A	1565	77.6
Time spent studying	None	8	0.5
	1–10 h	126	8.1
	11–20 h	365	23.3
	21–30 h	438	28.0
	> 30 h	599	38.3
	No response	29	1.9
Money spent on preparation	None	328	21.0
	< $1,000	776	49.6
	$1,001–$2,000	228	14.6
	$2,001–$3,000	122	7.8
	> $3,000	38	2.4
	No response	73	4.7
Attended an in-person preparation course	Yes	302	19.3
	No	1263	80.7
My knowledge of emergency medicine was increased by preparing for this examination.	Yes	689	44.0
	No	876	56.0
Preparing for this examination reinforced my knowledge of emergency medicine.	Yes	765	48.9
	No	800	51.1
Preparing for and taking the Oral Certification Examination makes me a better clinician.	Yes	473	30.2
	No	1092	69.8
I did not derive any benefit from preparing for and taking this examination.	Yes	311	19.9
	No	1254	80.1
I will receive a higher salary or more total compensation.	Yes	441	28.2
	No	1124	71.8
I will have more career opportunities.	Yes	1076	68.8
	No	489	31.2
Other physicians will view me more positively for becoming board certified.	Yes	962	61.5
	No	603	38.5
I will not benefit from becoming board certified.	Yes	123	7.9
	No	1442	92.1

Abbreviation: N/A, not applicable.

^aNumber of test takers = 2016.

This study focused on costs entirely under the control of applicants, specifically study courses and preparation materials, in contrast to nondiscretionary costs such as examination fees and travel arrangements. Survey instructions deterred respondents from monetizing their preparation time when estimating costs. This approach was similar to that of a prior study,¹⁴ which identified examination costs as the costs of attending in-person preparation courses as well as using online examination preparation courses or written and online preparation question banks and testing manuals. Respondents also were asked about the time they spent studying for the examination after residency.

Validity evidence for survey questions was derived from expert panel development, psychometric review, and prior use, as well as refinement of a similar postexamination survey. Specific questions about examination preparation and benefits have been used for the past 5 years. There was a variable number of nonresponses to individual questions, and frequencies were calculated based on the number of actual responses for each question. Questions about potential benefits could receive multiple affirmative responses. The best way to determine if physicians perceived no benefit was to have them check a negative response option (ie, “I did not derive any benefit from preparing for and taking this examination”).

This study was reviewed and received an exception for human research by the University of Pennsylvania Institutional Review Board.

Results

Of the 2016 physicians who took the 2018 OCEs, 38% (599 of 1565) spent more than 30 hours in preparation, and another 28% (438) spent between 21 and 30 hours. Regarding the expenses excluding preparation time, 21% (328) spent nothing to prepare for the examination, 50% (776) spent less than $1,000, and 2% (38) spent more than $3,000. Nearly 1 in 5 test takers took an in-person examination preparation course.

Most physicians (80%, 1254) reported a learning benefit to preparing for and taking the OCE: 49% of physicians (765) reported that preparing for the examination reinforced their knowledge of emergency medicine, 44% (689) reported that it increased their knowledge, and 30% (473) reported that it made them a better physician. No learning benefit was reported by 20% (311) of respondents. The majority of physicians (92%, 1442) reported that becoming ABEM certified provided career benefits, the most common being more career opportunities (69%, 1076), followed by other physicians viewing them more positively (61%, 962) and receiving a higher salary (26%, 411). No benefit from ABEM certification was reported by 8% (123) of respondents.

Discussion

Physicians spent a substantial amount of time preparing for their certifying examinations. Most physicians spent less than $1,000 on preparation materials and activities. Physicians identified the primary benefits as having greater career opportunities as well as reinforcing and learning new medical knowledge.

The value of initial certification remains undetermined. Discussions about the cost of certification¹⁶ and benefits of board certification¹⁵ have often been based on isolated analyses. Similar investigations^14,15 to the current study explored the time, cost, and benefit to continuing certification in emergency medicine. In those studies, one-third of physicians took an in-person board prep course, and 70% spent over 35 hours studying for the examination.^14,15 A learning benefit was reported by 90% of physicians and a career benefit by 91%, the most common response being more job opportunities.¹⁵

The perceived value of board certification is likely to be influenced by examination fees, cost of test preparation, and amount of time spent preparing for and taking the examination, as well as any learning and career benefits. The ABEM's examination fee was the same for all physicians: $420 application fee, $960 written examination (qualifying examination) fee, and $1,225 oral certification examination fee (total $2,605). As the oral certification examination is held only in Chicago and lasts 5 hours, costs for travel and, if needed, a place to stay overnight varied greatly for physicians. This variability may have affected whether physicians purchased preparation materials or participated in preparation activities. The fluctuating costs of travel may also have affected physicians' perceived benefits from certification in ways that this survey could not assess. The cost that is most influenced by physician choice is for examination preparation activities and materials. Most physicians surveyed (71%, 1104 of 1565) spent nothing to less than $1,000 in preparation costs, while just 2% (38 of 1565) spent over $3,000.

Physicians spent a considerable amount of time studying for the examination, which represents an expenditure of personal resources. One model used for determining maintenance of certification costs monetized a physician's time dedicated to maintenance-related activities.¹⁷ Using this approach, 90% of recertification costs were attributed to monetizing a physician's time.

Direct financial benefits are difficult to determine. Fewer than 1 in 3 physicians reported a financial benefit to obtaining ABEM certification; however, salary surveys have reported a financial benefit.^18,19 Based on self-reported compensation data,¹⁸ it is estimated that in aggregate, ABEM-certified physicians receive an additional $260 million in annual total compensation compared with noncertified physicians. Most respondents (80%, 1254 of 1565) reported learning benefits from preparing for and taking the examination, while 20% (311 of 1565) did not. Few physicians reported that they would receive no career benefit from becoming board certified.

Among physicians who claimed a career benefit, more career opportunities received the highest frequency of responses, which was also the most frequently reported benefit by physicians seeking to stay ABEM certified. These findings were anticipated in that nearly all job postings for emergency department positions require board eligibility or board certification.

This study is limited by self-reports of cost, with the possibility of recall bias and social desirability bias (tendency to give answers satisfactory to the questioner), especially because the survey was administered during the examination. That respondents had just finished or were in the midst of an intense assessment experience may also have influenced their answers. Survey validity evidence was limited; thus, respondents may not have interpreted questions as intended. An ideal survey would have avoided negatively worded response options such as “I did not derive . . .” Travel costs were not determined, and it is likely that, for some participants, travel costs exceeded participation costs, with unknown effects on perceived value of certification. Finally, surveys were not linked to examination performance. A prior study¹⁵ found that higher-performing physicians perceived greater benefit to the initial examination and maintaining ABEM certification.

There are opportunities for further exploration, such as determining the average and range of travel and lodging costs. In addition, although value is subjective, physician interviews about perceptions of the value of initial board certification might provide a better assessment of perceived benefits.

Conclusions

Initial certification requires a considerable investment of time and money. Physicians seeking initial ABEM certification found both learning and professional benefits, with the most frequently reported being the reinforcement of medical knowledge and greater career opportunities.

Ultimately, the value of board certification is best defined by those who use certification as a proxy for professional capability. As certification boards seek to enhance the value of certification, physician perception is a critical index to measure. Future surveys can be improved to meet best practice standards to better capture this.