Reforms of the Korean Medical Licensing Examination regarding item development and performance evaluation

Abstract

Purpose:

The Korean Medical Licensing Examination (KMLE) has undergone a variety of innovative reforms implemented by the National Health Personnel Licensing Examination Board (NHPLEB) in order to make it a competency-based test. The purpose of this article is to describe the ways in which the KMLE has been reformed and the effect of those innovations on medical education in Korea.

Methods:

Changes in the KMLE were traced from 1994 to 2014 by reviewing the adoption of new policies by the NHPLEB and the relevant literature.

Results:

The most important reforms that turned the examination into a competency-based test were the following: First, the subjects tested on the exam were revised; second, R-type items were introduced; third, the proportion of items involving problem-solving skills was increased; and fourth, a clinical skills test was introduced in addition to the written test. The literature shows that the above reforms have resulted in more rigorous licensure standards and have improved the educational environment of medical schools in Korea.

Conclusion:

The reforms of the KMLE have led to improvements in how the competency of examinees is evaluated, as well as improvements in the educational system in medical schools in Korea.

INTRODUCTION

The purpose of the Korean Medical Licensing Examination (KMLE) is to determine if examinees have sufficient knowledge and skills to be a primary care physician, as well as a proper clinical demeanor; therefore, its items must draw on these three domains. Before 1994, the KMLE was a written test with an abundance of items evaluating factual knowledge. The National Health Personnel Licensing Examination Board (NHPLEB) reformed the written test and incorporated a clinical skills test, in order to make the KMLE a competency-based test. The KMLE currently consists of a written test and a clinical skills test. Medical school graduates must pass both components of the test to receive medical licenses in Korea. This paper aims to summarize these reforms of the KMLE and to assess their effect on the qualification rate of examinees and the environment of medical education in Korea.

METHODS

Reforms to the KMLE were reviewed, along with the corresponding results. A particular focus was placed on assessing what was changed, what each change achieved, and how the educational environment of medical schools in Korea was affected.

RESULTS

Several innovations transformed the KMLE into a competency-based test.

First, the subjects tested on the exam were reformed through job analysis. Before 1994, the subjects were based on the curricula of medical schools. There were 15 subjects: internal medicine, surgery, obstetrics and gynecology, pediatrics, neurology, psychiatry, dermatology, urology, ophthalmology, otorhinolaryngology, radiology, laboratory medicine, anesthesiology, preventive medicine, and medical law. In 1995, the examination subjects were reduced to the following core medical areas: internal medicine, surgery, pediatrics, obstetrics and gynecology, psychiatry, preventive medicine, and medical law [1]. In 2002, a second reform reduced the number of subjects to three: general medicine, specialized medicine, and medical law.

Second, extended matching (R-type) items were developed and introduced to the KMLE. The total number of test items was increased to 550, but subsequently was gradually decreased; however, the proportion of problem-solving items and R-type items continuously increased [2]. The number of multiple true-false (K-type) items decreased over time, and K-type questions were ultimately eliminated from the exam in 2010, while the proportion of R-type items increased to 13% in 2013 from 5% in 2002 (Fig. 1).

Fig. 1.

Chronological change of the percentage of items of a given type in the Korean Medical Licensing Examination. A-type, one best answer type; K-type, multiple true-false type; R-type, extended matching type.

Third, problem-solving items were continuously developed and introduced. The number of recall items decreased from 26% in 2002 to 12% in 2013, and the number of interpretation items decreased from 48% in 2002 to 26% in 2013. In contrast, the number of problem-solving items increased from 25% in 2002 to 62% in 2013 (Fig. 2). These reforms of the written test involved an increase in items dealing with clinical cases [3].

Fig. 2.

Chronological change of the percentage of items testing a given level of knowledge in the Korean Medical Licensing Examination.

Fourth, a clinical skills test was introduced in 2009 alongside the written test [4]. The clinical skills test consists of six objective structured clinical examination stations where examinees perform the indicated tasks for five minutes in each station, and another six clinical performance examination stations, where examinees encounter standardized patients for ten minutes in each station. The clinical skills test evaluates the clinical competence and demeanor of each examinee. The KMLE was the first licensing examination in Asia to incorporate a clinical skills test. The evaluation sheet of the clinical skills test contains five categories: history taking, physical examination, physician and patient interaction, attitude toward patients, and basic technical skills.

As a result of introducing the clinical skills test into the KMLE, the clinical performance abilities of medical graduates improved [5,6]. A survey indicated that adopting the clinical skills test in the KMLE improved the quality of medical school education (Table 1) [5]. A high score on the written test does not always guarantee a satisfactory score on the clinical skills test, as an analysis of examination results has shown that some examinees who received high scores on the written test failed the clinical skills test [7]. Some examinees who failed the clinical skills test in 2013 also failed the written test, with a mean score of 213 (range, 123 to 239; SD=23.5). However, some examinees who failed the clinical skills test passed the written test, with a mean score of 274 (range, 240 to 330; SD=20.4). The cutoff score for passing the written test in 2013 was 240.

Table 1.

Changes in educational facilities and curricular content in 41 medical schools after the 2009 introduction of the clinical skills test into the KMLE

Facility & curricular content	Before 2009	After 2009
Clinical skills training center	27 (65.9)	40 (97.6)
Medical interviewing skills	30 (73.2)	40 (97.6)
Physical examination course	37 (90.2)	41 (100.0)
Introduction to clinical medicine	35 (85.4)	41 (100.0)
Standardized patient program for proving feedback	19 (46.3)	35 (85.4)
Inpatient practice clerkships	38 (92.7)	41 (100.0)
Outpatient practice clerkships	32 (78.0)	38 (92.7)
Writing medical records	33 (80.5)	38 (92.7)
Medical ethics	36 (87.8)	41 (100.0)
Clinical performance examinations	31 (75.6)	41 (100.0)
Objective structured clinical examination to test simple procedural skills	32 (78.0)	41 (100.0)
Special preparatory course for the clinical skills test of the KMLE	16 (39.0)	40 (97.6)

Values are presented as number (%).

KMLE, Korean Medical Licensing Examination.

Reproduced from Park HK. J Korean Med Assoc. 2012;55:116-123, with permission of Korean Medical Association [5].

DISCUSSION

Consolidating the examination subjects made it possible to evaluate the overall medical competency of examinees, since patients’ complaints and problems are not classified by specialization during presentation to a primary care physician. The inclusion of a greater proportion of problem-solving items reflects the practical situations that medical graduates face in the field. The introduction of more items based on case studies helps measure the ability of examinees to cope with real-world situations. The discriminating index of R-type items is higher than that of other types; therefore, the proportion of R-type items will increase to 15% [3]. Introducing the clinical skills test to the KMLE affected the educational environment of medical schools in Korea. Most schools have been equipped with centers to develop clinical skills, and have introduced objective structured clinical examinations and clinical performance examinations to test the clinical competence of their students. The KMLE allows each examinee’s abilities to be examined in a variety of domains.

The NHPLEB has improved the KMLE in a step-by-step manner, with cooperation from the Korean medical community. Innovative reforms have resulted in more rigorous licensure standards. The following challenges and reforms are currently being considered: first, the written test will be replaced by a computer-based test beginning in 2020 [8]; second, a scientific standard-setting method must be considered with regard to educational measurement; third, items that deal with medical ethics may be included as essential items in the test; fourth, a hybrid item combining objective structured clinical examinations and clinical performance examinations may be created; and fifth, the subject of general medicine will be removed in 2016, after which general and specialized medicine will be consolidated into a single topic dealing with medicine.

In conclusion, the reforms of the KMLE have succeeded in improving the evaluation of examinees’ competency and the educational system in medical schools in Korea. Although reforms are still in progress, it is to be hoped that the KMLE may become a model for other countries.

SUPPLEMENTARY MATERIAL

Audio recording of the abstract.