Nationwide cross-sectional survey on the necessity of including a clinical skills assessment in the national licensure examination for Doctors of Korean Medicine

Abstract

Clinical skills assessment (CSA) has been implemented in the licensure examination of many healthcare professionals to ensure the required performance in clinical practice. Despite the high demand revealed in previous surveys, CSA has not yet been implemented in the national licensure examination (NLE) for Doctors of Korean Medicine (DKMs). Furthermore, discussions on expected outcomes and examination blueprints remain fragmented. Here, we present the perceptions of DKMs on CSA in the NLE: what should be assessed, when to implement, and the characteristics of DKMs who favor or oppose CSA. A voluntary and anonymous online nationwide cross-sectional survey was conducted in August 2022. The questionnaire included questions on demographic characteristics, whether CSA is required in the NLE for DKMs, priority of competencies to be assessed, appropriate time of introduction, and specific items to be included. Suggestions and concerns regarding the CSA were collected comprehensively through open-ended questions. Approximately 8.7% of DKMs (n = 2221 of 25,500) participated in the survey. Regarding the necessity of introducing CSA in the NLE for DKMs, 81.9% agreed, 10.8% were neutral, and 7.3% disagreed. The top prioritized competency for the CSA was “patient care,” followed by “clinical skills” and “completion of the medical record.” Clinical presentations of various symptoms, such as headache, low back pain, chronic abdominal pain, dizziness, diarrhea, and dysmenorrhea, were prioritized items for the patient encounter test. For the procedural skills test, DKMs in primary clinics perceived tongue and pulse diagnoses as necessary, whereas those in public medical institutions and hospitals deemed most examinations necessary. CSA has been perceived as highly necessary for inclusion in the NLE for DKMs, particularly with regard to priorities that reflect DKMs’ duties and actual clinical practice. While our survey cannot determine causality, it provides important preliminary evidence and serves as a foundational resource for future research and decision-making in competency-based Korean Medicine education and evaluation.

1. Introduction

Competency-based medical education emphasizes the importance of ensuring that healthcare professionals possess the necessary skills and knowledge for optimal patient-care.^[1,2] In South Korea, the medical profession is legally categorized into physicians, dentists, and Doctors of Korean Medicine (DKMs). DKMs, recognized as medical professionals with authority equivalent to that of physicians, are required to demonstrate not only medical knowledge but also clinical skills, effective communication, professionalism, and social accountability to provide comprehensive care. In clinical practice, DKMs conduct various physical examinations for diagnosis, establish therapeutic relationships, and directly perform treatment procedures such as herbal medicine, acupuncture, and Tuina (Chuna) therapy. Given these professional responsibilities, ongoing discussions have emphasized the need to incorporate assessments of attitudes and clinical skills, in addition to knowledge, into the licensure process for DKMs.^[3]

In the current licensure examination system of South Korea, a clinical skills assessment (CSA) was introduced for physicians in 2009 and for dentists in 2021.^[1] Consequently, medical and dental licenses are granted only to candidates who pass both written and clinical examinations. By contrast, the national licensure examination (NLE) for DKMs consists solely of a written test, and passing this exam alone is sufficient for licensure.

Similar professions in other countries, such as Doctors of Osteopathy in the United States, who must pass the COMLEX-USA licensure examination, as well as traditional Chinese medicine practitioners in China and Taiwan, are all required to undergo both written and clinical skill assessments as part of their licensure process.^[4–6] Despite these global precedents, there has been limited discourse on whether a similar approach should be adopted for DKMs in South Korea and which competencies should be prioritized in such an assessment.

In South Korea, DKMs share primary care responsibilities with MDs under the dual system of medical care, promoting integrative medicine for holistic treatment and management of patients.^[7,8] Korean Medicine (KM) colleges, which operate under the same academic system as conventional medical schools, have already diversified their curricula to strengthen graduates’ competencies in patient-care, communication, professionalism, and clinical management.^[9–11] Nevertheless, since the current NLE for DKMs focuses solely on knowledge-based written examinations, there is a recognized need to evaluate competencies such as clinical performance, communication, cooperation, and professionalism through CSA.^[12] Although CSA has been implemented in South Korea since 2009 for medical doctors (MDs) and since 2021 for dentists, there is limited evidence on the validity and feasibility of introducing CSA into the NLE for DKMs.^[13]

A previous survey aimed at assessing the need for innovation in the NLE for DKMs indicated a high demand for CSA, particularly among DKMs within 5 years of licensure, based on responses to the statement: “A clinical skills test should be introduced to evaluate clinical work performance.”^[14] However, no study has provided a specific blueprint for competencies and content assessment in a large-scale CSA. Nonetheless, several studies have designed and implemented CSAs, such as the Clinical Performance Examination (CPX) and objective structured clinical examination (OSCE) for KM students, these assessments have been confined to individual courses and institutions, lacking nationwide standardization and long-term outcome evaluation. Therefore, it is crucial to prioritize the perspectives of DKMs who have completed KM curricula and passed the NLE in order to comprehensively examine the feasibility of introducing CSA, identify essential competencies to be assessed, and determine skills required for licensure. This study aimed to investigate the necessity of CSA in the NLE through a nationwide survey of DKMs and to explore the sociodemographic factors associated with their perspectives. Based on these findings, we sought to identify priority areas for CSA that could bridge the gap between theoretical knowledge and clinical proficiency in KM practice.

2. Materials and methods

2.1. Participants and recruitment

This nationwide cross-sectional online survey was conducted among all the DKMs in South Korea. An online link to the survey was sent via text message and email to all registered members of the Association of KM. The number of registered DKMs in 2022 is estimated to be 25,500. The survey questionnaire, created using the online survey platform Moaform, was distributed from August 16 to 26, 2022, and participation was voluntary with no sampling involved. All licensed DKMs in South Korea were eligible to participate in the survey. Inclusion criteria were voluntary agreement to participate after providing informed consent and completion of the online questionnaire. Exclusion criteria were incomplete responses and duplicate entries.

2.2. Survey design

Based on a literature review, the survey questions were drafted by 3 researchers and then finalized through 5 meetings of the project management group of 7 researchers. A pilot test was conducted among the 7 DKMs to ensure completeness and understanding of the survey. Feedback from the pilot test was incorporated into the final version of the questionnaire by refining item wording, removing redundant items, and clarifying ambiguous terms. The survey included 14 questions and was organized into 3 sections: the demographic characteristics of the respondents, the need to introduce CSA, and test items required for CSA (Fig. 1, Additional File 1, Supplemental Digital Content, https://links.lww.com/MD/Q451). Therefore, a questionnaire on the requisite items for CSA was administered to respondents who were positive or neutral about the introduction of CSA in the NLE. The current CSA for MD by the KHPLEI includes patient encounters and procedural skills tests to assess overall clinical competence and basic technical skills, respectively. Accordingly, clinical presentations suggested by the KM education committee for simulation were drafted as survey topics for the patient encounter test. We analyzed the top 20 topics suggested for the patient encounter test. The topics of the procedural skills test were collected from the literature on KM clinical skills and the questions were drafted through project management group discussions. In the survey, the order of choices was randomized across respondents to prevent bias, and the number of items that respondents could choose was not limited. Participants who were negative about the introduction of CSA were asked only to state the reasons for their perspectives before the survey ended.

Figure 1.

Depiction of the survey process flow. CSA = clinical skills assessment, DKM = Doctor of Korean Medicine, NLE = national licensure examination.

2.3. Ethical considerations

This study was approved by the Institutional Review Board (IRB) of Gachon University (approval no. GIRB-22-107). Participation was entirely voluntary, and electronic informed consent was obtained from all respondents prior to survey initiation. Only DKMs who provided and completed the questionnaire were included in the analysis. No personally identifiable information was collected in the survey, and all responses remained anonymous to ensure confidentiality. The survey results were accessible only to the researcher who was responsible for ensuring anonymity and confidentiality.

2.4. Statistical analysis

Incomplete responses were excluded to handle missing data, and the final responses to duplicate responses were included in the analysis. Survey responses are presented as the mean ± standard deviation for continuous variables and as frequencies (%) for categorical variables. Differences in percentages according to sociodemographic characteristics were tested using chi-square or Fisher’s exact test. For the ranking question (competency priority), which allowed respondents to select up to 3 choices, the weighted average was calculated by multiplying the number of responses for each item by assigning 3, 2, and 1 points for the first, second, and third choices, respectively, and dividing the score by the total number of responses. Statistical analyses were performed using R, version 4.2.2. (R Core Team), and GraphPad Prism 10 (GraphPad Software, Inc., Boston). Opinions about CSA and suggestions for its introduction were collected using open-ended questions, which were categorized by 2 researchers. Any disagreements were resolved by discussion with a third researcher. A P-value of less significance was set at P < .05.

3. Results

3.1. Demographic characteristics of participants

Of the 2575 responses collected, 2221 were included in the analysis, after removing duplicates and incomplete responses. After excluding 163 respondents who opposed the introduction of CSA, 2058 participants answered questions on patient encounters and procedural skills tests. Participants were most likely to be in their 30s (40.2%), followed by their 40s (26.9%), 50s (15.8%), and 20s (13.2%). Most of the participants had worked in KM clinics (61.8%) at the time of the study. There were 59 and 28 DKMs affiliated with the basic and clinical departments of the KM College, respectively. A total of 68.8% of participants did not have a graduate KM education, 6.4% had completed an internship, and 24.8% had completed or were undergoing residency training. More than half of the respondents (50.1%) had held licences for DKM for <10 years (Table 1).

Table 1.

Sociodemographic characteristics of participants (N = 2221).

Classification	n (%)
Age (yr)
20–29	294 (13.2)
30–39	893 (40.2)
40–49	598 (26.9)
50–59	350 (15.8)
60–69	76 (3.4)
≥70	10 (0.5)
Gender
Male	1555 (70.0)
Female	666 (30.0)
Type of medical institution
KM clinic	1372 (61.8%)
KM hospital	454 (20.4%)
Public medical institution	221 (10.0%)
College of KM, Department of Clinical KM	59 (2.7%)
College of KM, Department of Basic KM	28 (1.3%)
Others (research organizations, etc)	87 (3.9%)
Postgraduate training
Graduated from college of KM	1528 (68.8%)
Internship completion	143 (6.4%)
Specialist or residency in training	550 (24.8%)
Years of licensure
<5	640 (28.8)
5–9	474 (21.3)
10–14	335 (15.1)
15–19	278 (12.5)
20–24	238 (10.7)
25–29	110 (5.0)
30–34	91 (4.1)
35–39	33 (1.5)
≥40	22 (1.0)

KM = Korean Medicine.

3.2. Perception of the need to implement CSA in the NLE for DKMs

A total of 81.9% of participants agreed with the need to introduce CSA for NLE, 10.8% remained neutral, and 7.3% disagreed (Table 2). After excluding those who were neutral, we compared respondents who agreed or disagreed with the need for CSA, and found that age and years since licensure were associated with the perceived need for CSA. DKMs younger than 40 years and those with <10 years of licensure were more likely to agree that CSA was necessary, whereas respondents older than 50 years and those aged 20 to 29 years were more likely to disagree (Table 3).

Table 2.

Perception of the need for clinical skills assessment in the national licensure examination for doctors of Korean Medicine by respondents’ years since licensure.

Years of licensure	The need for clinical skills assessment
	Strongly agree	Agree	Neutral	Disagree	Strongly disagree
Total	1060	759	239	82	81
<5	373 (35.2)	211 (27.8)	38 (15.9)	9 (11.0)	9 (11.1)
5–9	261 (24.6)	157 (20.7)	35 (14.6)	12 (14.6)	9 (11.1)
10–14	137 (12.9)	122 (16.1)	50 (20.9)	13 (15.9)	13 (16.0)
15–19	113 (10.7)	97 (12.8)	42 (17.6)	9 (11.0)	17 (21.0)
20–24	91 (8.6)	85 (11.2)	34 (14.2)	14 (17.1)	14 (17.3)
25–29	40 (3.8)	34 (4.5)	17 (7.1)	10 (12.2)	9 (11.1)
30–34	26 (2.5)	32 (4.2)	16 (6.7)	10 (12.2)	7 (8.6)
35–39	13 (1.2)	9 (1.2)	5 (2.1)	5 (6.1)	1 (1.2)
≥40	6 (0.6)	12 (1.6)	2 (0.8)	0 (0)	2 (2.5)

Table 3.

Perception of the need for clinical skills assessment in the national licensing examination for doctors of Korean Medicines by respondents’ characteristics.

Classification	Total (n = 1982)	The need for CSA		P-value
		Agree (n = 1819)	Disagree (n = 163)
Age (yr)				<.0001*
20–29	274 (13.8)	262 (14.4)	12 (7.4)
30–39	841 (42.4)	813 (44.7)	28 (17.2)
40–49	504 (25.4)	456 (25.1)	48 (29.4)
50–59	287 (14.5)	226 (12.4)	61 (37.4)
60–69	67 (3.4)	54 (3.0)	13 (8.0)
≥70	9 (0.5)	8 (0.4)	1 (0.6)
Gender				.7465
Male	1400 (70.6)	1288 (70.8)	112 (68.7)
Female	582 (29.4)	531 (29.2)	51 (31.3)
Type of medical institution				.1759
KM clinic	1205 (60.8)	1084 (59.6)	121 (74.2)
KM hospital	414 (20.9)	386 (21.2)	28 (17.2)
Public medical institution	205 (10.3)	197 (10.8)	8 (4.9)
College of KM, Department of Clinical KM	54 (2.7)	54 (3.0)	0
College of KM, Department of Basic KM	27 (1.4)	26 (1.4)	1 (0.6)
Others (research organizations, etc)	77 (3.9)	72 (4.0)	5 (3.1)
Postgraduate training				.1654
Graduated from college of KM	1353 (68.3)	1227 (67.5)	126 (77.3)
Internship completion	128 (6.5)	116 (6.4)	12 (7.4)
Specialist or residency in training	501 (25.3)	476 (26.2)	25 (15.3)
Years of licensure				.0004*
<5	602 (30.4)	584 (32.1)	18 (11.0)
5–9	439 (22.1)	418 (23.0)	21 (12.9)
10–14	285 (14.4)	259 (14.2)	26 (16.0)
15–19	236 (11.9)	210 (11.5)	26 (16.0)
20–29	297 (15.0)	250 (13.7)	47 (28.8)
30–39	103 (5.2)	80 (4.4)	23 (14.1)
≥40	20 (1.0)	18 (1.0)	2 (1.2)

Variables were presented as numbers (N) or frequencies (%). P-values were calculated using the chi-squared test.

CSA = clinical skills assessment, KM = Korean Medicine.

^*P < .05.

The reasons given for not believing that CSA is necessary were grouped into 11 categories: “Don’t feel the need for CSA,” “Traditional written exam is sufficient for assessment,” “Skills can be acquired while practising after licensure,” “The gap between real-world practice and CSA,” “CSA is not appropriate for the NLE for DKMs,” “Difficulties in developing a scoring rubric,” “Difficulty of objective assessment,” “Increase in academic workload,” “The training is already sufficient,” “Universities lack the capacity to teach skills,” and “CSA should be administered in the school curriculum” (Additional File 2, Supplemental Digital Content, https://links.lww.com/MD/Q451).

3.3. What should be assessed in CSA

“Patient care” was the most prioritized (53.3%) competency for CSA (Fig. 2). The weighted average of the priorities was as follows: “Patient care” (2.055), “Clinical skills” (1.733), “Completion of the medical record” (0.983), “Selection of patients who need to be transferred to a higher level of care” (0.798), and “Communication with patients” (0.431). Competencies prioritized by postgraduate training levels showed significant differences (P = .0447). The percentage of specialists or residents in training was higher among those who selected “Selection of patients who need to be transferred to a higher level of care” as their top priority (32.4%), whereas the percentage of graduates without internship experience was higher among the respondents who selected “Completion of the medical record” (72.0%). The type of medical institution and the year of licensure were not associated with the first prioritization of competency (Table 4).

Figure 2.

The priority of competencies that are to be considered for the clinical skills assessment as calculated by the weighted average.

Table 4.

Stratification of competencies to be prioritized for the clinical skills assessment based on the respondents’ characteristics.

Characteristics of respondents	Total (n = 2058)	PC (n = 1097)	CS (n = 493)	MR (n = 207)	CP (n = 82)	ST (n = 179)	P-value
Type of medical institution							.717
Primary clinic	1251 (60.8)	683 (62.3)	298 (60.4)	130 (62.8)	46 (56.1)	94 (52.5)
KM hospital	426 (20.7)	202 (18.4)	106 (21.5)	44 (21.3)	25 (30.5)	49 (27.4)
Public medicine	213 (10.3)	107 (9.8)	52 (10.5)	18 (8.7)	6 (7.3)	30 (16.8)
College-Clinical KM	27 (1.3)	15 (1.4)	8 (1.6)	3 (1.4)	1 (1.2)	0 (0)
College-Basic KM	59 (2.9)	43 (3.9)	10 (2.0)	4 (1.9)	2 (2.4)	0 (0)
Others	82 (4.0)	47 (4.3)	19 (3.9)	8 (3.9)	2 (2.4)	6 (3.4)
Postgraduate training							.0447*
No internship training	1402 (68.1)	747 (68.1)	346 (70.2)	149 (72.0)	53 (64.6)	107 (59.8)
Internship completion	131 (6.4)	65 (5.9)	26 (5.3)	13 (6.3)	13 (15.9)	14 (7.8)
Residency completion or in training	525 (25.5)	285 (26.0)	121 (24.5)	45 (21.7)	16 (19.5)	58 (32.4)
Years of licensure							.4342
<5	622 (30.2)	286 (26.1)	176 (35.7)	57 (27.5)	30 (36.6)	73 (40.8)
5–9	453 (22.0)	223 (20.3)	115 (23.3)	58 (28.0)	15 (18.3)	42 (23.5)
10–19	561 (27.3)	315 (28.7)	119 (24.1)	64 (30.9)	18 (22.0)	45 (25.1)
20–29	301 (14.6)	188 (17.1)	64 (13.0)	24 (11.6)	13 (15.9)	12 (6.7)
≥30	121 (5.9)	85 (7.7)	19 (3.9)	4 (1.9)	6 (7.3)	7 (3.9)

CP = communication with patients, CS = clinical skills, MR = completion of the medical record, PC = patient-care, ST = selection of patients who need to be transferred to a higher level of care.

^*P < .05.

Headache (82.1%), lower back pain (81.8%), chronic abdominal pain/dyspepsia/heartburn (79.1%), dizziness (77.8%), and tingling/paresthesia in the hands and feet (77.8%) were the most commonly requested topics for the patient encounter test (Additional File 3, Supplemental Digital Content, https://links.lww.com/MD/Q451). Respondents with internship or residency experience perceived the following topics to be more essential: headache, dizziness, neck pain, diarrhea, acute abdominal pain, weakness, limb numbness, fever, chest pain, and chest discomfort (Table 5). The differences in perceptions based on the respondents’ institutional affiliations are presented in Additional File 4, Supplemental Digital Content, https://links.lww.com/MD/Q451. Topics including tingling/parasthesia in the hands and feet, shoulder pain, sleep disorders, nausea/vomiting, and knee pain were more likely to be considered essential by DKMs working in public medical institutions, whereas DKMs working in other workplaces were less likely to consider the above mentioned topics as essential (Table 6).

Table 5.

Perception of topics required for patient encounter and procedural skills tests by internship experience.

Rank	Topics	Response	Total (n = 2058)	GP (n = 1402)	Internship completion† (n = 656)	P-value
Patient encounter test
1	Headache	P	1690 (82.1)	1130 (80.6)	560 (85.4)	.0085*
		N	368 (17.9)	272 (19.4)	96 (14.6)
4	Dizziness	P	1601 (77.8)	1066 (76.0)	535 (81.6)	.0050*
		N	457 (22.2)	336 (24.0)	121 (18.4)
6	Neck pain	P	1550 (75.3)	1037 (74.0)	513 (78.2)	.0378*
		N	508 (24.7)	365 (26.0)	143 (21.8)
8	Diarrhea	P	1532 (74.4)	1023 (73.0)	509 (77.6)	.0250*
		N	526 (25.6)	379 (27.0)	147 (22.4)
9	Acute abdominal pain	P	1524 (74.1)	1004 (71.6)	520 (79.3)	.0002*
		N	534 (25.9)	398 (28.4)	136 (20.7)
14	Weakness/numbness in limbs	P	1475 (71.7)	978 (69.8)	497 (75.8)	<.0001*
		N	583 (28.3)	424 (30.2)	159 (24.2)
18	Fever	P	1449 (70.4)	968 (69.0)	481 (73.3)	.0475*
		N	609 (29.6)	434 (31.0)	175 (26.7)
19	Chest pain/discomfort	P	1438 (69.9)	956 (68.2)	482 (73.5)	.0148*
		N	620 (30.1)	446 (31.8)	174 (26.5)
Procedural skills test
5	Motor examination	P	1518 (73.8)	1010 (72.0)	508 (77.4)	.0095*
		N	540 (26.2)	392 (28.0)	148 (22.6)
7	Completion of MR and certificate	P	1446 (70.3)	960 (68.5)	486 (74.1)	.0095*
		N	612 (29.7)	442 (31.5)	170 (25.9)
11	Sensory examination	P	1335 (64.9)	882 (62.9)	453 (69.1)	.0072*
		N	723 (35.1)	520 (37.1)	203 (30.9)
12	Reflex test	P	1295 (62.9)	833 (59.4)	462 (70.4)	<.0001*
		N	763 (37.1)	569 (40.6)	194 (29.6)
13	Wound dressing	P	1110 (53.9)	720 (51.4)	390 (59.5)	.0006*
		N	948 (46.1)	682 (48.6)	266 (40.5)
15	CPR and defibrillation	P	1068 (51.9)	675 (48.1)	393 (59.9)	<.0001*
		N	990 (48.1)	727 (51.9)	263 (40.1)
16	Chest/lung examination	P	1035 (50.3)	684 (48.8)	351 (53.5)	.0460*
		N	1023 (49.7)	718 (51.2)	305 (46.5)
17	Needle-embedding therapy	P	942 (45.8)	658 (46.9)	284 (43.3)	.1224
		N	1116 (54.2)	744 (53.1)	372 (56.7)
18	Imaging interpretation	P	940 (45.7)	611 (43.6)	329 (50.2)	.0053*
		N	1118 (54.3)	791 (56.4)	327 (49.8)
19	Abdominal examination	P	926 (45.0)	610 (43.5)	316 (48.2)	.0476*
		N	1132 (55.0)	792 (56.5)	340 (51.8)
20	Examination of the cranial nerves	P	868 (42.2)	514 (36.7)	354 (54.0)	<.0001*
		N	1190 (57.8)	888 (63.3)	302 (46.0)

GP, a general practitioner who graduated from a college of KM and doesn’t have internship experience.

P-values were calculated using the chi-square test.

CPR = cardiopulmonary resuscitation, KM = Korean Medicine, MR = medical record, N = negative, P = positive.

^†Internship completion includes respondents who completed internship (without residency), in residency training, and specialist (who completed residency).

^*P < .05.

Table 6.

Perception of topics required for patient encounter and procedural skills tests by medical institution type.

Rank	Topics	Response	Total (n = 2058)	PMI (n = 213)	Not PMI† (n = 1845)	P-value
Patient encounter test
2	Lower back pain	P	1683 (81.8)	187 (87.8)	1496 (81.1)	.0148*
		N	375 (18.2)	26 (12.2)	349 (18.9)
3	Chronic abdominal pain/dyspepsia/heartburn	P	1628 (79.1)	180 (84.5)	1448 (78.5)	.0407*
		N	430 (20.9)	33 (15.5)	397 (21.5)
5	Tingling/Paresthesia in hands and feet	P	1586 (77.1)	180 (84.5)	1406 (76.2)	.0058*
		N	472 (22.9)	33 (15.5)	439 (23.8)
8	Diarrhea	P	1532 (74.4)	182 (85.4)	1350 (73.2)	<.0001*
		N	526 (25.6)	31 (14.6)	495 (26.8)
9	Acute abdominal pain	P	1524 (74.1)	177 (83.1)	1347 (73.0)	.0012*
		N	534 (25.9)	36 (16.9)	498 (27.0)
11	Shoulder pain	P	1510 (73.4)	172 (80.8)	1338 (72.5)	.0110*
		N	548 (26.6)	41 (19.2)	507 (27.5)
12	Sleep disorder	P	1500 (72.9)	172 (80.8)	1328 (72.0)	.0056*
		N	558 (27.1)	41 (19.2)	517 (28.0)
14	Weakness/numbness in limbs	P	1475 (71.7)	175 (82.2)	1300 (70.5)	.0003*
		N	583 (28.3)	38 (17.8)	545 (29.5)
15	Nausea/vomiting	P	1469 (71.4)	166 (77.9)	1303 (70.6)	.0251*
		N	589 (28.6)	47 (22.1)	542 (29.4)
16	Constipation	P	1458 (70.8)	166 (77.9)	1292 (70.0)	.0168*
		N	600 (29.2)	47 (22.1)	553 (30.0)
17	Knee pain	P	1454 (70.7)	168 (78.9)	1286 (69.7)	.0053*
		N	604 (29.3)	45 (21.1)	559 (30.3)
Procedural skills test
5	Motor examination	P	1518 (73.8)	177 (83.1)	1341 (72.7)	.0010*
		N	540 (26.2)	36 (16.9)	504 (27.3)
9	Tongue diagnosis	P	1406 (68.3)	115 (54.0)	1291 (70.0)	<.0001*
		N	652 (31.7)	98 (46.0)	554 (30.0)
10	Pulse diagnosis	P	1370 (66.6)	98 (46.0)	1272 (68.9)	<.0001*
		N	688 (33.4)	115 (54.0)	573 (31.1)
11	Sensory examination	P	1335 (64.9)	173 (81.2)	1162 (63.0)	<.0001*
		N	723 (35.1)	40 (18.8)	683 (37.0)
12	Reflex test	P	1295 (62.9)	164 (77.0)	1131 (61.3)	<.0001*
		N	763 (37.1)	49 (23.0)	714 (38.7)
13	Wound dressing	P	1110 (53.9)	147 (69.0)	963 (52.2)	<.0001*
		N	948 (46.1)	66 (31.0)	882 (47.8)
15	CPR and defibrillation	P	1068 (51.9)	137 (64.3)	931 (50.5)	.0001*
		N	990 (48.1)	76 (35.7)	914 (49.5)
16	Chest/lung examination	P	1035 (50.3)	120 (56.3)	915 (49.6)	.0703
		N	1023 (49.7)	93 (43.7)	930 (50.4)
17	Needle-embedding therapy	P	942 (45.8)	119 (55.9)	823 (44.6)	.0022*
		N	1116 (54.2)	94 (44.1)	1022 (55.4)
18	Imaging interpretation	P	940 (45.7)	121 (56.8)	819 (44.4)	.0006*
		N	1118 (54.3)	92 (43.2)	1026 (55.6)
19	Abdominal examination	P	926 (45.0)	112 (52.6)	814 (44.1)	.0199*
		N	1132 (55.0)	101 (47.4)	1031 (55.9)
20	Examination of the cranial nerves	P	868 (42.2)	121 (56.8)	747 (40.5)	<.0001*
		N	1190 (57.8)	92 (43.2)	1098 (59.5)

P-values were calculated using the Fisher’s exact test.

CPR = cardiopulmonary resuscitation, KM = Korean Medicine, N = negative, P = positive, PMI = public medical institution.

^†Including KM clinic, KM hospital, College of KM, and others.

^*P < .05.

Acupuncture was the most common topic in the procedural skills tests (82.1%). More than 70% of the respondents perceived the need for abdominal examinations in KM, pharmacopuncture, wet cupping, motor examinations, Chuna manipulation, and completion of medical records and certificates. Furthermore, the unique diagnoses of KM, tongue, and pulse were among the top ten skills (Additional File 5, Supplemental Digital Content, https://links.lww.com/MD/Q451). The procedural skills test items perceived as more essential by respondents with internship experience included motor examinations, completion of medical records and certificates, sensory examinations, reflex tests, wound dressing, cardiopulmonary resuscitation and defibrillation, and chest/lung examinations (Table 5). Tongue and pulse diagnoses were more likely to be selected by DKMs than public medical institutions, whereas most examinations were more likely to be selected by DKMs in public medical institutions and hospitals (Table 6 and Additional File 6, Supplemental Digital Content, https://links.lww.com/MD/Q451).

Open-ended responses to items that should be assessed in the CSA were categorized as “Clinical presentations,” “Diseases,” “Disease name or pattern/syndrome of Korean Medicine (KM),” “History assessment,” “Physical examination,” “Clinical examination and diagnosis,” “Procedural skills,” “Treatment and management,” and “Others.” Items that were mentioned more than once included “cold hands and feet” and “growth delay” for clinical presentations, and “Suture,” “Chuna,” “Dressing,” and “Cardiopulmonary resuscitation” for procedural skills (Additional File 7, Supplemental Digital Content, https://links.lww.com/MD/Q451).

3.4. Suggestions for introducing CSA

The 2058 respondents who did not oppose the introduction of CSA believed that the appropriate time for the introduction of the CSA is 2025 (64.5%), 2027 (11.9%), 2026 (10.7%), 2029 (8.0%), and 2028 (4.8%). Perceived need to introduce CSA was related to the appropriate time for its introduction (P < .0001). Respondents who chose 2025 were more likely to strongly agree that CSA was needed, whereas those who chose 2029 were more likely to be neutral regarding CSA (Additional File 8, Supplemental Digital Content, https://links.lww.com/MD/Q451).

The open-ended responses on what to suggest for the practical exam were categorized as “Change of KM education,” “Preparation to implement CSA,” “Authenticity of assessment,” “Urgent need to implement CSA,” “Ethics,” “Test methods,” and “Criteria for pass/fail.” Comments included the need for CSA to be “relevant to real-world clinical practice” and to be “thorough,” prepared for “objective scoring,” and for “scoring based on standardized measures of performance” (Additional File 9, Supplemental Digital Content, https://links.lww.com/MD/Q451).

4. Discussion

Although current licensure examinations for DKMs have focused on knowledge rather than assessing various clinical competencies, this study provides insights into the necessity of CSA and what should be assessed considering the tasks of DKMs. We identified the need to include CSA, such as patient encounters and procedural skills tests, to assess comprehensive patient-care and the performance of clinical skills in NLE for DKMs. We present various opinions on DKMs as well as their suggestions and concerns regarding CSA.

The impact of introducing CSA in licensure examinations has been demonstrated in previous studies. The US Medical Licensure Examination currently assesses clinical skills in standardized patients in step 2; the introduction of the clinical skills exam is thought to have strengthened the assessment of practice-related competencies.^[15] CSA also focuses on the clinical practice of traditional medicine. In China, licensure examinations for traditional Chinese doctors and integrated modern traditional doctors include basic physical examination, diagnosis, and treatment skills.^[16] Recently, a content-valid and reliable OSCE was developed in Japan, where MDs currently use Kampo medicine, to evaluate their clinical skills of Kampo medicine.^[17]

Although the response rate was estimated to be 8.7% (2221 of approximately 25,500), which may limit the representativeness of the findings, it was nearly twice as high as that of other recent national surveys of DKMs,^[14,18,19] indicating heightened interest among DKMs in the introduction of CSA. Our survey found that most DKMs believed CSA was necessary, with recently licenced DKMs being more likely to agree. As indicated in the open-ended responses, since many DKMs perform various KM therapeutic techniques directly on patients without undergoing internship training in a hospital, some DKMs believe the introduction of CSA is “urgent” and that “CSA for KM procedural skills needed in real clinical practice is essential.” This is consistent with previous survey results, which showed that the introduction of CSA was the most frequently selected improvement to the licensure examination by DKMs within 5 years of licensure, whereas it was ranked fourth by DKMs who had been licenced for more than 5 years.^[14] Considering that CSA has been implemented since 2011 in KM education^[11] and that studies on the implementation of CPX and OSCE in colleges of KM have increased since 2018,^[20] most DKMs within 5 years of licensure would have encountered CSA in their undergraduate curriculum. Thus, it can be inferred that individuals who have encountered CSA in the curriculum have a more favorable view of its implementation in NLE. In contrast, the respondents who opposed the introduction of CSA argued that it is not necessary, that the existing written exam is sufficient, and that “clinical skills improve quickly with clinical practice” after graduation. Nonetheless, the impact of CSA on education and students as well as on the assessment itself should be considered. Although valid evidence of the impact of large-scale examinations on the quality of care remains controversial,^[21] the NLE for DKMs can have a significant impact on KM education, as it qualifies individuals to practice KM.

Among the various competencies of DKMs,^[10] it is necessary to define the core competencies to be assessed in NLE and design the NLE accordingly. The competencies that respondents perceived should be prioritized for CSA were patient-care, clinical skills, and completion of the medical record, which are very similar to the competencies assessed in USMLE Step 2.^[15,22] DKMs who selected the completion of medical records as their first priority were mostly graduates with no internship experience. In KM hospitals, relatively detailed charting is emphasized, and charting of medical records is requested during training. Graduates who have not undergone internship training may experience charting difficulties. The highly selected “Completion of medical record and certificate” item for the procedural skills test in this survey indicates that “clinical management” competency of DKMs needs to be emphasized. A relatively high percentage of DKMs with internship experience who selected “Selection of patients who need to be transferred to a higher level of care” as the most important competency showed that they perceived the importance of dealing with emergencies from their experience. Furthermore, the main KM treatments and examinations included in the top 10 procedural skills items were deemed necessary regardless of internship experience. The items that ranked low were examinations, clinical tests, and treatments used in hospitals. They were significantly more likely to be selected by trained DKMs currently working in public medical institutions and hospitals. These results indicate that DKM experience has a significant impact on the educational content that they value.

The task and practice characteristics of KM should be considered when developing authentic CSA that can help examinees perform their duties after graduation. According to the Korean Ministry of Health and Welfare’s 2022 Survey on the Utilization of KM, “musculoskeletal disorders” accounted for 70.1% of the reasons for using outpatient KM services.^[23] However, in our survey, the top 10 items required for the patient encounter test included not only musculoskeletal disorders but also clinical presentations of various symptoms, such as headache, chronic abdominal pain, dizziness, diarrhea, and dysmenorrhea. Therefore, while musculoskeletal disorders are fundamentally important in CSA for DKMs, it is important to strengthen competency in a variety of clinical presentations commonly encountered in primary care. The treatments used in outpatient KM clinics were “acupuncture” (92.2%), “cupping” (51.5%), “moxibustion” (51.0%), and “pharmacopuncture” (29.9%),^[23] all of which were highly ranked as necessary topics for the procedural skills test. The results of this survey can be considered valid because the items that the DKMs reported as necessary reflected their actual practice.

Similar to MDs, DKMs use the International Classification of Diseases for their diagnosis. Furthermore, a unique KM pattern identification diagnosis is utilized in practice, which serves as an important basis for treatment plans. Therefore, patient encounter tests for DKMs should include both disease diagnosis and the identification of KM patterns. In this survey, we suggested clinical presentations as topics for the patient encounter test, to include pattern identification in the diagnosis of clinical presentations. However, some respondents mentioned that pattern identification should be included in CSA because of the misunderstanding that KM pattern identification diagnosis would not be included in CSA.

Nonetheless, before CSA for DKMs can be implemented in NLE, many steps must be completed.^[24] Existing studies on the CPX and OSCE in KM are mainly case reports of individual classes aimed at developing and implementing modules, and evaluation of the test, such as validity and reliability analyses, is limited.^[20,25] The survey also revealed several concerns about the difficulty of developing valid and reliable tests, which must be addressed during development.^[26] Although more than 60% of the respondents who did not oppose the introduction of CSA advocated for its implementation in 2025, responses from clinical professors responsible for CSA preparation and related education were evenly divided between 2025 and 2029 as the appropriate time to introduce CSA. This indicates that stakeholders recognize the difficulty of implementing CSA in the short term.

The method of assessing DKMs should be structured as a blueprint for the systematic evaluation of their competencies to help prepare for a future introduction of CSA.^[24] Systematic methodologies, such as multifaceted analysis, should be established and implemented to develop valid and reliable items.^[25] Cooperation among professors in nationwide KM colleges is necessary. The competencies and topics to be prioritized in CSA can be found in the results of our survey. Furthermore, the KM curriculum should be improved by focusing on comprehensive patient-care competencies. Some opinions pointed out the problems of existing education and the lack of capacity to teach clinical skills in KM colleges as reasons for opposing CSA. Further in-depth analysis is required to advance KM education through the introduction of CSA.

This study has some limitations. First, some respondents did not have basic knowledge and experience of CSA, as patient encounters or procedural skills tests have only recently been implemented in KM colleges. CSA, such as the OSCE and CPX, have been included in the KM curriculum only since the early 2010s.^[11] Graduates who completed their education at KM colleges before the 2010s may not have experienced CSA, which may have affected their responses. Second, the survey was conducted online, which could have led to respondent selection bias. Third, as this was a cross-sectional survey, only associations could be established, and causal inferences cannot be drawn. Fourth, the relatively low response rate and the limited proportion of DKM’s included may reduce the external validity of the finding, and caution is warranted when generalizing the results to the entire population of DKMs. Finally, multiple subgroup analysis was performed, but no correction for multiple testing was applied, which may increase the risk of false positive finding. Nonetheless, to the best of our knowledge, this is the first nationwide study to investigate DKM perceptions regarding the introduction of CSA, and a relatively large number of responses were obtained. Furthermore, we found differences in the perception of CSA depending on the sociodemographic characteristics of DKMs, such as postgraduate training and years since licence acquisition. The survey results suggest core competencies and items that should be prioritized in CSA, reflecting actual practice based on most responses from currently practising DKMs. These findings should be considered not only in planning future CSA but also in the KM curriculum. Furthermore, the identified and presented concerns regarding the introduction of CSA and the tasks to be performed in the future will contribute to improving the feasibility of CSA for NLE.

5. Conclusion

The competencies of DKMs that are essential in actual clinical practice should be assessed by introducing CSA into NLE for DKMs. Our study highlights the high demand for CSA among DKMs, with 81.9% agreeing that CSA should be introduced in NLE. However, this study focused on survey-based perceptions, and further research is required to validate these findings through practical assessments. The clinical presentation of various systems encountered in primary care, as well as the diagnostic and therapeutic skills commonly used by DKMs, should be prioritized in CSA. Future studies should explore the implementation challenges and long-term impacts of CSA on DKM education and practice. This survey will be useful for planning a specific blueprint of CSA for DKMs and preparing a survey regarding NLE for global healthcare professionals. By addressing these considerations, this study contributes to the development of a more comprehensive and competency-based licensure examination of DKMs.

Author contributions

Conceptualization: Aram Jeong, Sanghoon Lee, Chungsik Cho, Sangwoo Shin, Hye-Yoon Lee.

Data curation: Eunbyul Cho, Dong-Hyeon Kim.

Formal analysis: Dong-Hyeon Kim.

Investigation: Aram Jeong, Hye-Yoon Lee.

Methodology: Chan-Young Kwon, Chungsik Cho, Sangwoo Shin, Min Hwangbo.

Writing – original draft: Eunbyul Cho.

Writing – review & editing: Aram Jeong, Chan-Young Kwon, Hye-Yoon Lee.