Clinical-Performance Remediation Program for Dyscompetent Medical Students

Abstract

Purpose:

Medical schools endeavor to ensure that students are competent with regard to clinical skills. Skills remediation is implemented in cases of poor clinical performance examination (CPX) grades, although little is known about the effectiveness of such techniques. In this study, we examined the effectiveness of a remediation program that was designed to improve the clinical performance of medical students.

Methods:

A 6-week remediation program, administered jointly by Seoul National University College of Medicine’s Departments of Internal Medicine (IM) and Family Medicine (FM), was initiated. The program was divided into 2 parts: 3 weeks each of IM classes that were run by specialists in various fields and FM classes that were conducted by a chief resident. Twenty-three students were required to undergo remediation after posting poor scores on 2 sessions of a CPX. On completion of the remediation program, the students’ clinical performance was re-evaluated, and the changes in clinical performance scores were analyzed.

Results:

After the remediation program, the students’ total scores and scores on history taking, physical examination, physician’s manner, and physician-patient interaction improved significantly. However, patient education did not improve. Most students found the remediation program to be instructive and helpful in preparation for the CPX. They were more satisfied with the chief resident’s serial tutoring than with specialists’ tutoring sessions.

Conclusion:

The remediation program improves clinical performance. Continued development and implementation of this program will help failing students be competent physicians.

INTRODUCTION

Making students competent in clinical skills as well as in clinical knowledge is the primary goal of medical schools. Since the National Medical Licensing Examination Board (NMLEB)’s introduction of clinical skill assessment as an independent examination for MD qualification in 2009, the issue of competence has become more important than ever [1,2].

Korea-wide clinical skills examination data for 2009 to 2010 reveal a 5% failure rate. The clinical skills assessment examination has six clinical performance stations and six objectively structured clinical examination stations. At Seoul National University College of Medicine, the clinical performance examination (CPX)component is the most commonly failed. It includes patient data gathering, clinical reasoning, and physicianpatient interaction.

Recent trends in medical education emphasize medical student performance rather than curriculum content [3].Whereas most students come to be competent in clinical skills, some initially fail to attain the expected standard,and require remediation. In fact, any student that fails any part of the comprehensive assessment needs to receive remediation [4]. However, most medical schools,regarding skills and knowledge improvement as matters for personal resolution, have not paid adequate attention to this issue [5]. Furthermore, and relatedly, teaching staff are unfamiliar with remediation methodologies. In fact, faculty rarely examine clinical skills or provide feedback to students during clerkships, lack confidence in their abilities to identify incompetent students, and feel reluctant to fail them [6,7,8,9].

Failing students have either cognitive skill (history taking, physical examination, and clinical reasoning) ornon-cognitive skill (physician-patient interaction) problems or both. To improve the clinical competencies of poor performing students, tailored education programs are required [10,11,12,13]. However, the medical education literature is relatively lacking in effective methodological strategies or verifications [14,15,16].

As the remediation program for at-risk medical students is very important and an urgent issue, we have made 6 weeks of clinical performance remediation program and examined the efficacy of the program.

SUBJECTS AND METHODS

The CPX remedial course was designed to help students develop the clinical reasoning, communication,history taking, and physical examination skills required for the MD qualification. During this 6-week course,students undergo 1:1 tutoring sessions, re-examination,and feedback from a standardized patient (SP).

1. Subjects

All 147 fourth (final)-year students at Seoul National University College of Medicine underwent a two-session CPX in March and May of 2011. All of them were scheduled to subsequently undergo a clinical skills examination at the end of 2011. Both exams included eight stations of 10-minute SP encounters, each station representing a clinical case including acute, chronic, and psychological problems. There were no same cases from the first to the second examination. Performances were scored by trained SPs using a checklist comprising the following six evaluation categories: overall assessment,history taking, physical examination, physician’s manner,patient education, and physician-patient interaction. The history taking skills were scored dichotomously as 0 or 1 (indicating incorrect or correct performance, respectively),and the physical exam skills were scored on a three-point scale (0, 0.5, and 1; 0.5 for partial credit).On the basis of the first and second CPX scores, 18 students, who had failed at more than three stations in both of the exams, and who showed deficits in data gathering (such as history taking and physical examination),clinical reasoning, and physician-patient interaction,were judged to require remediation.

2. Programs

The remediation program was designed and administered jointly by the Departments of Internal Medicine(IM) and Family Medicine (FM). The Departments of IM and FM along with the Office of Medical Education discussed the aims of the program and determined the essential clinical presentation areas and specific classes.The program’s 6-week duration was divided into twoparts: 3 weeks of IM, and 3 weeks of FM. Half of the students completed the program in the IM-FM order,and the others in the FM-IM order. During the respective 3-week periods, the IM classes were conducted by professors of various specialties such as gastroenterology and nephrology, whereas the FM classes were taught by one chief resident.

The remediation program was designed with three parts: diagnosis of failing students, learning activities,and re-examination. For diagnosis of failing students,we examined student’s score of each component(history taking, physical examination, patient-physician interaction and so on), digital videodisc (DVD) during SP encounter, feedback from SP, grade point average (GPA)and tried to find out the cause of poor performance. The main learning activities were interactive lectures, making a schema and checklist, role play, SP encounter and DVD review of various cases, and observation in outpatient clinic. In the Department of IM, the students were coached by professors, expert in eachclinical presentation. In the Department of FM, one chief resident diagnosed each of the students’ deficits and leaded all of the learning activities. Upon completion of the 6-week remediation program, the students’ clinical performances were re-evaluated.

3. Evaluation

The students were asked to complete a postremediation questionnaire, which included the following items: 1) The remedial course was helpful in improving clinical skills; 2) Rate each of these course activities: a) DVD review of CPX performance examination, b)making a schema of each clinical presentation, c) making a checklist for each clinical presentation, d) role play, e) SP encounter, f) mini-CPX examination, g) lecture, h) observation in outpatient clinic; 3) The program in IM was satisfactory; 4) The program in FM was satisfactory; 5) Comments or suggestions regarding the remediation program. Students were asked to respondon a five-point scale (5=strongly agree; 1=strongly disagree) or 100% scale (100=strongly agree; 0=strongly disagree).

4. Data analysis

A statistical analysis was performed using SPSS version 18.0 (SPSS, Chicago, USA). The changes in CPX scores with or without remediation were analyzed by ANCOVA. A p-value <0.05 was considered to indicate statistically significant differences. The students’ and tutors’ evaluations of the remediation program were analyzed both quantitatively and qualitatively.

RESULTS

1. Improvement of students’ clinical performances following remedial course

A total of 147 students applied for the third CPX examination. Among them, 18 students participated in the remedial course (the with-remediation group), and 129 students (the without-remediation group) did not. The CPX re-examination comprised eight stations representing clinical presentations not encountered during the prior CPX exam. In the results, it was evident that in the 2-month period between the second and third re-exams, most of the students in both groups had studied hard and achieved improved performances. The total scores and those for physical examination, physician’s manner and patient education were higher than on the second exam. The with-remediation group achieved improvements in total score, history taking, physical examination, physician’s manner, and physician-patient interaction. As for the reason the patient education score was not improved statistically significantly, a clueprobably lies in the fact that the score was also improved in the without-remediation group (Table 1). A compari on of the CPX score changes between the two groups revealed that the with-remediation group had made better gains in almost categories. All of these results show that the CPX remediation program effectively improved the performances of under-achieving students.

Table 1.

Clinical Performance Examination Scores

	Students without remediation	Students with remediation	p-value
No.	129	18
Sex, M:F	85:44	15:3	0.181
Total score			0.000
2nd exam	69.33 (5.08)	63.77 (6.01)
3rd exam	70.18 (3.93)	69.68 (6.07)
History taking			0.009
2nd exam	75.04 (7.03)	68.13 (6.67)
3rd exam	72.67 (5.54)	70.63 (7.53)
Physical examination			0.000
2nd exam	53.68 (10.96)	47.01 (14.45)
3rd exam	65.43 (9.02)	70.84 (11.63)
Physician’s manner			0.016
2nd exam	82.27 (6.00)	78.73 (6.92)
3rd exam	82.85 (6.48)	83.76 (5.90)
Patient education			0.268
2nd exam	53.26 (14.99)	48.70 (20.63)
3rd exam	56.66 (15.49)	57.19 (15.06)
Physician-patient Interaction			0.011
2nd exam	65.40 (4.06)	61.23 (4.32)
3rd exam	64.75 (4.18)	63.06 (4.17)

Each score is an average (standard deviation).

2. Program evaluation

The average response scores of the 19 students who completed the post-program questionnaire are listed in Table 2. General satisfaction with the program wasexpressed. With regard to the specific remediation activities, SP encounter was rated the best, and observation in outpatient clinic the worst. The student comments and suggestions regarding the remediation program were as follows:

Table 2.

Program Evaluation Questionnaire

	Mean	SDa)
The remedial course was helpful in improving clinical performance skills	4.09	0.66
Rate each of the activities in the course
DVD review of my CPX performance examination	66.3	31.3
Making a schema of each clinical presentation	66.2	26.4
Making a checklist for each clinical presentation	63	25.3
Role play	89.7	13.6
SP encounter	93.5	11.9
Mini-CPX examination	92.3	14.3
Lecture	66.2	20
Observation in outpatient clinic	38.4	26
The IM program was satisfactory	3.26	1.1
The FM program was satisfactory	4.42	0.77

SD: Standard deviation, DVD: Digital video disc, CPX: Clinical performance examination, SP: Standardized patient, IM: Internal Medicine, FM: Family Medicine.

^a)Mean and SD derived from 5-point Likert scale: 1=strongly disagree, 2=disagree, 3=neither disagree nor agree, 4=agree, 5=strongly agree.

• 1) I came to realize what my problem was in physician-patient interaction, and had the chance to fix it.

• 2) I appreciate the tutors’ interest!

• 3) I was content with the program. Thank you for everyone’s effort.

• 4) I would have preferred a more compact and shorter program.

• 5) Identifying my weak points was of practical benefit.

• 6) One of the tutors was unfamiliar with the CPX, and I was not confident of his feedback.

DISCUSSION

The results of this study demonstrate that Seoul National University College of Medicine’s remediation program was useful for improving students’ specific clinical skills. And, in the present case at least, it was found that tutoring by one chief resident for all of the sections was more satisfactory than that by per-section specialists.

Among the categories of CPX scores, history taking,physical examination, and patient education are cognitive areas requiring clinical knowledge, whereas physician’s manner and physician-patient interaction are noncognitive areas requiring interpersonal and communication skills. The first step of our remediation program is the diagnosis of deficits; it was found that most students had multiple problem-solving deficits in areas such as knowledge, data gathering, clinical reasoning, and communication[17,18].

The present study showed that over a short period,non-cognitive areas and cognitive areas could be improved equally well.

Initially, most of the students were hostile to having been judged a failure, and were unwilling to participate in remediation. However, as the program ensued, they eagerly took part in the classes and, as the program evaluation indicated, were mostly satisfied with its conduct and results.

Among the program evaluation findings, we focused on that which indicated greater student satisfaction with FM tutoring than with IM. Although the IM tutors were all professors and experts in their respective fields, most of them did not grasp the nature or methods of the CPX; thus, they were not able to function effectively as tutors.The chief resident of FM, by contrast, had experience in the CPX, and was also in the process of preparing the CPX component of the FM board exam. This chief resident, moreover, was always with students during the course, whereas the IM professors met students only during the scheduled, 1- or 2-hour class. Further, the various FM classes were harmonized, and so the program could be consistent from beginning to end; the IM program, however, with its differing classes and professor competencies and qualities, and so was not unified. These results suggest that the general physician,which is to say the chief resident of FM, was a better tutor than the IM experts.

Perhaps not surprisingly, familiarity might have had a significant impact on student satisfaction with the FM classes during program. Such familiarity could be developed mainly by being together, not because the chief resident tutor was closer to the students’ age than were the IM professor tutors, or because he was a general physician under training. Being with the students all day long, throughout the course, made the tutor knowthe students on an individual basis, check individual student’s outcome daily, and adjust the program as required.

Actually, to evaluate or compare the efficacy of the remediation program, students either in IM or FM group were standardized and allocated into either group and their exam scores after the remedial course are needed.But placing students in either group might be unfair.

Examination after each program (3 weeks) could be another option. However, the clinical presentation taught at IM or FM was different. Therefore, exact comparison of efficacy of each program was not easy. For these reasons, we just surveyed students’ satisfaction using focus-group interview and questionnaire.

Although the fact that tutoring and remediation activities were not standardized could be the weak point of our program, it has its strengths. First, the students were allowed sufficient time to practice clinical skills under a tutor’s supervision. Second, the students were provided the opportunity to review their behaviors and performances using DVD recordings [1,19]. In fact, students demonstrated considerable insight into both the positive and negative aspects of their encounters with SPs. Third, students could receive feedback from SPs and tutors repetitively. Overall, the remediation program showed students how to acquire clinical skills and prepare for the CPX.

In conclusion, remediation program improved clinical performance of medical students. Continued development and identifying the effective component of remedial course is needed.