A comparative study of the effects of multimedia training materials on mini CEX scores of internal medicine residents in Isfahan University of Medical Sciences

Abstract

BACKGROUND:

Role of multimedia training materials on Mini-CEX scores of internal medicine residents. We aimed to assess the effect of multi multimedia training materials on Mini-CEX scores of internal medicine residents of Isfahan University of Medical Sciences.

SETTINGS AND DESIGN:

A quasi-experimental action research study on 1^st, 2^nd, and 3^rd-year internal medicine residents were implemented.

MATERIALS AND METHODS:

The Mini-CEX test measures students' performance in six core skills necessary for medical practice. Mini-CEX scores of 135 internal medicine residents in 2017–2018 were compared before and after the training with prepared multimedia materials. We used repeated measured ANOVA and Mann–Whitney U test to compare the distribution of Mini-CEX scores across corresponding groups. Analysis was done using the SPSS software version 23 (IBM SPSS Statistics for Windows. Armonk, NY, USA: IBM Corp).

RESULTS:

The median Mini-CEX score (IQR) of students in preintervention and postintervention groups were 16.14 (5.19) and 19.62 (3.13), respectively. Findings of this study showed a significant increase in mini-CEX scores of the groups who used the multimedia learning material compared to those who did not use it (P < 0.001).

CONCLUSIONS:

Multimedia learning resources demonstrated a promising influence on internal residents' mini-CEX scores in this study. They demonstrate significantly greater performance after using multimedia learning materials compared to their same-year residents who did not benefit from it. This demonstrates the favorable effect of multimedia on the acquisition of practical skills such as obtaining a history or performing a physical examination.

Introduction

Education is an essential component of the health system and is considered a mainstay for training health-care workers and works as a means of assisting the community's health by guiding individuals and society toward the preservation and promotion of health.[1] Among different fields of education, medical specialty education play a distinctive function in the community health system, as it directly influences the care provided in a health system. Hence, the authorities place a premium on the development and enhancement of the quality of medical education training. Training medical students and residents regarding physical examination skills have been identified as a critical component of medical education by the country's health authorities. Medical education officials are concerned about a lack of sufficient training in this field,[2] which may result in irreversible harm to patients. In addition, the information gathered throughout the history taking and physical examination processes, together with laboratory investigations, is beneficial for the physician's clinical judgment.[3] Effective acquisition of these abilities helps medical trainees to develop the necessary skills for obtaining a history and doing a physical examination, a primary aim of medical education that most of the students were concerned about.[4]

A proven method for medical students and assistants to learn is through the use of movies and computers as a learning aid.[5,6] In medicine, and particularly in the field of physical examinations, the use of instructional multimedia is a valuable teaching resource for learning basic science and clinical medicine.[7] Meanwhile, like any other educational system, tests are recognized as one of the most influential and dynamic components of the training process, and so enhancing the quality of tests, and standardizing tests are the most effective strategies.[8,9] Thus, postgraduate medical education involving more advanced training addressing abilities need to accompany new methods of assessments that effectively could measure the success of educational programs. Objective Structured Clinical Examination and Mini-CEX are among new physical examination methods.[10,11]

Given the existence of multiple educational sources for learning physical examination, which can confuse medical trainees and the need for a unified source as a prerequisite for standard examination, we sought to evaluate the effect of unified standard multimedia-based teaching on the skills of medical trainees in this study. Hence, we decided to compile all accessible educational materials based on existing references regard physical examinations and create a comprehensive multimedia educational resource for history taking and physical examination. Finally, we compared Mini-CEX tests scores of medical trainees to determine the effect of educational multimedia on their learning.

Materials and Methods

Study participants and sampling

This study was a quasi-experimental action research study addressing the effect of using multimedia learning materials on the skill performance of medical trainees. We recruited 135 internal medicine residents of different entrance years by the census. They were all the internal medicine residents who entered the residency program in 2016–2018 [Figure 1].

Figure 1.

CONSORT diagram for participants enrollment in the study

Data collection tool and technique

In Iran, the internal medicine residency program lasts 4 years, during which all the residents should pass certain examinations for entering the next year. Tests that evaluate the skill performance of the residents were of importance, given the critical role of skills for clinical decision making. Mini-CEX (Mini-Clinical Evaluation Exercise) examination is intended for the assessment of core clinical skills in any clinical rotation. The Mini-CEX is a 10–20-min direct observation assessment of a trainee-patient encounter.[12] As the consistency of mini-CEX scores increased in multiple measurements,[13] trainees were urged to participate in mini-CEX bi-monthly in each clinical rotation. The mini-CEX validity and reliability were previously confirmed.[14,15,16]

The Mini-CEX test measures students' performance in six domains including the medical interview, physical examination, professionalism, clinical judgment, data organization/sufficiency, and counseling skills. These six abilities are graded on a 0–9 scale (0: Absence of the desired behavior, 1–3: Below the expectations for the level of training, 4–6: Meets the expectations, and 7–9: Above the expectations for the level of training). The sum of the checklist's scores was considered the student's performance score.[17]

Study design and setting

Educational materials for physical examination were collected and prepared in the compact disk (CD) format after a thorough search in all relevant clinical resources suggested by national authorities. In each CD, there was a complete course for training residents regarding all the fields that were going to be assessed during a mini-CEX examination. The multimedia training material was provided by the joint contribution of clinical professors and medical education specialists.

The study was a quasi-experimental before-after trial. Participants of this study were monitored for 1 year before any intervention. They – including 1^st-, 2^nd-, and 3^rd-year internal medicine residents – took 6 mini-CEX examinations during the 1^styear of the study (2017). Two months before the initiation of Mini CEX examinations in the 2^ndyear, multimedia learning material was given to all internal medicine residents. All the necessary information has been presented to the instructors, examiners, and residents. Mini-CEX examinations were held similar to the 1^styear. Second-and third-year residents in the 2^ndyear (who were the 1^st-and 2^nd-year residents in the 1^styear), along with newly entered 1^st-year residents successfully took part in six mini-CEX examinations. Mini-CEX scores of the residents were recorded anonymously and confidentially.

Ethical consideration

The ethics committee of Isfahan University of Medical Sciences has approved the study protocol (IR.MUI.RESEARCH.REC.1398.228). The quasi-experimental nature of the study did not cause any ethical issues, given that the intervention was an innovative educational activity, with unknown effects. However, after the initiation of intervention, all the residents were benefited from it and no one is excluded from the intervention.

Statistical analysis

For analyzing the data, anonymous mini-CEX scores were gathered and entered into statistical software. Continuous variables were presented as mean and standard deviation, while categorical variables were reported as count and percentage. The normality of scores was evaluated by the Shapiro–Wilk test and histogram plots. Given the lack of normal distribution, nonparametric tests were used to compare the medians between groups. Corresponding groups (first in the 1^styear vs. first in the 2^ndyear, and …) before and after intervention were compared using repeated measure ANOVA test followed by Sidak multiple comparison test, given that the group members were not the same, e.g., 3^rd-year residents in the 2^ndyear were 2^nd-year residents in the 1^styear. Analysis was done using the SPSS software version 23 (IBM SPSS Statistics for Windows. Armonk, NY, USA: IBM Corp). In all tests, P < 0.05 was considered significant.

Results

In this study, 135 residents of internal medicine were enrolled. 68 of them participated in the mini-CEX examination before intervention and 67 participants took mini-CEX after the intervention. [Figure 1] As shown in Table 1, the preintervention group consisted of 29 1^st-year, 19 2^nd-year, and 20 3^rd-year residents of internal medicine. Similarly, from 67 residents in the postintervention group, 30, 16, and 21 residents were 1^st, 2^nd-, and 3^rd-year internal medicine, respectively. There were 31 (45.6%) and 27 (40.3%) males in preintervention and postintervention groups. Pre- and post-intervention groups were not statistically different regarding the distribution of gender and year of education [Table 1].

Table 1.

Characteristics of the enrolled internal medicine residents by their gender and year of education

Variable	Preintervention (n=68), n (%)	Postintervention (n=67), n (%)	Total (n=135)	P
Gender
Male	31 (45.6)	27 (40.3)	58	0.603
Female	37 (54.4)	40 (59.7)	77
Year of education
3rd year	20 (29.4)	21 (31.3)	41	0.865
2nd year	19 (27.9)	16 (23.9)	35
1st year*	29 (42.6)	30 (44.8)	59

*The higher number of 1^st year residents is due to the participation of 1^st year cardiology residents in the internal medicine program for 1 year

The median scores of residents were presented in Table 2. The average mini-CEX score of residents before and after intervention was 15.75 (3.95) and 19.62 (2.66), respectively. The median score of students in preintervention and postintervention groups was 16.14 (5.19) and 19.62 (3.95), respectively. This showed a significant increase in mini-CEX of the groups who used the multimedia learning material compared to those who did not benefit from it (P < 0.001) [Table 2].

Table 2.

Comparison of mini-clinical evaluation exercise scores between corresponding residents of each year (out of 30)

Internal residents of …	Preintervention score		Postintervention score		P
	Median (IQR)	Mean (SD)	Median (IQR)	Mean (SD)
3rd year	18.07 (3.93)	17.78 (3.69)	21.43 (3.47)	21.05 (2.78)	0.0029*
2nd year	17.14 (3.93)	16.74 (3.36)	20.46 (2.02)	19.72 (3.39)	0.0261*
1st year	13.86 (5.14)	13.79 (3.60)	18.71 (1.95)	18.87 (1.69)	≤0.0001*
Total	16.14 (5.19)	15.75 (3.95)	19.62 (3.13)	19.76 (2.66)	≤0.001**

*Mann-Whitney U test, **Repeated measure ANOVA with Sidak multiple comparison correction. IQR=Interquartile range, SD=Standard deviation

Next, we compared the scores across the residents with the same level (i.e., 1^st-year preintervention vs. 1^st-year postintervention, 2^nd-year preintervention vs. 2^nd-year postintervention, 3^rd-year preintervention vs. 3^rd-year postintervention). After correction for multiple comparisons, each postintervention group had a higher score in comparison with its corresponding preintervention group [Table 2 and Figure 2].

Figure 2.

Changes in Mini-CEX scores in 1^st, 2^nd, and 3^rd-year residents before and after intervention

Discussion

Multimedia learning resources demonstrated a promising influence on internal residents' mini-CEX scores in this study. They showed significantly greater performance after using multimedia learning materials compared to their same-year residents who did not benefit from it. This demonstrates the favorable effect of multimedia on the acquisition of practical skills such as obtaining a history or performing a physical examination. Similarly, in a study on physical examination of the extremities, computer-assisted learning was found to be beneficial when combined with traditional physical examinations.[18,19,20]

Physical examinations were previously taught in a hospital setting, at the bedside, under the supervision of clinical professors. It, however, had several drawbacks, including a limited opportunity for professors to cover full details, unsettling patients, making them fearful, induction of low self-esteem in the medical trainee, and a lack of opportunities for all students to practice under the observation of their teacher. These issues resulted in incoherent and ultimately inefficient instruction.[21,22,23] On the other hand, due to the dynamic and ever-changing nature of medical knowledge, it is impossible to teach students all aspects of clinical skills.[24] As a result, students are urged to pursue alternative training techniques to gain the necessary knowledge and skills. It is believed that innovative strategies will increase the effectiveness of medical education.[25,26,27] These strategies include learning about physical examinations in small groups with exercises on each other and on moulages under the supervision of physicians, using simulators of abnormal physical examinations, and employing educational multimedia programs that combine the aforementioned methods.[28] For instance, a recent study assessing the effect of integrated teaching-clinical and theoretical-in the learning of undergraduate students, reported improved acquisition, retention, and applicability of knowledge along with greater student satisfaction.[29] In another study, enhancing the course material using multimedia materials, improved student engagement and attitudes regarding the course materials.[30] Another study established that presenting learning materials in a variety of representation forms/channels (voice, visuals, and screen text/labels) results in superior outcomes to solely speech or simultaneous presentation.[31]

Studies have shown that the use of combinational strategies in learning can be beneficial.[32] Among the aided learning approaches, computer-assisted learning and educational videos offer certain advantages since a computer training package can include not only written content but also tables, graphics, animations, and movies.[32] These multimedia packages and educational movies have two primary benefits: First, they enable students to participate in and manage their learning experience.[33] Notably, student-based strategies place a premium on student involvement in the learning process, and the adoption of such methods in education is trending toward student-based learning.[34] Second, these instructional packages can be prepared and used as a scientific reference, which academics rely on during performance assessments. These methods are already being utilized to teach some procedures such as nasogastric tube installation, peripheral blood smear preparation, intravenous catheter placement, electrocardiogram acquisition, and basic surgical skills training. In addition, instructional resources were used to teach specific clinical skills such as respiratory sounds and heart sounds that may not be fully grasped through clinical experience.[35,36,37]

Limitation and recommendation

Our major limitation of the study was its quasi-experimental nature. The compared groups were not identical in the case of the members, which potentially confound the results, given the difference in their baseline characteristics. However, a similar pattern of changes was seen across all the participants, which might indicate a true change. Another limitation of the study was that the residents did not participate in the same rotations each month, however, they rotated in a similar list of rotations but with a different order; so, they could gain different clinical experiences that could potentially impact the scores they got afterward.

Conclusion

Multimedia learning resources demonstrated a promising influence on internal residents' mini-CEX scores in this study. They demonstrate significantly greater performance after using multimedia learning materials. This demonstrates the favorable effect of multimedia on the acquisition of practical skills such as obtaining a history or performing a physical examination.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.