The comparison of the efficiency of traditional lectures to video-supported lectures within the training of the Emergency Medicine residents

Abstract

Objectives

Recent developments in computer and video technology, multimedia resources enter quickest way possible into medical education and have started to gain popularity. The aim of this study is to evaluate the impact of video-supported lectures on leaning, with comparison to traditional lectures.

Methods

According to lecture techniques, two separate groups; one is the traditional lectures group (TLG) and the other is video-supported lectures group (VSLG), are formed. While the TLG is offered a traditional lecture the VSLG is offered a video-supported lecture with imbedded videos which are related to the topics in the traditional lecture. Both study groups take pretest and posttest with MCQs (multiple choice questions) and OSCEs (objective structured clinical examination).

Results

The study includes 30 volunteer residents in Dokuz Eylul University School of Medicine Department of Emergency Medicine. No difference is observed between TGL and VSLG in pretest and posttest scores (p = 0.949, p = 0.580). And additionally, comparing the scores of both groups, we cannot observe any difference between the pretest OSCE scores of each group (p = 0.300), however posttest OSCE scores shows a dramatic odd in-between (p = 0.010). When pretest MCQs and posttest MCQs mean scores are compared, both tests (TLG, VSLG) has not any significant difference (p=0.949, p = 0.580). Nevertheless, after comparing OSCEs pretest and posttest mean scores, we can see significant difference in mean scores of both (TLG, VSLG), (p = 0.011, p = 0.001).

Conclusions

Taken into consideration, the findings of this study shows possibility of improving educational techniques to acquire clinical skills by using local resources and low-cost technology.

1. Introduction

With the advances in technology and global connectedness, medical knowledge is developing itself with utmost speed because of this reason, the efficiency, quality and updatability of lectures are of great importance in medical education. With recent improvements in computer and video technology, multimedia resources are entering rapidly into medical education and gaining popularity.1, 2, 3, 4, 5 Within technical education, which is generally used in anatomic lecture slides,⁶ clinical scenarios⁷ and videoconference are commonly used.⁸ It is debated in this study that video assisted teaching makes learning easier for students.⁸

Thanks to the computer softwares that allow multimedia files which play on portable media players or on the internet via computers, access to educational sources has become more reachable. The most important advantage of this learning method is being able to access knowledge without any limitation of time or place.⁹ However, in studies where all lectures are presented with videos over the Internet, student video usage rates seem to be lower than expected.¹⁰

Traditional lectures and video supported lectures differ the advantages and limitations. The advantages of traditional lectures are common usage on all medical educational levels, increased motivation of student to participate in question and answer dialog, less time required for preparation, and the practical usage of the new information into presentations by educators with ease. Video lectures offer the following advantages; the provision of rich educational materials, the superior demonstration of subtle details during procedures, the ability of repetition, and the ending with standardization in medical education.9, 10, 11

Traditional lectures have the limitations depending on the lecturers' skills, inability of repetition of the lecture, and limited educational material usage. On the other hand, the disadvantages of video supported lectures varies from the time it takes to setup the necessary equipment to low listener motivation and to participation of students which can be lower than expected.10, 11, 12 In this study, “traditional lecture” and “video-supported lecture” by combining superior aspects of traditional lectures and video tutorials will be shown on the basis of student learning efficiency.

2. Materials and methods

Dokuz Eylül University, Ethics Committee has approved this study of comparison. The research is done in compliance with the Helsinki Declaration. Learning objectives are derived from the first two subjects of ATLS (Advantage Trauma Life Support); “General Approach to Multiple Trauma Patients” (GAMTP) and “Airway and Breathing Management in Trauma Patients” (ABMTP). The lectures are prepared using the same content but delivered as traditional lectures and as video-supported lectures. In traditional lectures, we use PowerPoint slides without any video content, and in the video-supported lectures we use video involved PowerPoint slides. The video supported lectures take 60 min, twenty minutes of which is composed of video content, the remaining 40 min are used as traditional lecture.

The study includes 30 volunteers from Emergency Medicine residents (EMR) in Dokuz Eylul University, School of Medicine Department of Emergency Medicine. The volunteers are randomly separated into two groups, equally distributed in terms of their class of residency which is 1–5 years of residency program. The first group is assigned as the traditional lecture group (TLG) and the second group is assigned as the video-supported lecture group (VSLG). First of all, each group take a multiple choice questions (MCQs) pretest. Afterwards, all volunteers take an objective structured clinical examination (OSCEs) in three stations. OSCEs are graded by using evaluation table guidelines, which are based on standard steps, and three-point scale (Failed = 0, partially done = 1 and done = 2). Evaluation of all stations are performed for 3–5 min. OSCEs are blindly evaluated by three Emergency Medicine Attending Physicians. Moreover, during the study, they take part in the same step. Afterwards both group attend their lectures. In order to avoid interaction between the groups, presentations and reviews are conducted in two different classrooms. Furthermore, to mitigate the difference in instructor teaching styles, the same presentation is given by the same instructor to the both groups using TLG and VSLG. After the lectures, both MCQs and OSCEs exams are performed. At the end of the study feedback on lectures is given by the participants (Fig. 1).

Fig. 1.

Group and randomization chart.

2.1. Material preparation

2.1.1. Presentation material

The presentations are prepared with the relation to “Advanced Trauma Life Support Program for Doctors, 8th Edition” and other evidence-based trauma management sources. PowerPoint (Microsoft Office PowerPoint 2010, Microsoft^® Corporation, Redmond, Washington) is the program of choice as the presentation tool.

2.1.2. Video material preparation

The subjects of the video clips are chosen from examination and interventional procedures. These video clips are made without any professional help and in the videos simulated patients participated, who are volunteers (Fig. 2A and B). Educational models and cadavers are used for the video clips of interventional procedures and surgical airway techniques. The Ulead^® MediaStudio^® Pro Video Editor 8.0 is chosen as video-editing software.

Fig. 2.

A. and B. Preparation before video shots.

2.1.3. Preparing the evaluation questions

MCQs and OSCEs questions are prepared in accordance with presentation goals. 30 questions are written for MCQs and then separated randomly into pretest and posttest groups which are based on the content items. For OSCEs stations, three skill subjects are chosen and patient scenarios are installed. At the first and second stations, standardized simulation patients are used, whereas in the third station training manikins are used. Participants are asked the same questions before and after the lectures. Scoring is assessed by using the checklist which is prepared for evaluation. Scores are converted to 100-point scale and analyzed.

2.1.4. Statistical analysis

The data obtained is analyzed in the program; called, “Statistical Package for the Social Sciences (SPSS) 16.0 for Windows”. For the statistical analysis, Mann Whitney U and Fisher X² tests are used to compare the groups and mean values. The repetitive value comparisons are performed using Wilcoxon test. The score ‘p < 0.05’, is considered to be statistically significant.

3. Results

3.1. Comparison of pretest and posttest results of TLG and VSLG groups

There is not any statistically significant difference between TLG and VSLG in pretest and posttest MCQ scores. Additionally, with the comparison of the scores of both groups, neither there is statistically meaningful difference among the pretest OSCE scores of each group. However, posttest OSCEs scores show a significant difference between the groups (Table 1).

Table 1.

Pretest and posttest mean scores.

Education groups	MCQs		p*	OSCEs		p*
	Pretest	Posttest		Pretest	Posttest
TLG (n:15)	67.1 ± 13.0	80.9 ± 10.0	0.003	54.8 ± 10.1	63.7 ± 9.1	0.011
VSLG (n:15)	67.1 ± 9.9	78.7 ± 8.1	0.006	50.9 ± 9.1	72.8 ± 12.5	0.001
p**	0.949	0.580		0.300	0.010

Mann Whitney U Test.

p*: Comparisons of each groups pretest and posttest mean score.

p**: Comparisons of the TLG and VSLG groups pretest and posttest mean score.

TLG: Traditional lecture group, VSLG: Video-supported lecture group, MCQs: Multiple choice questions, OSCEs: Objective structured clinical examination.

3.2. Intragroup pretest and posttest results

When MCQs pretest and posttest mean score are compared, both (TLG and VSLG) have a significant increase in the mean score of the test result (p = 0.003 and p = 0.006).

On the other hand, when OSCEs pretest and posttest mean score are compared, both (TLG and VSLG) have a significant increase in the mean score of the test result (p = 0.011 and p = 0.001) (Table 1).

3.3. Comparison pretest and posttest of success based on duration of residency

When MCQs and OSCEs pretest and posttest mean score are compared with the view of the EMRs' duration of residency, all participants have an increase in rate of success. Whereas MCQs, third, fourth and fifth years of residency show success, OSCEs, second, fourth and fifth years of residency show an increase in rate of success, which is found to be statistically significant (Table 2).

Table 2.

Average increase in success rate in MCQs and OSCEs based on duration of residency.

		Pretest	Posttest	Success rate increase expressed in points	p*
1.year n:7	MCQs	56.7 ± 14.1	70.0 ± 4.7	13.4	0.317
	OSCEs	45.9 ± 9.6	54.5 ± 17.7	8.6	0.180
2.Year n:6	MCQs	64.0 ± 13.8	72.0 ± 5.6	8.0	0.157
	OSCEs	52.0 ± 6.7	66.9 ± 5.6	14.0	0.043
3.Year n:7	MCQs	56.7 ± 7.0	75.6 ± 6.9	18.9	0.026
	OSCEs	52.9 ± 14.9	70.6 ± 14.3	17.7	0.075
4.Year n:6	MCQs	72.4 ± 7.1	87.6 ± 9.0	15.2	0.046
	OSCEs	51.8 ± 10.3	70.5 ± 9.3	18.6	0.018
5.Year n:6	MCQs	73.3 ± 8.9	82.7 ± 6.4	9.3	0.026
	OSCEs	55.3 ± 7.71	68.7 ± 10.7	13.4	0.022

*Wilcoxon.

MCQs: Multiple choice questions, OSCEs: Objective structured clinical examination.

3.4. Feedback results

Finally, for the conclusion, all participants are asked, on the feedback form after the lectures, about the sufficiency of lecture time, the sufficiency of educational materials, the efficiency of educational techniques, and the general usefulness with combination with the participants knowledge level. The final scores are between 3,1–4,1 out of 5 points, which is above the average. In the TLG and VSLG groups, the evaluation of the educational techniques show that satisfaction of the study is much greater in the VSLG group in general (Table 3).

Table 3.

Educational technique feedbacks mean scores.

Feedback issues	TLG	VSLG	p
Sufficiency of Educational time	3.2 ± 0.8	3.6 ± 0.8	0.041
Sufficiency of Educational materials	3.1 ± 0.9	3.8 ± 0.7	0.001
Efficiency of Educational techniques	3.1 ± 0.9	4.0 ± 0.8	0.001
Contribution to the subject's knowledge	3.4 ± 1.0	4.1 ± 0.8	0.002
Contribution and usefulness in general	3.6 ± 0.8	4.0 ± 0.9	0.019
Presentation Comprehensibility	3.6 ± 0.8	4.0 ± 0.9	0.065

*Points were given between 1 and 5 (1 = minimum, 5 = maximum).

TLG: Traditional lecture group, VSLG: Video-supported lecture group.

Satisfaction levels are high with the regard to the image and sound quality, technical competence, and the levels of interest that participants show, meaning all of the performances make a contribution to their education and educational goals (Table 4).

Table 4.

Feedback on video clips.

Subjects of the feedback	Average pointsa
Image and sound quality comprehensibility	4.1 ± 0.8
Video technical competence (scenario. content. roles)	4.0 ± 0.8
Subject level of interest in the videos	4.0 ± 1.0
The contribution levels of videos to education and educational goals	4.1 ± 0.8

^aPoints were given between 1 and 5 (1 = minimum, 5 = maximum).

4. Discussion

As mentioned in Mayer's cognitive learning theory, the most important feature that differentiates traditional lectures from video-supported lectures is the information which is given on videos that is processed both visually and auditory, at the same time, making learning easier.12, 13, 14, 15 Traditional lectures may decrease listening comprehension because listeners might lose their attention in lectures in that figures and words are processed only visually. Another reason that listening comprehension decreases is due to synchronization problems in traditional lectures in which there is a disjunction between figures and words because of the lecturers' ineffective presentation skills, which is especially used during slides. For this reason, the synchronized visual and auditory information in video clips can increase listening comprehension particularly on interventional procedures.

In the study by Ali et al, it is advocated that OSCEs should be used for the evaluation of ATLS training, while MCQs should be used for the evaluation of cognitive success.¹⁶ In our study, however, we cannot observe any difference between each group in terms of the MCQs results. Nevertheless, there is a considerable difference in the results of OSCEs. The questions in the MCQ exam covers general knowledge in the lectures given, while the questions in the OSCE stations contains only questions regarding clinical skills. Hence, OSCEs enable us to make a better comparison between the groups' clinical and interventional skills. The usage of video images in lectures increases the skills in clinical and interventional fields, and the information learned from the lectures can be used practically in real life is regarded as an important finding in the field of medical education.

In the study by Schreiber et al, in which classical traditional lectures and video-supported lectures via internet access are compared using undergraduate students, there was no difference found in the MCQ results.⁵ Moreover, in the study by Davis et al, in which undergraduate groups are compared with the basis of classical traditional lectures, which are prepared with Microsoft^® PowerPoint^®, and the lectures are supported with computers and voice records, they cannot find any difference in the MCQ results.¹¹ Similarly, the same result is found in our study, showing none of the difference between the group results in MCQs, but meaningful differences in OSCEs.

Additionally, trauma patients are seen in our Emergency Department almost every day, most of the EMRs already have preliminary knowledge of trauma patients. Hence, it becomes difficult to increase a level of knowledge that is already more than mean score. Even though, when intra-group MCQ and OSCE results are compared, it is found that both TLG and VSLG statistically increase educational success.

While preparing the video-supported lecture video clips, we do not use any professional help for video scenarios of treatment, camera shootings, make-up, wardrobe and models which are used. From the feedback form, there are positive comments for both lecture techniques, anyhow VSLG get much higher ratings. The video clips, which we manage to prepare economically and without professional help, are favored by the participants and increased educational success. With this reason, we conclude that video-supported lectures can be prepared economically and used easily.

5. Limitations and suggestions

Since this is a single focal study, research participants are limited to 30 people. In addition to this, another limitation is that only two topics in two hours to present and evaluate their knowledge and skills to assess learning in medicine. It would be better if the study is designed for few days of course with different educational subjects. So that video-supported lectures can be used more widely. It is suggested that studies with a larger number of undergraduate participants or residents from other provinces can entail more subjects to be performed. For this reason, we believe that this study is a model for future studies regarding video supported lectures in similar subjects.

It is believed that a standard of medical education, with respect to clinical skills, can be established among educational institutions if video clips of particular subjects are prepared by professionals and shared by these educational institutions.

6. Conclusions

To sum up, regardless of lecture technique, lectures on “the management of a trauma patient” significantly increase the knowledge level of EMRs. Besides, in this study in MCQ scores, none of significant difference between the two groups is found, yet VSLG perform better clinical skills than TLG. This is an exemplary study which shows the possibility of improving educational techniques to acquire clinical skills by using local resources and low-cost technology.

Funding

None Declared.

Conflicts of interest

None Declared.