Evaluating learning of medical students through recorded lectures in clinical courses

Abstract

One solution offered by technology is learning through recorded lectures (RLs). The main objective of the study was to evaluate the efficacy of online RLs in learning of clinical courses at a School of Medicine. The study comprised four phases: (i) pre-post uncontrolled study to evaluate knowledge gain with RLs, (ii) non-randomised crossover study to compare learning with RL before or after a face-to-face lecture (FL), (iii) focus groups to evaluate perceptions from students about RLs and, (iv) randomised controlled trial to verify whether the addition of questions every 10 min and a summary webpage to an RL improve knowledge. Results showed that knowledge gain occurred through RLs and was similar to FLs. Additionally, either watching an RL after attending an FL or vice versa showed comparable additional knowledge gain. Furthermore, students were in favour of the use of RLs but not as a replacement of FLs. At last, the insertion of questions and a summary in RLs meant no additional knowledge gain. In conclusion, it is feasible to learn through RLs in clinical courses, but the importance given to FLs suggests adopting blended learning.

Undergraduate medical education; Online systems; eLearning; Recorded lectures.

1. Introduction

One of the applications of information and communication technologies is in the field of education. E-learning is a broad concept that refers to the use of electronic systems [1], mainly Internet technology, to offer a wide range of solutions to improve knowledge and performance [2]. Also, access to mobile devices has been facilitated with various low-cost products, so more and more students use them as tools for information search [3]. Medical students are no strangers to these innovations. Moreover, they will use technologically advanced instruments for diagnosis, treatment, and decision-making in their professional future.

A part of having various learning styles, students nowadays have a greater affinity with technology [4, 5]. Their technology use patterns suppose new ways of learning [6]. Moreover, traditional teaching methods, such as face-to-face lectures (FLs), no longer meet alone the needs of undergraduate students [7]. Information technology has the potential to revolutionize medical education, and students are the ones that most easily adapt to it [8]. Rapid access to information provides a great opportunity for teachers to innovate education and deal with the current exigencies of the curriculum.

One method that fits the student's availability to learn is the use of recorded lectures (RLs) posted online via the Internet or a local area network. RLs give the students the advantage of learning according to their rhythm and availability of time; and offer them the possibility to review difficult concepts and take more notes by pausing, rewinding, or fast-forwarding [9]. Some continuing medical education programs provide RLs to their students to overcome commute and working schedule problems [10, 11].

Several papers explored the use of online RLs in undergraduate medical education. In preclinical courses, certain studies reported equal or better performance with RLs in courses such as biochemistry [12], embryology [13, 14], and anatomy [15, 16]. Some studies also showed a high percentage of students watch RLs when are offered [17, 18, 19], except in one in which most participants used RLs sparingly and was inversely associated with their grades in basic science courses [20]. In a study conducted at Harvard Medical School, 20.2% of first- and second-year students valued scheduling flexibility as the most important reason for replacing attendance to live lectures with RLs [21]. Meanwhile, one retrospective study detected slightly better scores on Step 1 as RLs were more readily available [22]. Conversely, another study indicated no correlation between the use of RLs and final grades in three basic science courses [23]. Zureick et al. surveyed three cohorts of first-year medical students and found that students who always attended FLs or watched RLs in histology had higher average scores than those who opted for a mixed strategy [24]; and several interruptions and distractors impacted negatively on histology performance in both lecture delivery methods.

In clinical courses, there are mixed results. One before-and-after study compared knowledge gain between fourth-year medical students and residents who watch online RLs and those who not during a pediatric emergency medicine rotation and reported better post-test results in trainees from the intervention group [25]; however, a comparison between RLs and FLs could not be performed. Maher et al. conducted a prospective study to compare RLs and FLs on substance use disorders among third- and fourth-year medical students and found no differences in improved scores; however, students in the FL group had small but significantly higher satisfaction levels [26]. A crossover randomised controlled trial showed similar scores after watching RLs or attending FLs about two topics in rheumatology, but participants tended to prefer FLs [27]. Brockfeld et al. also found that RLs and FLs have the same effect for the clinical part of the second section of the medical state examination in Germany, and nearly half of students preferred FLs [28]. A randomised controlled trial made by Alnabelsi et al. showed similar improvements in scores between attending an FL and viewing that lecture in a streamed online way [29].

In the Alberto Hurtado School of Medicine (AHSM) at Cayetano Heredia Peruvian University (UPCH), few RLs were already available in some clinical courses in 2014. However, the effectiveness of RLs in learning has not been evaluated to the best of our knowledge in the clinical period in Latin American medical schools. Changes in the lecture delivery method should be analysed to verify whether they meet the desired outcomes. Additionally, it is necessary to know the possible benefits and disadvantages perceived by students before expanding their use.

The main objective of this study was to evaluate the efficacy of RLs published online for students' learning in clinical courses of the Undergraduate Program of Medicine by comparing the measured knowledge with objective evaluations. The secondary objectives were [1]: to compare the effectiveness of viewing an RL versus attending an FL [2]; to compare the effectiveness of receiving an FL before viewing the RL, versus seeing the RL first and then attending the FL [3]; to compare the effectiveness of an RL with additional content to reinforce learning, versus an RL without these supplements; and [4] to know the students’ perceptions towards the use of RLs as a learning tool.

2. Methods

2.1. Study design

The research consisted of four phases: (A) uncontrolled before-after study, (B) non-randomised crossover study, (C) focus groups (FGs) study, and (D) parallel-group randomised controlled trial.

2.2. Description of the medical curriculum at AHSM

The medical curriculum at AHSM lasts seven years and is divided into four sequential periods: basic sciences (first year), preclinical (second and third year), clinical (fourth and fifth year), externship (sixth year), and internship (seventh year). In the clinical period, medical students make their clinical sessions in different hospitals in the morning for four hours (from 8 a.m. to 12 p.m.). Later, in the afternoon, they attend FLs (usually two one-hour classes).

2.3. Participants

Participants were fifth-year medical students from AHSM who carried out clinical courses in 2014. The inclusion criteria for each phase were: registered students in Medical Clinics II (phase A), registered students in Pediatric Clinics II (phase B), students who participated either in phase A or B (phase C), and registered students in Neurology Clinics (phase D). Students who took one of the courses a second time or those who came from other universities were excluded.

Table 1 shows the baseline characteristics of participants in phases A, B, and D. Gender and age were comparable between groups of each phase. The overall participants were between 20 and 27 years old, and both genders were equally distributed. In phase C, a total of 25 students participated, having a mean age of 23 years (see Table 2).

Table 1.

Characteristics of participants in phases A, B and D.

		Group 1	Group 2	p
Phase A	n	19	22
	Female, n (%)	9 (47.37%)	15 (68.18%)	0.216a
	Age, mean (range)	22.37 (20–25)	21.95 (20–24)	0.492b
Phase B	n	22	19
	Female, n (%)	8 (36.36%)	13 (68.42%)	0.062a
	Age, mean (range)	22.05 (19–25)	22.58 (20–27)	0.347b
Phase D	n	62	62
	Female, n (%)	29 (46.77%)	33 (53.23%)	0.590a
	Age, mean (range)	22.76 (20–27)	22.42 (20–26)	0.320b

^aFisher's exact test.

^bMann-Whitney U test.

Table 2.

Age and VARK questionnaire results in FGs.

FG	Number of participants	Age		VARK
		Mean	Range	Visual	Auditory	R/W	Kinesthetic
1	5	23	22–26	1	1	1	2
2	6	23	21–25	0	2	2	2
3	7	22	21–24	1	0	2	4
4	7	22	21–23	1	0	4	2

2.4. Procedures and techniques

2.4.1. Phase A

This phase evaluated the gain of knowledge through RLs. Participants in this phase were randomly assigned to watch one of two edited RL from actual class sessions presented as part of the course for a different cohort. The RLs were about liver tests (group 1) and enteric viral hepatitis (group 2). Forty-one (75.93%) of 54 randomised students accessed the platform and completed this phase within a week of enrollment, as shown in Supplementary Material 1 (SM1). These RLs were available online in a virtual classroom for a week. The students individually completed a pre-test at first and then watched their assigned RL. Afterwards, they completed a post-test.

2.4.2. Phase B

This phase aimed to compare knowledge gain in FLs with that in RLs using a crossover design. Forty-one (68.33%) of 60 students completed this phase (see SM2). Students of group 1 were tested before (pre-test) and after (mid-test) an FL about eating disorders. While the teacher gave the lecture, it was being recorded. The next day, students were gathered in a room with laptops for each of them to access a virtual classroom and watch the RL. Finally, they answered another test (post-test).

Afterwards, students of group 2 were gathered in the same room with laptops to take a pre-test, watch the RL and solve a mid-test. After six days, they attended the FL and responded to a post-test.

At the end of the course, students of both groups answered a survey on their perceptions about online RLs as an alternative to traditional FLs. Likert scales were used for responses ranging from 1 (totally disagree) to 5 (totally agree).

2.4.3. Phase C

This phase was a qualitative FGs study that drew on grounded theory. The reporting of this study complied with the guidelines of the consolidated criteria for reporting qualitative research (COREQ) [30].

Four FGs were conducted to explore their learning methods and collect their perceptions about learning through RLs. The aims were: in FG1 to know the students' perceptions about an RL and the effects in their learning, in FG2 to identify the students' reasons for not watch an RL, in FG3 to describe the students' perceptions towards the experience they had when they first attended an FL and then watched the RL of that class, and in FG4 to define the students’ perceptions towards their experience when they first watched an RL and then attended to the FL of that class.

A random sample of students who participated in phase A was invited to the first FG (FG1), while all of those who did not watch any RLs in that phase were invited to the second FG (FG2). A random sample of students from groups 1 and 2 of phase B was invited to the third FG (FG3) and fourth FG (FG4), respectively (see SM3). Participants for this phase were recruited after phases A and B had ended and were invited to participate via e-mail. We did not explore reasons for not participating.

FGs were moderated by an independent female anthropologist (FG1 and FG3) and a male sociology student (FG2 and FG4), both trained in qualitative research. We did not have any evidence of bias or previous assumptions from the moderators concerning the study topic. The principal investigator participated as an assistant. There was not any relationship between the participants and the moderators or the assistant. The participants only knew that the assistant was a medical professor. The moderators opened the sessions by introducing themselves and providing information about the study.

FGs were conducted in August 2014. The meetings were in the administrative rooms of AHSM. Only the participants, the assistant, and the moderator were present during the sessions. During the development of the FGs, the moderators maintained a neutral stance and explored in depth the participants' different opinions. The assistant helped the moderators clarify some of the participants’ answers related to their courses or explain the medical terminology used by them.

Additionally, before the sessions started, all participants answered the Spanish version of the VARK questionnaire [31] to know about their learning preferences for taking information.

2.4.4. Phase D

This phase evaluated whether knowledge gain through RLs increases more with additional resources. In the second semester, fifth-year students took the Neurology Clinics course in which the lecture about meningitis was only available as an RL in a virtual classroom. One hundred twenty-four students participated in this phase (see SM4), with a mean age of 22.59. Two groups were randomly formed: Participants from group 1 watched the original RL, and those from group 2 watched the modified RL.

Some suggestions to reinforce learning made by the participants in phase C were considered to modify the original RL for group 2. The RL was divided into three sections and presented as a lesson, a module that contains HTML pages with a progressive flow according to the actions performed by the student. The lesson included multiple-choice (5 alternatives) single-answer questions after each video section, with additional feedback after an option had been chosen, and ended with a summary page. Students continued with the remaining pages even if they marked an incorrect option.

Both groups were gathered in separate rooms with laptops for each student to access a virtual classroom. In there, they first completed a pre-test, then accessed the lesson or the RL, and finally answered a post-test.

2.5. Usage of software for RLs

Lectures were recorded with Camtasia Studio 8.3 (TechSmith, Okemos, MI), which offers various functions for screen capture and audio. This program stored the slides presented and the teacher's speech. Then, editions were made to eliminate idle times and any other unnecessary periods to make the presentation more fluid.

One of the servers of the AHSM hosted the virtual classrooms for phases A, B, and D, which were built with Moodle 2.6 (Moodle Pty Ltd, Perth, Australia), an open-source learning management software. Moodle allowed the creation of tests, registration of answers from participants, and other necessary settings. The RLs of phases A and B were stored within Moodle, while those of phase D were uploaded to a Vimeo account to reduce the risk of overloading the AHSM server. Additionally, the option “restricted access settings” was activated in Moodle to ensure the correct sequence of events; for example, an RL will not be available until the pre-test is completed.

In phase B, Safe Exam Browser (SEB; ETH Zurich, Zurich, Switzerland) 2.0 software was installed on each laptop to deliver tests. In Moodle, the option of “using a secure browser” in test configuration was activated to make them only accessible from SEB. Moodle was also configured to show a main page that requested students their usernames and passwords. Once activated, SEB prevented the students from using the laptops for any task other than the test. In phase D, SEB was not used because the RLs were housed off the platform, as described above. However, the options for randomization of questions and alternatives were still activated.

2.6. Data collection

Online assessments of phases A, B, and D consisted of 10 multiple-choice single-answer questions. These tests were applied as stated in the description of phases to determine the knowledge gain. The maximum allowed time to complete the tests was 5 min. The order of questions and alternatives was random to make cheating more difficult. Question review options were disabled after completing the tests, and only the first answer to a question was accepted. All questions were designed by the study investigator and reviewed by the professor giving the lectures or by specialists in the topics.

For phase C, a semi-structured interview guide was developed for each FG by the investigators. Common contents of these guides are shown in SM5. All the FGs were held in Spanish. FGs sessions were recorded using two audio recorders, with consent from participants. The moderators also took field notes during FGs. Each FG lasted approximately 1 h. We considered that data saturation could be achieved with the number of FGs in this study. Recordings were then transcribed verbatim for examination. Transcripts were not returned to participants for comment or correction.

2.7. Ethical considerations

Written informed consent was obtained from participants. We explained to the students that a decision to not participate would not harm them in any way. Confidentiality of the data obtained was guaranteed. The study involved educational interventions and corresponded to research with minimal risk. The results of the tests from this study were not used as grades of the participants. Study records were anonymized and will be retained until the publication of this paper. Also, the participants of each phase were informed and made familiar with the objectives of the study. The study proposal was reviewed and approved by the institutional review board at UPCH (IRB #62655).

2.8. Data analysis

Frequencies and percentages are presented for categorical variables, while means and ranges are presented for numerical variables. Categorical variables were compared using Fisher's exact test. Kolmogorov-Smirnov tests showed that scores and the change in scores were not normally distributed; therefore, nonparametric tests were used. Two independent sample scores were compared with the Mann-Whitney U test. The Wilcoxon signed-rank test was used to evaluate differences between paired scores. A p-value less than 0.05 was considered statistically significant.

In phase C, the moderators coded data of their assigned FGs. Then, data were organized in a coding tree with three major themes: 1) Perceptions about FLs, 2) Perceptions about RLs, and 3) Expectations regarding RLs. For the derivation of subthemes, we applied the constant comparative method of analysis from grounded theory to the data by employing the three steps of coding and categorizing: open, axial, and selective coding [32]. A comprehensive review of the data was made to produce descriptions and interpretations for their classification in accordance with a table of codes. Afterwards, the results were compared, and subthemes were identified. Data management was done directly with a word processing program instead of a qualitative data analysis software. We did not get feedback from participants on the results of the analysis.

2.9. Sample size and power

For the present study, the sample size was determined by the number of students assigned to groups for academic reasons, so no sample size calculation was performed. For phases A, B, and D, the minimum detectable difference between the study arms with the available sample size was calculated. In all three cases, a standard deviation of 2, a power of 80%, and an alpha level of 0.05 were used. The dependent variable was the change in the score between the post-test and the pre-test (mid versus pre and post versus mid in phase B). The available sample size (19 students) allowed the detection of changes of at least 1.360 points in the average score in each group of phase A. In phase B (with 19 and 22 students in each group), the minimum detectable difference in change of score was 1.799 points. The minimum detectable difference in phase D, with 62 students in each group, was 1.014 points.

3. Results

3.1. Phase A

Knowledge gain in each group was statistically significant (see Figure 1). These results confirm that learning occurs through RLs.

Figure 1.

Test results in phase A. ∗p ≤ 0.001 (Wilcoxon signed-rank test).

3.2. Phase B

There was no difference in knowledge between the two groups at baseline (p = 0.800, Mann-Whitney U test). Gain score from pre to mid-test was significant in both groups (p ≤ 0.001), as shown in Figure 2, but similar either watching the RL o attending the FL (p = 0.915). Knowledge gain between mid and post-test was relatively modest, but again statistically significant (p = 0.021 and 0.022 for groups 1 and 2, respectively) and similar for both groups (p = 0.417); meaning that receiving a lecture a second time but with different format results in some additional knowledge gain, regardless of the order of the formats.

Figure 2.

Test results in phase B. ∗p ≤ 0.001 (Wilcoxon signed-rank test). ^†p (Group 1) = 0.021, p (Group 2) = 0.022 (Wilcoxon signed-rank test).

The survey on perceptions towards the use of RLs indicated that the participants agreed to its implementation (see SM6), with an average score of 4.07.

3.3. Phase C

After applying the VARK questionnaire, the predominant learning styles were kinesthetic (10 participants, 40%) and reading/writing (9 participants [36%]). None of the participants had a multimodal style. Neither those who watched the RL (FG1) nor those who did not (FG2) had a preferred style. The subthemes of the three main themes that resulted from data analysis are summarized in Table 3 and presented below in detail. Each quotation (Q) is shown in SM7.

Table 3.

Themes and subthemes of FGs.

Theme	Subtheme
Perceptions about FLs	Importance of a dynamic methodology. Importance of the teacher. Taking notes. Balance between theory and practice.
Perceptions about RLs	RLs and use of time. Online RLs as reinforcements for learning. Relationship between medical teacher and student. Reasons for not watching online RLs.
Expectations regarding RLs	The importance of the audiovisual. Assistance of the teacher. Variety of topics and availability.

3.3.1. Perceptions about FLs

Importance of a dynamic methodology. According to the students, in order to improve classes, professors should apply a dynamic methodology, different from just showing and reading slides. Attending a monotonous lecture makes no sense to them when instead they could review the slides of the class comfortably at their homes. Furthermore, they expressed that teachers should promote participation, breaking with the classical vertical teacher-student relationship. They preferred a lecturer that establishes a horizontal relationship that favours a more fluid knowledge exchange (Q1). They also reported losing interest in a lecture—getting bored and even falling asleep—when the teacher applies a flawed methodology (Q2).

Importance of the teacher. Participants considered that medical teachers should serve as guides or mentors. Their role should not be limited to imparting knowledge. They should contribute with their practical experience and medical judgment in clinical cases. Students manifested that they take into account any orientation given by their teachers about recommended readings, including research topics, among others (Q3-5). Additionally, they reproach that not all their teachers carry out such role, since some students get the sensation that most physicians “just read the slides and at the end say goodbye!” (FG4). Students valued professors that are permanently up-to-date physicians who investigate, have many publications, and are recognized professionals. These attributes encourage them to attend classes. Moreover, receiving a lecture from a well-known physician is for them an element that reinforces their status, a qualitative differentiation that distinguishes them as students from an elite university (Q6-7). Likewise, students appreciate the image projected by the teacher as a role model, paying attention to the doctor's behaviour, way to talk, appearance, etc. (Q8).

Taking notes. Many of the participants said that they make annotations on mobile devices (smartphones or tablets) because these facilitate the drawing of graphics. Additionally, they download the presentations to make annotations next to the slides. However, they recognized that on many occasions taking notes provokes a negative impact on their attention, making it difficult to keep track of the speech (Q9). In consequence, some of them record the speech of the lecturer for review and expand their notes.

Balance between theory and practice. The students agreed that classes should possess a balance between theory and practice. They considered that, apart from theoretical knowledge, teachers should transmit their experience as physicians, explaining how they manage patients with a certain diagnosis. In this way, new concepts would correlate with real professional settings, facilitating learning (Q10-11).

The environment of classes. An issue not included in the guide, which spontaneously arose in the FG, was the importance of the environment or the physical infrastructure of the classrooms in the learning process. Bad conditions of these rooms would be a factor that negatively impacts their attention (Q12-13).

3.3.2. Perceptions about RLs

RLs and use of time. One of the advantages that some participants identified in RLs was the possibility of establishing their own study schedules, which they value as part of the responsibility of being adults. This is related both to the status as university students and the freedom of decision in the use of their time (and an indirect criticism to FLs as a symbol of a tutorial order) (Q14). Also, the attendance of clinical practices and FLs in distanced places was mentioned as a major drawback of FLs, which entails a considerable investment of time in travel. With RLs published online, they considered that they could better spend their time, avoiding these continuous trips, and distribute their study schedule more efficiently (Q15-16). Some students stated that they adapt their study habits when they use electronic devices. The alternatives of pausing and speed playing offer them greater manoeuvrability and disposition of their time, in addition to studying in the most comfortable way, “accompanied by a coffee or lying in bed” (FG1), which is also highly valued by students (Q17-18).

Online RLs as reinforcements for learning. Additionally, RLs represented for the participants a more efficient mechanism of transmission of knowledge when the class is adapted to the requirements of students. This efficiency is sustained in two points: First, an adequate RL reduces the possibility of distraction (a common occurrence in FLs produced by drowsiness, bad classroom conditions, and ineffective habits of taking notes) (Q19). Second, the possibility to watch RLs with their peers at any time permits a greater interaction between them and a better understanding of the study topics (Q20). However, some students said that they are less distracted in FLs because they are under pressure and control of the teacher (Q21). The participants also considered important to be exposed to the topics ahead of time, to gain more from FLs and contribute to them. Some students proposed that the RLs should be available online before hospital practices (Q22). Despite this, the participants also pointed out the importance of implementing different dynamic methods in the RLs that adapt better to their learning abilities and remain attentive. Changing to RLs by itself does not warrant a productive learning process (Q23-25).

Relationship between medical teacher and student. Some participants expressed their concern about the implementation of online RLs as the only learning delivery method. They feared that RLs replace the critical, human and professional training, which can only be achieved through direct interaction with teachers (Q26-27).

Reasons for not watching online RLs. Of the six participants of FG2 (invited to watch an RL in phase A but did not watch it), 2 reported technical problems (Q28-29), two considered it was not worth watching it (Q30-31), and two were too busy to watch it (Q32-33).

3.3.3. Expectations regarding RLs

The importance of the audiovisual. Most participants suggested applying audiovisual techniques in video editions to facilitate learning. A group of students also recommended that RLs should not show the teacher's face all the time; shot transitions between slides and the lecturer's face would remedy the monotony of watching only slides (Q34). If some elements such as sound effects, closed captioning, arrows, images and videos were added for reinforcing and emphasizing what the teacher says, the learning experience would be more efficient (Q35-36). Students considered that focusing on the teacher's face in the video also helps to understand the class due to the emphasis of their gestures or the body language that remarks a relevant issue (Q37). They also considered important the audio quality of RLs. One participant stated that in one of the RLs of Clinical Pediatrics 2 (which were not part of this study), the teacher's voice was too low. So, it is advisable to check the quality of the RLs before publication.

Assistance of the teacher. Other suggestions of the students were linked to the support from the teacher. One of their concerns about implementing online RLs was the absence of a real-time question/answer system to solve their doubts timely, like in FLs. Some participants proposed the inclusion of a chat plugin at the bottom of the screen. However, this implies that the RLs were planned on a strict schedule when the teacher is available to answer questions (Q38) and opposed to the free time management.

Variety of topics and availability. Participants desired more available RLs, covering more topics to increase their use. In addition, these RLs should be fully available to the students in a video library to reinforce learning. In fact, they considered that the RLs should be downloadable at any time, instead of the limited access offered by the official educational platform (Q39). Also, students proposed the inclusion of reinforcement questions in certain time intervals during the playback of RLs and pre-test and post-test to benefit their learning process (Q40-41). However, they were against restricting access to the following sections of an RL up to selecting the correct answer. That limitation could interfere with managing their learning process, according to their cognitive abilities and velocities, ultimately increasing the time spent watching the RL.

3.4. Phase D

Both groups had similar amount of knowledge before the intervention (p = 0.609, Mann-Whitney U test). As occurred in phases A and B, there was a significant increase of knowledge in both groups after watching the RLs (see Figure 3 and SM8). Incorporating additional resources into the RL for group 2 did not produce a greater knowledge gain (p = 0.767).

Figure 3.

Test results in phase D. ∗p ≤ 0.001 (Wilcoxon signed-rank test).

4. Discussion

This study demonstrates that recorded lectures produce the same knowledge gain as traditional face-to-face lectures, making them suitable for clinical courses.

Phase A showed that RLs are effective at increasing knowledge in medical students. Similar findings were observed in other researches such as cataract surgery [33], dermatology [34], and violent person management [35].

In phase B, students performed equally regardless of the method of content delivery. Equivalent knowledge gain was found in studies about topics in cardiac ultrasound [36] and sleep medicine [37]. We also found that watching an RL before or after the corresponding scheduled FL generated knowledge gain, making them a useful complement to FLs. However, it seems mandatory to know previously about the skills of students to use technology for learning. Backhaus et al. found that “digital natives” got significantly lower scores in comparison with “traditional learners” when exposed to an FL on goitre [38]; in contrast, both groups had similar knowledge gain with the RL. Despite students’ affinity with technological tools for knowledge gain, this does not ensure that all of them will watch all the available RLs. Nevertheless, this should not impede whether RLs are incorporated as complementary material.

Phase C shows that a dynamic methodology in lectures was of utmost importance for students. This aspect seems to be key for a successful implementation of a course with online RLs. The latter is also pointed out in a study at the Robert Wood Johnson Medical School that aimed to identify the factors that influence decisions to attend FLs in first- and second-year students, the ways in which they use RLs and their perception about FLs; finding that students value the content and nature of the material presented in class, so much so that they expressed their willingness to attend FLs that included “strong visual components” in their methodology [9].

In contrast, the participants of the current investigation criticized the existence of a monotonous methodology in FLs, characterized by viewing presentations while the teacher explains or reads them without promoting the participation of students. So much so that they warned that if online RLs maintain those methods, their benefits would be lost against the advantages of FLs and the presence of the teacher to absolve doubts. Similar opinions were reported in a study performed at the University of Queensland in where the content, organization and structure, and design and format of online RLs emerged as major themes from FGs with third-year medical students [39], describing the desirable and undesirable attributes.

The students of our study identified a series of benefits offered by online RLs to facilitate learning, linked to the possibility of managing their study time since they can be accessed at any time and manipulate its reproduction at convenience (pause, rewind, and forward). Jordan et al. reported a study about RLs of a course in acute care in which most of the students enjoyed flexibility [40]. The same was found by Chapman et al. in a course in advanced medical therapeutics [41].

It should not seem contradictory when it is asserted that the students valued RLs as tools that facilitate learning by adapting very well to their study habits. Unlike previous generations, students born in the 90s socialized sharing the spaces offered by new technologies, acquiring ways of understanding and apprehending the world [42]. Also, these technologies influenced their study habits and cognitive possibilities (rhythm, retention capacity, among others).

Although most students favoured the use of RLs, many of them also considered that these should not replace FLs. The role of medical teachers in classrooms is important since they recognized their function as part of their training, valuing experience and image, and mainly human contact that cannot be dissociated and should not be lost with the use of RLs. They valued the role of the lecturer as a guide in the process of learning. This interaction allows them to know from the perspective of the teacher how to face different situations. Consequently, changing FLs to RLs would be inappropriate. These characteristics are not unique to students of AHSM. In a study conducted at the University of Massachusetts Medical School, 82% of 163 participants cited previous experiences (positive or negative) with a particular lecturer as a major factor influencing their decisions about lecture attendance [43]. Also, Nast et al. showed that second-year medical students welcomed RLs in dermatology at the Charité – Universitätsmedizin Berlin and view them as a valuable addition instead of replacing FLs [44].

Although the VARK questionnaire was only applied to participants in phase C, the results could guide towards the characterization of the learning style of fifth-year medical students. The two predominant styles were kinesthetic and reading/writing. The proposal made by students in FGs to change afternoon FLs with clinical case discussion (since they consider learning more through the presentation and analysis of experiences) coincides with the description of kinesthetic style. Likewise, the preference for maintaining lectures but delivered in video (because students considered important reading the contents of slides and taking notes) is adjusted to the reading/writing style. On the contrary, graphical representations (like diagrams) that befit the visual style, would not contribute to the learning of these students, as verified by the results of phase D, where there were no significant differences in knowledge gain from the summary schemes. Additionally, the possibility to repeat fragments of video as needed, expressed as an advantage over FLs, would indicate a low affinity for the auditory style.

While in phase D, the addition of questions every 10 min and summary slides at the end of RL for students in group 2 showed no differences in learning with students in group 1. Probably, a better ability of the students to obtain important information in the final period of clinical studies or the content (in both the slides and the speech) of the RL were enough factors to increase knowledge. In addition, some students in both groups got the maximum possible score in the post-tests. If this phase is replicated in a subsequent study, the number of questions in both tests should be increased to detect small differences in learning. Contrary to the findings of this phase, a study conducted at Harvard University showed that the incorporation of questions during an RL could help students extract the content quickly and efficiently by reducing the occurrence of distractions, increasing the frequency of annotations, and facilitating learning [45]. They also found that this form of frequent evaluation decreased anxiety levels and subjective estimation of cognitive efforts over the final test of the presentation.

This study has some limitations. Due to academic reasons, it was not possible to perform a phase that consisted of a parallel-group randomised controlled study in which one group receives an RL and the other one the FL. Another limitation was that not all students in phase A watched the video. Regarding phase B, the assignment of groups was made as established by the Undergraduate Direction and could not be randomised. Also, the number of participants was small in that phase and consequently, minor differences could not be detected. Finally, although the call to participate in the FGs was gender-balanced, the students who attended were mostly women, except for FG1, which included three men, resulting in participation bias. It was not possible to explore cross-gender differences in learning styles. Future studies should explore gender differences in learning preferences and performance.

In view of the perceived importance of FLs for students and the benefits of RLs for learning, we suggest opting for flipped classroom, a rotation-model implementation in blended learning [46]. This method aims to facilitate students make use of their time through the online publication of RLs and complementary material (such as readings) while the time designated for theoretical classes is used instead in discussions under problem-based learning methods, resolution of clinical cases, and response to the doubts that emerged during the review of available material (including RLs) before the formal session. Recently, a meta-analysis showed that flipped classrooms improve learning in health profession education, including undergraduate medical education, compared with traditional teaching [47]. The same was found with a good reception by students in a scoping review on preclinical and clinical years of undergraduate medical education [48].

Declarations

Author contribution statement

Carlos Orellano, Cesar Carcamo: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper.

Funding statement

This work was supported by the Fogarty International Center, United States National Institutes of Health (D43TW008438-0109).

Data availability statement

The authors do not have permission to share data.

Declaration of interests statement

The authors declare no conflict of interest.

Additional information

No additional information is available for this paper.

Appendix A. Supplementary data

The following is the supplementary data related to this article: