Design, Implementation and Evaluation of a Distance Learning Framework to Expedite Medical Education during COVID-19 pandemic: A Proof-of-Concept Study

Abstract

Background:

The COVID-19 pandemic has forced medical schools to suspend on-campus live-sessions and shift to distance-learning (DL). This precipitous shift presented medical educators with a challenge, ‘to create a “simulacrum” of the learning environment that students experience in classroom, in DL’. This requires the design of an adaptable and versatile DL-framework bearing in mind the theoretical underpinnings associated with DL. Additionally, effectiveness of such a DL-framework in content-delivery followed by its evaluation at the user-level, and in cognitive development needs to be pursued such that medical educators can be convinced to effectively adopt the framework in a competency-based medical programme.

Main:

In this study, we define a DL-framework that provides a ‘simulacrum’ of classroom experience. The framework’s blueprint was designed amalgamating principles of: Garrison’s community inquiry, Siemens’ connectivism and Harasim’s online-collaborative-learning; and improved using Anderson’s DL-model. Effectiveness of the DL-framework in course delivery was demonstrated using the exemplar of fundamentals in epidemiology and biostatistics (FEB) course during COVID-19 lockdown. Virtual live-sessions integrated in the framework employed a blended-approach informed by instructional-design strategies of Gagne and Peyton. The efficiency of the framework was evaluated using first 2 levels of Kirkpatrick’s framework. Of 60 students, 51 (85%) responded to the survey assessing perception towards DL (Kirkpatrick’s Level 1). The survey-items, validated using exploratory factor analysis, were classified into 4-categories: computer expertise; DL-flexibility; DL-usefulness; and DL-satisfaction. The overall perception for the 4 categories, highlighted respondents’ overall satisfaction with the framework. Scores for specific survey-items attested that the framework promoted collaborative-learning and student-autonomy. For, Kirkpatrick’s Level 2 that is, cognitive-development, performance in FEB’s summative-assessment of students experiencing DL was compared with students taught using traditional methods. Similar, mean-scores for both groups indicated that shift to DL didn’t have an adverse effect on students’ learning.

Conclusion:

In conclusion, we present here the design, implementation and evaluation of a DL-framework, which is an efficient pedagogical approach, pertinent for medical schools to adopt (elaborated using Bourdieu’s Theory of Practice) to address students’ learning trajectories during unprecedented times such as that during the COVID-19 pandemia.

A preprint version of this manuscript is available with modifications is available at: DOI: 10.21203/rs.3.rs-30932/v1

Introduction

As the COVID-19 pandemic becomes unescapable, its detrimental effects on medical education turn out to be all pervasive. Several medical schools have suspended all on-campus live sessions with hopes of mitigating viral transmission and have shifted to distance learning (DL).^1-3 This abrupt and unforeseen shift to DL has presented the medical education faculty with an arduous challenge, ‘to create a “simulacrum,” that is, representation of the learning environment that medical students experience during live on-campus sessions in a DL milieu’.^4,5 This difficulty in creating such a simulacrum of a live on-campus learning environment in DL, is analogous to Hand met Spieglende Bol problem of Maurits Cornelis Escher, best explicated in Neal Stephenson’s book Fall, or Dodge in Hell, where the author identifies that an attempt to create live or ‘real-life’ experiences in an in-silico setting is often fraught with multifarious challenges. The point being, that in order to create a faithful rendering of a live on-campus learning environment for medical education through DL, one needs to consider infinitesimal aspects, best explicated by employing the precepts of ‘complexity theory’.^6-8 In line with these precepts, proprietors of medical education need to tackle if DL is elastic enough to be adapted through forms of learning suitable for the cultural context. Further, the complexity question (‘does DL work?’) remains unresolved and requires more consideration (‘how might DL be adapted?’). Also, one needs to account for externalisation factors that challenge ‘pure’ DL, including cultural issues, such as how an ‘online group’ is conceived, not losing face with peers while pursuing an online discussion, respecting hierarchy and tradition, coping with uncertainty and integrating achievement and competition. Additionally, internalization factors such as variations in the form of self-directed learning, types of discussion and communication skills need to be contemplated. Therefore, by applying the precepts of complexity theory, one can conclude that although DL may resemble mainstream traditional education, its philosophical and theoretical underpinnings as well as its methods of practice are distinct.

In fact, from its outset, DL was differentiated from traditional education by catering to non-traditional learners. Professor Charles Wedemeyer, a leading theorist and visionary in the field, emphasised that traditional institutions of education hinder learning. He included K-12 schools, colleges and universities. ⁹ Wedemeyer posited that traditional education is comparatively new in human history, and conveys with it, anachronisms foisted from earlier eras. ¹⁰ One such relic still in effect is the commencement of the academic year in the fall (after the harvest), which essentially stems from the agricultural era. Similarly, offering instruction in standardized time intervals of 45 minutes is a holdover of the industrial era. ¹⁰ Non-traditional learning methods allowed by DL jettisons such restrictions. They augment responsiveness to the academic needs of the student, while reducing the cost associated with college education, and the time to obtain a degree.^11,12 They also serve the needs of individual students by offering them personal autonomy that leads to creative thinking and innovation. ¹³ Although many misconstrue DL as a form of traditional education enhanced by information and communication technologies, an efficiently administered DL activity offers the student with a more enriching learning experience than that of traditional education. ¹⁴ In spite of the inherent complexities, this raises the question: ‘can an easily adaptable DL framework be designed, which will provide a “simulacrum” of learning-experience encountered during live on-campus sessions, and can be efficiently implemented in the dissemination of learning objectives of a course, specifically in the undergraduate medical curriculum?’.

In this study, we endeavoured to tackle this question. Firstly, by employing 3 core educational theories of DL: A. model of community enquiry by Garrison et al;^15,16 B. principles of connectivism by Siemens;^17-20 and C. model of online collaborative learning by Harasim,^21,22 we derived an integrated model to blueprint a DL framework (Figure 1). This integrated model was improved by applying the precepts of Anderson’s DL model ²³ (Figure 1). Secondly, by exploiting the consequential and improved model, we derived all-purpose teaching principles that can be employed to strategize a guide-plan for any DL course (Figure 1). Thirdly, we expounded on the implementation of these derived all-purpose teaching principles during the COVID-19 lockdown using the exemplar of the fundamentals in epidemiology and biostatistics (FEB) course in our competency-based medical curriculum, where virtual live-sessions integrated into the DL framework were designed according to a blended approach that involved Gagne’s 9-events of instruction and Peyton’s 4-step approach ²⁴ (Table 1; Figure 1). We chose the FEB course to demonstrate the DL framework as this course requires dissemination of cognitive, practical and non-cognitive skills. The efficiency of the DL framework in the delivery of FEB course was evaluated using the first 2 levels of Kirkpatrick’s framework ²⁵ (Figure 1). Based on the obtained results, we conclude that the strategized framework presents a versatile approach for medical education faculty to deliver intending learning outcomes of a designated course in undergraduate medical education, through DL, especially during unprecedented circumstances as presented by the COVID-19 pandemia.

Figure 1.

The design of the DL framework. Note: Firstly, by employing 3 core educational theories of DL: A. model of community enquiry by Garrison et al;^15,16 B. principles of connectivism by Siemens;^17-20 and C. model of online collaborative learning by Harasim,^21,22 we derived an integrated model to blueprint a DL framework. This integrated model was improved by applying the precepts of Anderson’s DL model. ²³ Secondly, by exploiting the consequential and improved model, we derived all-purpose teaching principles that can be employed to strategize a guide-plan for any DL course. Thirdly, we expounded on the implementation of these derived all-purpose teaching principles during the COVID-19 lockdown using the exemplar of the fundamentals in epidemiology and biostatistics (FEB) course in our competency-based medical curriculum, where virtual live-sessions integrated into the DL framework were designed according to a blended approach that involved Gagne’s 9-events of instruction and Peyton’s 4-step approach (Refer to Table 1 for details). The efficiency of the DL framework was evaluated using first 2 levels of Kirkpatrick’s framework that is, perception and cognitive development.

Table 1.

Activities and timeframe pertaining to each step of the designed blended lesson plan of gagne and peyton, implemented in the first-year fundamentals of epidemiology and biostatistics course, medical students (N = 60), during COVID-19 distance learning lockdown, 2020. ^a

Step	Key event (allocated time)	Activity on Microsoft teams (MST)
1	Gain attention (5 minutes)	A pre-recorded ping was used as the sudden auditory stimulus.
		An epidemic curve relating to the current situation of COVID-19 was displayed on the screen via MS Teams (MST) Platform. Tutor posed the following critical-thinking question: ‘How would you describe the UAE curve compared to that of China?’.
2	Inform the student learner about the learning objectives (5 minutes)	Tutor presented the learning objectives for the session.
		Upon completion of this session, the student should be able to: 1. Recall the epidemiological terms endemic, epidemic, outbreak and pandemic. 2. Enumerate the use of SPSS generated graphs to describe a health-related state or event. 3. Recognize a descriptive study design by its description. 4. Describe the extent of a public health problem in terms of person, place and time. 5. Recognize an epidemic curve. 6. Differentiate between the shapes of epidemic curves.
3	Stimulate recall of prior learning (15 minutes)	Students participated in a group discussion (in sub-teams) on MST to recall the different types of epidemiologic graphs and the different epidemiological terms. The guide plan for discussing the graphs and terms were uploaded by the tutor on the learning management system (LMS) a week prior to the session. References to these aspects were also uploaded. c
4 b	Present content material (20 minutes)	Tutor presented through live-stream modality the current COVID-19 epidemic curves for selected countries d ; along with pertinent questions to be responded to or solved by students. The detailed steps involving the construction method for these curves using SPSS statistical package and interpretation of an epidemic curve were depicted in a flowchart and presented.
5 b	Provide learning guidance (15 minutes)	Tutor explained the descriptive epidemiologic measures used to describe the different epidemic curves and emphasized the essential ‘dos’ and ‘don’ts’ pertaining to each step of the SPSS analysis and interpretation.
		Tutor encouraged students to ask questions to clarify any doubts relating to the SPSS analysis and the curves.
6 b	Eliciting performance (30 minutes)	Students were distributed in sub-teams on MST according to selected countries. Predesigned and reviewed SPSS data was shared with the students. Students analysed the data and constructed the appropriate curve using SPSS modules. This was followed by intra- and inter-group discussions to construe and compare the plotted curves.
7	Provide informative feedback (20 minutes)	Tutor and students provided feedback to each other, using the Pendleton’s feedback model on A. the ease of applicability of SPSS to interpret/plot epidemiological data on a virtual platform; B. Pre-reading material uploaded on LMS; C. Guide plan to tackle epidemiological data using SPSS.
8	Assessing performance (25 minutes)	An assignment with specific deliverables in the form of formative assessment was uploaded on the LMS by the tutor prior to the session. Students addressed the deliverables and submitted the assignment on the LMS. The tutor assessed the student’s formative assignments according to a pre-set rubric shared with the students earlier. This step was concluded by revisiting the learning objectives and clarifying any doubts stemming from the deliverables.
9	Enhance retention and transfer (10 minutes)	At the conclusion of the session, the tutor addressed general misconceptions emanating from the assignment and the session
		Practice questions with specific deliverables (along the with model answers) aligned with the learning objectives were provided to the students by the tutor.
		Tutor concluded the session by summarising the key concepts delivered using the lesson plan, urging the students to go through the pre-reading material for the next session.

^aDistance learning was implemented in the Fundamentals of Epidemiology and Biostatistics course for the first-year medical students enrolled in the Bachelor of Medicine, Bachelor of Surgery (MBBS), College of Medicine, Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, United Arab Emirates. These students had no prior knowledge in Epidemiology and Biostatistics.

^bBlended step incorporating events of both Gagne’s and Peyton’s instructional models.

^cA dictionary of epidemiology. A handbook sponsored by the IEA. Sander Greenland, Miguel Hernán Isabel dos Santos Silva John M. Andrea Burón, 2014.

http://www.irea.ir/files/site1/pages/dictionary.pdf

^dhttps://www.who.int/emergencies/diseases/novel-coronavirus-2019/situation-reports#, retrieved 28 April 2020.

Methodology

Study landscape

Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU) is a new medical school located in Dubai Health Care City, the health care hub of the United Arab Emirates (UAE), with an undergraduate entry medical programme, where the curriculum is founded on a competency-based educational model, ²⁶ that spans over 6-years. The MBRU curriculum is divided into 3 phases. Each phase of the MBBS curriculum includes integrated courses and builds on the preceding one, such that the curriculum is a ‘spiral’, ²⁶ where with each successive encounter, concepts build on the previous one. The school caters to a diverse student population, emanating from different countries worldwide and diverse curricula, where ~75% of the students are females. ²⁶

Delivery of distance learning

DL was implemented in the FEB course in phase 1 of the MBBS curriculum during the COVID-19 lockdown. This FEB course provides the background for understanding, epidemiology and biostatistics to students who have no previous knowledge in these disciplines. Students are introduced to the basic principles and methods of epidemiology and biostatistics as applied to public health problems. Students also learn to recognize the role of quantitative methods in understanding clinical questions, especially in clinical decision-making.

The cloud-based software, Brightspace learning management system (LMS) by D2L (Kitchener, ON, Canada) was employed for dissemination of the course content, and for assessments. Instructional materials in the form of study-guide, pre-recorded power point presentations (which an initial survey indicated was the mode of content delivery preferred by the students(data not shown)), data sets and research articles to be reviewed, and formative assessments were uploaded onto the LMS by the concerned instructor(s) at least one-week prior to the delivery of a given session. The LMS was linked to an intelligent timetabling module by Wise Technologies Ltd (Ljubljana, Slovenia), which allowed students to simultaneously view the weekly schedule of the course-sessions with the intended learning outcomes for each session.

Due to the mandated COVID-19 lockdown and social-distancing, all on-campus sessions were substituted with virtual off-campus live sessions, organized using Microsoft (MS) Teams (MS Corporation, USA) compatible with Windows, Linux, macOS, iOS and Android operating systems. Prior to each live session, a reminder email was sent to all students registered for the course to ensure their participation.

Design of the virtual off-campus live sessions

The delivery of the virtual off-campus live sessions employed a blended pedagogical framework. ²⁴ The framework employed (A) Gagne’s 9 events of instruction, ²⁷ which is based on the information processing model of the mental events that occur when adults are presented with various stimuli; and (B) Peyton’s 4-step approach, ²⁸ an instructional model for teaching practical skills. From a behaviourist perspective, Gagne’s model advocates that, to assess whether or not learning has occurred, is to track an improvement in performance. In the FEB course, the intended learning outcomes are directed towards acquiring sufficient psychomotor skill, since a good theoretical knowledge of the biostatistics and epidemiology concepts alone will not suitably address the curriculum requirements. This, in turn, necessitates that a significant part of a session in the course should be earmarked for the students to apply the disseminated theoretical knowledge/concepts to analyse data using a specific statistical software package (hands-on exercises). Therefore, for the framework, we utilized both Gagne’s 9-events of instruction and Peyton’s 4-step approach, where specific steps from Gagne’s model were blended with steps from Peyton’s approach ²⁴ (Table 1). Also, this approach was previously implemented with success in the delivery of biochemistry (Banerjee et al, manuscript under preparation); molecular biology and genetics; ²⁴ and anatomy courses (Naidoo et al, manuscript under review) in the MBRU curriculum, yielding positive feedback from students.

Evaluation of distance learning

DL in the course was evaluated using the first 2 levels of Kirkpatrick’s 4-level evaluation framework. ²⁵ Level 1 of the framework evaluates reaction/perception, that is, ‘Did the student/learners enjoy the learning process?’. For evaluation of Level 1, a validated 18-item 5-point Likert scale (1 = strongly disagree; through 5 = strongly agree) questionnaire ²⁹ was adopted with minor modifications (Supplemental Table 1). Briefly, the survey interrogated students’ perception regarding DL administered in the FEB course during the COVID-19 lockdown, under 4 principal categories: computer expertise; flexibility of DL; usefulness of DL; and DL satisfaction; in addition to the following aspects: anxiety surrounding DL; preference of DL delivery mode; demographic data and operating system used for DL.

Level 2 of Kirkpatrick’s framework evaluates cognitive development, that is, ‘Did the learning occur?’. To address this, performance of the current cohort of students (course delivered through DL) in the summative assessment of the FEB course, was compared with that of the previous cohorts (course delivered employing no DL principles).

Data analysis

The statistical package, SPSS (V25, IBM Corp, NY, USA), was used for the data analysis. Exploratory factor analysis (EFA) ³⁰ was employed to validate the survey items for each subscale, thereby measuring each variable successfully. Reliability was determined using Cronbach’s alpha values, where a level of 60% or above was taken to denote strong split-half consistency. ³¹

Categorical variables were described by frequency, while the scores for the above-mentioned 4 principal categories were calculated by adding the items within each category, and then describing by the mean and standard deviation (SD). The percentage of the means of the scores was calculated according to the number of items per category and scaled according to the Likert system. The obtained score was tested for normality by using the Shapiro-Wilk test. ³² For comparisons of the scores by different demographical variables, student’s t-test was used. ³³ A P-value of .05 was used as a level of significance for all tests. ³⁴

Results

Demographics

Sixty students registered for the FEB course, of which 51 (85%) responded to the survey. Of the respondents, 37 (72.5%) were females and 14 (27.5%) males. The mean age of the respondents was 18.7 (2.0) years. The gender and age distributions of the respondents were representative of that of the whole class.

Design of the virtual off-campus live sessions

Eight virtual off-campus live sessions were delivered in the FEB course, employing the blended instructional model (Table 1). An exemplar of such a session, along with the architecture of the blended pedagogical model, is shown in Table 1. Since the delivery of the virtual off-campus live sessions represented a complete shift from on campus to DL modality due to the COVID-19 lockdown, we assessed the anxiety of the students registered for the FEB course associated with this sudden transition (Item #5 in the questionnaire shown in (Supplemental Table 1)). Majority (37, 72%) respondents expressed that they felt relaxed using DL, whereas 15 (28%) expressed feelings of anxiety. This difference in anxiety levels may be attributed to the different learning approaches availed by students. In fact, a pilot study at MBRU showed that 57% of our students availed a deep learning approach, 16% used a strategic learning approach and 27% used a surface learning approach. ³⁵ Pursuing studies through DL requires the execution of a demanding academic environment in the face of distress resulting from fear of failure, all of which are linked by necessity to learning approaches. ³⁶ A surface learning approach applies to passive learners who rely on rote learning to cope with tasks so that they can pass assessment. By contrast, deep or strategic learners are inherently interested in and enjoy learning, spurring them to seek, to understand meaning and have a genuine curiosity in the subject, which is essential to pursue studies through DL. ³⁷ However, this aspect of correlating learning approaches to anxiety associated with DL, requires further dedicated investigations, especially when students are exposed to a confined habitat, as created by the present COVID-19 lockdown and social-distancing directives.

Evaluation of Level 1 of Kirkpatrick’s framework

• I. Validity and reliability analysis

  EFA confirmed the validity and reliability of the survey items (Table 2). Majority of factor loading values were greater than 0.70, ³⁰ indicating that each item was positively correlated to the respective items within the 4 defined categories: computer expertise; flexibility of DL; usefulness of DL; and DL satisfaction (Table 2). The sample size was also adequate for all factors as the minimum Kaiser, Meyer, Olkin and Bartlett test that measure the sampling adequacy (MSA) ³⁸ and was found to be greater than 0.5 and statistically significant (P < .0001) (Table 2). Since Cronbach’s Alpha per each item was greater than 60%, factor reliability was also achieved (Table 2). ³²

• II. Reaction/perception

  Students’ perception towards DL was assessed through the questionnaire (Supplemental Table 1) that investigated 4 principal categories: computer expertise; flexibility of DL; usefulness of DL; and DL satisfaction, each of which comprised of several items (Table 3). The overall response for all 4 categories indicated that the respondents’ perception about the DL delivery mode in the FEB course ranged from ‘neutral’ to ‘agree’ (Table 3).

  Computer expertise assessed the role of DL in augmenting students’ computer skills and cyber resources. This category, which comprised of 4-separate items, yielded an overall mean score (SD) of 3.5 (1.1) (Table 3), indicating that the respondents’ computer know-how and proficiency were increased through DL. Also, the highest mean of 3.8 (1.1) was recorded for the item ‘My use of computers increases after taking this course’ (Table 3), among all the items in the 4 categories. This can most likely be attributed to the use of a statistical software package incorporated into the hands-on exercises during the virtual off-campus live sessions.

  The category, flexibility of DL, measured students’ perceptions regarding adaptability/ versatility of DL. In line, higher scores in this category were suggestive of more affirmative/positive beliefs about the versatility of DL. An overall mean score of 3.4 (1.2) (Table 3) in this category showed that students perceived DL to be convenient and flexible. Also, for this category, higher mean scores of 3.7 (1.2) and 3.8 (1.0) were recorded for the 2 items ‘DL allows me to work at home comfortably’ and ‘In terms of use of time and location, DL is flexible’, respectively (Table 3). Students pursing DL are often burdened with severe academic stress as they have limited access to conventional university health services. ³⁹ However, the fact that students felt that our DL strategy allowed them flexibility in-terms of time-management, simultaneously providing them with a comfortable learning milieu amidst the stressors created by COVID-19, proves the success of our DL pedagogical framework.

  The third category, that is,, usefulness of DL, encompassed items vis-à-vis students’ attitudes towards the usefulness of DL. Higher scores reflected positive beliefs regarding DL’s role in learning. In our study, we obtained an overall mean score of 3.2 (1.1) (Table 3) for this category, which attested to the students’ perception that DL augmented and enriched their learning experience.

  The fourth category of satisfaction regarding DL assessed the extent to which the students were gratified with DL. Higher scores indicated higher levels of contentment from DL. For this category, we obtained an overall mean score of 3.6 (0.8) (Table 3), alluding to overall student satisfaction pertaining to the delivery of the FEB course through DL. Also, for this category, high mean scores of 3.6 (0.8), 3.8 (0.9), 3.6 (0.7) and 3.7 (0.8) (Table 3) were recorded for 4 items: ‘The content of this course meets my expectations’; ‘I like the content of the course which draws examples from real life’; ‘I advise other students to take this course’ and ‘In this course I am pleased with the timely responses to my questions’, respectively. In spite of the absence of live on-campus sessions, the high scores in these items revealed that not only did our pedagogical framework disseminate the intended learning outcomes effectively and efficiently to the students, but the course team concerned were also able to provide students with effective feedback and study support.

• III. Evaluating the effect of gender, operating system (OS) and mode of DL delivery

  The perception of DL was similar for male and female respondents as gender was not statistically significant (P > .05) for each of the 4 categories (Table 4). Respondents using the iOS operating system emerged with better computer expertise (P < .01) and DL satisfaction (P = .02) as opposed to those using the Microsoft operating system. Moreover, the flexibility of DL (P = .02) and its usefulness (P < .01) were both statistically significant in favor of internet classes as the preferred DL mode of delivery (Table 4).

Table 2.

Results of exploratory factor analysis and reliability analyses from first-year medical students (N = 51), during COVID-19 distance learning lockdown, 2020. ^a

Category	Items in each category	Factor loading	KMO b and Bartlett test	Cronbach’s alpha c (%)
Computer expertise	This course helps me use the internet source more efficiently	0.88
	My use of computers increases after taking this course	0.74	0.55 d	65.7
	This course contributes to my knowledge of searching on the internet	0.84
	My computer knowledge increases with the course assignments and projects	0.68
Flexibility of distance learning	Distance education allows me to allocate my time better	0.83
	Distance education allows me to work at home comfortably	0.81	0.69 d	69.4
	In terms of use of time and location, distance education is flexible	0.68
	Distance education is appropriate to students with different learning capacities	0.56
Usefulness of distance learning	I believe distance education is useful	0.86
	A degree in distance education is as valuable as a degree in traditional education	0.77	0.81 d	82.5
	Distance education provides me with a valuable learning experience	0.91
	Distance education minimizes the inequalities in education	0.86
	Evaluation of the success in distance education is quite objective	0.39
Distance learning satisfaction	The student-centred instruction offered in this course through distance education is enjoyable	0.50
	The content of this course meets my expectations	0.70	0.71 d	65.7
	I like the content of the course which draws examples from real life	0.75
	I advise other students to take this course	0.78
	In this course I am pleased with the timely responses to my questions	0.60

^aFirst-year medical students enrolled in the Bachelor of Medicine, Bachelor of Surgery (MBBS), College of Medicine, Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, United Arab Emirates.

^bKMO, Kaiser-Meyer-Olkin.

^cShows the internal consistency.

^dP-value <.0001

Table 3.

Perception of the First-Year Medical Students (N = 51) Towards Items of Distance Learning and its Factors, during COVID-19 Lockdown, 2020. ^a

Category	Items in category		Mean (SD) b
Computer expertise	Overall, for the items in this category (4 items) This course helps me use the internet source more efficiently My use of computers increases after taking this course This course contributes to my knowledge of searching on the internet My computer knowledge increases with the course assignments and projects		3.5 (1.0) 3.6 (0.9) 3.8 (1.1) 3.3 (1.0) 3.3 (1.2)
DL Flexibility	Overall, for the items in this category (4 items) Distance learning allows me to allocate my time better Distance learning allows me to work at home comfortably In terms of use of time and location, distance learning is flexible Distance learning is appropriate to students with different learning capacities		3.4 (1.2) 3.0 (1.4) 3.7 (1.2) 3.8 (1.0) 3.2 (1.3)
DL Usefulness	Overall, for the items in this category (5 items) I believe distance learning is useful A degree in distance learning is as valuable as a degree in traditional education Distance learning provides me with a valuable learning experience Distance learning minimizes the inequalities in education Evaluation of the success in distance learning is quite objective		3.2 (1.1) 3.6 (1.2) 3.1 (1.2) 3.3 (1.1) 3.0 (1.2) 3.2 (0.9)
DL satisfaction	Overall, for the items in this category (5 items) The student-cantered instruction offered in this course through DL is enjoyable The content of this course meets my expectations I like the content of the course which draws examples from real life I advise other students to take this course In this course I am pleased with the timely responses to my questions		3.6 (0.8) 3.5 (1.0) 3.6 (0.8) 3.8 (0.9) 3.6 (0.7) 3.7 (0.8)

^aFirst-year medical students enrolled in the Bachelor of Medicine, Bachelor of Surgery (MBBS), College of Medicine, Mohammed Bin Rashid University of Medicine and Health

Sciences (MBRU), Dubai, United Arab Emirates. No data was missing, all questions were answered.

^bMean (standard deviation).

DL, distance learning.

Table 4.

Appraising the effect of gender, operating system and mode of distance learning delivery on perception of first-year medical students (N = 51) during COVID-19 lockdown, 2020. ^a

Variables	Categories	Computer expertise b	Flexibility of DL b	Usefulness of DL b	DL satisfaction b
Gender	Male	13.3 (2.2)	14.2 (3.8)	15.0 (4.8)	18.2 (1.4)
	Female	14.2 (3.1)	13.5 (3.4)	16.5 (4.0)	18.2 (3.1)
	P-value	.31	.53	.55	>.99
Operating system	OS Apple	14.9 (2.4)	14.0 (3.6)	16.6 (4.5)	19.1 (2.7)
	Microsoft	12.9 (3.0)	13.4 (3.4)	15.5 (3.9)	17.3 (2.5)
	P-value	<.01	.53	.38	.02
Internet classes c	No	14.1 (2.4)	12.9 (3.3)	14.7 (3.8)	18.0 (2.7)
	Yes	13.0 (7.7)	15.3 (3.4)	18.9 (3.6)	18.7 (2.9)
	P-value	.66	.02	<.01	.39
Hybrid class d	No	14.1 (1.9)	12.3 (3.7)	15.6 (4.8)	18.1 (2.7)
	Yes	13.9 (3.1)	14.1 (3.4)	16.2 (4.1)	18.3 (2.8)
	P-value	.87	.16	.71	.86
Web-base e	No	13.5 (2.7)	13.4 (4.1)	15.8 (4.0)	18.1 (3.3)
	Yes	14.2 (3.0)	13.9 (3.1)	16.2 (4.4)	18.3 (2.4)
	P-value	.45	.62	.64	.75

^aFirst-year medical students enrolled in the Bachelor of Medicine, Bachelor of Surgery (MBBS), College of Medicine, Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, United Arab Emirates.

^bMean (standard deviation).

^cDefinition of Internet classes: Internet classes are carried-out only online. Course communication, instruction, materials and assignments are completed online. Internet students complete their courses without on-campus meetings but may be required to participate in a proctored exam. Before participating in an online class, students are required to meet skills requirements.

^dDefinition of Hybrid classes: Hybrid class sections combine online learning with face-to-face instruction. All Hybrid courses will meet on-campus on specified days. The remaining instruction and course content will be delivered as an Internet course.

^eDefinition of Web-Based classes: Web-Based class sections primarily meet face-to-face on specified days, but do have a required online component, which requires students have Internet access as part of the course.

DL, distance learning; OS, operating system.

Evaluation of Level 2 of Kirkpatrick’s framework

Cognitive development

Figure 2 shows the mean and 95% confidence interval (CI) of the grades for the summative assessment of the FEB course delivered to first-year undergraduate medical students over the past 4 years at MBRU (since MBRU inception). The number of students enrolled in the MBBS programme in each academic year (AY) were: 54 (AY 2016/2017), 37 (AY 2017/2018), 63 (AY 2018/2019) and 60 (AY 2019/2020). The summative assessment for the current AY 2019/2020 cohort (which entailed the DL modality) was compared to that of the previous years’ cohorts (where, DL modality was not employed). All assessments were similar in content, weightage and duration, however. Analysis of Variance (ANOVA) was performed on all student grades for all 4 academic years. The results revealed that the performance of students in the AY 2019/2020 (which entailed the DL modality) was similar to that of previous years’ student cohorts (where DL modality was not employed) (P = .053). Students’ performance was therefore not affected by the sudden incorporation of DL into the curriculum.

Figure 2.

Mean grades of students for the past 4-years in the summative assessment of the FEB course. Note. Student cohort, who experienced the FEB course during the COVID-19 lockdown period through DL modality (indicated by red arrow in the figure) had comparable mean grade with those of the other cohort of students, where the FEB course was delivered through traditional methods (indicated in green).

Discussion

Our study presents a pedagogical framework for DL, which was disseminated efficiently and effectively during the unprecedented times of the COVID-19 lockdown period. To our knowledge, this is one of few studies in the literature, in which the pedagogical framework was strategized by employing learning theories that support DL. In order to derive the initial blueprint, we amalgamated the model of community inquiry by Garrison et al,^15,16 principles of connectivism by Siemens;^17-19 and C. model of online collaborative learning by Harasim^21,22 (Figure 1).

The community of inquiry theoretical framework characterises a process of constructing a deep and eloquent (collaborative-constructivist) learning milieu through the development of 3 interdependent elements – social, cognitive and teaching presence (Figure 1).

Social presence, as defined by Garrison, is ‘the ability of participants to identify with the community (eg, course of study), communicate purposefully in a trusting environment and develop inter-personal relationships by way of projecting their individual personalities’. ¹⁵ Teaching presence is the ‘design, facilitation and direction of cognitive and social processes for the purpose of realising personal meaningful and educationally-worthwhile learning outcomes’. ⁴⁰ Cognitive Presence is the extent to which learners are able to construct and confirm meaning through sustained reflection and discourse. ⁴¹ The community of inquiry model allowed us to efficiently integrate computer and cyber resources as a versatile medium for the delivery of our teaching principles (Figure 1). Further, the use of this theoretical framework allowed us to generate a ‘Community of Inquiry’ that is composed of instructors and students, the key participants in the educational process.

Next, we integrated precepts from connectivism of Siemens into our pedagogical framework.^17-19 Connectivism considers learning as a multifaceted process that is catalysed by technology and socialization. ¹⁸ The foundations of connectivism are fuelled by chaos, connectivity, complexity and self-organization theories. According to Downes, connectivism also finds its roots in connectionism, associationism and graph theory. ⁴² The principles of connectivism as per Siemens, ¹⁷ are as follows:

• I. Learning and knowledge rest in the diversity of outlook.

• II. Learning is a process of connecting specialized nodes or information sources.

• III. Learning may reside in nonhuman appliances.

• IV. Capacity to know is more critical than what is currently known.

• V. Nurturing and maintaining connections are needed to facilitate continual learning.

• VI. Ability to see connections between fields, ideas and concepts is a core skill.

• VII. Currency (accurate and up-to-date knowledge) is the intent of all connectivistic learning activities.

• VIII. Decision making is itself a learning process. Choosing what to learn and the meaning of incoming information is seen through the lens of a shifting reality. Although there is a right answer now, it may be wrong tomorrow due to alterations in the information climate affecting the decision.

Therefore, instructors in a connectivist learning milieu guide students to information (that can preferably be accessed with ease) and address queries as required, thereby encouraging students’ learning and sharing on their own accord, through which students inadvertently create a learning community. Students are also spurred to seek out information on the Web, critique the information, and share their findings and opinions within the learning community that they have created (guided by the precepts of community of inquiry theoretical framework discussed above).

Lastly, in order to derive our initial pedagogical framework, we also integrated principles stemming from the model of online collaborative learning by Harasim^21,22 (Figure 1). Online collaborative learning theory posits the role of peer discourse as a pivotal process to learning and demarcates erudition as intellectual convergence, accomplished through 3 progressive stages of group discourse: Idea Generating, Idea Organising and Intellectual Convergence (Figure 1). Idea Generating or the first phase, points to divergent thinking within a group; brainstorming, verbalization, creating information and thus sharing of ideas and positions on a specific topic or problem. Here, the role of the instructor is to facilitate idea generation and encourage active participation by all members of the group. Idea Organising or phase 2, is the initiation of a conceptual change. As students confront novel or singular ideas, which had been encountered or generated by their peers, they begin discussions in a more motivated way, thus clarifying and bundling these many ideas according to their relationship and similarities to one another. Idea organising behaviour demonstrates intellectual progress and the beginning of convergence, as students discuss and/or debate to select the strongest and weed out weaker positions (using such processes as referencing, agreement, disagreement or questioning). Intellectual Convergence or the third phase, is typically reflected in collective understanding, a collective position (including agreeing to disagree), or a mutual contribution to and construction of shared knowledge. Harasim’s collaborative learning theory is distinct from the constructivist learning theory; nonetheless, ‘collaborativism’ builds on constructivist (Garrison’s theoretical precepts) and connectivistic (Siemen’s theoretical precepts) learning theories by investigating and underscoring the role of discourse as theorized by Lev Vygotsky. ⁴³ In summary, the preliminary architecture of our teaching framework aimed, not only at the delivery of knowledge, but also intended to create a milieu of collaborative learning, concomitantly augmenting student autonomy; or as put better in the words of Holmes and Ramos ‘to help learners to assume greater control over their own learning, it is important to help them to become aware of and identify the strategies that they already use or could potentially use’. ⁴⁴

After the preliminary architecture was defined, we modified it using Anderson’s DL model.^23,40 Anderson proposes that DL has 2 major actors: learners and instructors, and their interactions with each other and with the content. Learners can interact directly with the content they find and any way they choose. However, they may choose to have their learning sequenced and directed through the assistance of the instructor. These multi-component learning environments allow for the learning of social skills, collaboration and the development of personal relationships among participants (Figure 1). However, the community binds learners in time (as happens during live on-campus sessions), and thus forces group-paced learning. This is the key reason why we employed dedicated instructional design strategies of Gagne and Peyton for our virtual live-sessions (Table 1). This allowed us to successfully create a ‘simulacrum’ of live on-campus sessions when delivering courses employing the DL pedagogical framework. Also, by integrating instructional design strategies, we were able to efficaciously create a learning environment in which students were not only able to interact with the content, but also could tackle questions/problems collaboratively. as well as receive timely and effective feedback from peers/instructors to identify their learning gaps and address them adequately – a learning experience that happens only in the classroom.

In order to appraise the efficiency of our DL framework, we implemented the framework in the delivery of the FEB course. This course not only involves dissemination of knowledge, in terms of different epidemiological and biostatistical principles, but also requires the student to gain practical skills, in terms of the use of statistical software packages to analyse data. The design of our virtual live sessions employed inputs from Peyton’s 4-step approach (Table 1), which allowed the students to rehearse and apply the disseminated concepts.

Didactic teaching, which students generally experience in the classroom, rarely facilitates potentiation of long-term memory. ⁴⁵ When students ‘go-over’ lecture materials or tutorial problems, maintenance rehearsal or rote memorization occurs, leading to ‘Surface-learning’. ⁴⁶ Our pedagogical framework facilitates rehearsal and accelerates ‘Deep-learning’, ⁴⁷ as the student repeatedly applies the concepts delivered in analysing data. The process is diagrammatically elaborated in Figure 3. This is one of the key advantages of our designed framework, which to our knowledge, has not been presented by any other study in the literature till date.

Figure 3.

The effect of the DL framework on internal cognitive process. The discussion during virtual sessions and the hands-on exercises integrated in the DL framework promote elaborate rehearsal, which facilitate Deep-Learning by potentiation of long-term memory.

We evaluated our pedagogical strategy employing Kirkpatrick’s framework. ²⁵ Evaluation of Level 1 of our framework was pursued using a questionnaire (Supplemental Table 1). In this study, we validated the individual items of the questionnaire using EFA (Table 2). In line, this questionnaire is a tool that can be used for the evaluation of perception for any DL framework designed and implemented in medical education.

As evident from the results, the DL pedagogical framework was positively received by the students (Table 3). Mean scores (Table 3) obtained for individual items (Supplemental Table 1) as those indicated below:

1. DL allows me to work at home comfortably,

2. In terms of use of time and location, DL is flexible,

3. The content of this course meets my expectations,

4. I like the content of the course which draws examples from real life,

5. I advise other students to take this course and

6. In this course I am pleased with the timely responses to my questions,

attested to the fact that students not only had an enriched learning experience through the DL framework, but also alluded to the detail that students engaged with the content in a way that fostered:

• I. Student-Student Interaction, which is a key component in online learning environments. In fact, modern constructivist and connectivistic theorists^15,17 stress the value of peer-to-peer interaction in investigating and developing multiple perspectives. In fact, student-led discussion groups and collaborative learning are ‘reciprocal teaching’ and help to develop communities of learning.

• II. Student-Content Interaction, a major component of formal education and DL. Through the discussion of real-life clinical problems (Table 1) individual needs of each unique learner was addressed. This is pivotal because in a given cohort, students, including ours, ³⁵ avail different learning approaches.

• III. Student-Teacher Interaction, a key component of both classroom and DL experience, was supported in our framework through live-discussion sessions organized using MS-teams. We ensured that the flow of communication during these sessions was less instructor-centred than in traditional classrooms. This augmented greater learner commitment and participation.

• IV. Instructor-Content interaction focuses on the instructor’s creation of content, learning objectives as well as units of study, complete courses and associated learning activities. In our framework, the students were provided with take home assignments and formative assessments, which allowed the instructors to monitor, construct and update course content resources and activities; and provide effective feedback.

• V. Instructor-Instructor, interaction allows instructors to develop professionally and support one another through communities. These interactions encourage instructors to take advantage of knowledge growth and discovery in their own subject area and within the scholarly community of teachers. In our teaching framework, the instructors communicated the obtained student-feedback to the other concerned instructors, such that instructors could effectively identify the learning gaps of the students and tailor the material to be disseminated suitably in future sessions.

• VI. Content-Content, interaction is a new mode of educational interaction in which content is programmed to interact with other automated information sources to constantly update. As the FEB course was delivered through our designed DL framework during the COVID-19 lockdown, a significant part of the instructional material was moulded in a way to inform students about the epidemiology of the pandemic (Table 1).

In summary, students’ perception indicated that we were able to effectively address our strategized teaching principles through the delivery of the intended learning outcomes. Also, perception of the students effectively attested to the success and versatility of our designed DL pedagogical framework.

Level 2 of Kirkpatrick’s framework evaluates students’ cognitive development following exposure to the DL pedagogical framework. To address that, in the summative assessment, we compared the performance of the students who experienced the course through the DL modality, to that of the students who experienced FEB through traditional teaching methods. As shown in Figure 2, performance in the summative assessment of the students experiencing DL was similar to those who were taught through traditional teaching methods. This showed that students’ performance was not affected by the sudden shift from traditional teaching methods to DL modality. Also, the learning process of the students was not impeded when our DL framework was adopted for course delivery; if such would have occurred, it would have affected their performance adversely.

Based on the above observations, the designed DL framework presented a superior pedagogical approach, which, when adopted by any medical school, will have long-term advantages. This aspect is important as a recent study indicated that temperature and latitude do not appear to be associated with the spread of COVID-19, instead school closures and other public health measures seem to have a positive effect, ⁴⁸ which reflects that it is pertinent that medical schools avail the DL modality to address learning needs of students. Further, this can also be elaborated using Bourdieu’s Theory of Practice. ⁴⁹ Bourdieu developed 3 intimately related concepts: field, capital, habitus (Refer to Figure 4 for details of the individual concepts). By applying Bourdieu’s Theory of Practice, the designed framework, when executed and integrated in a competency-based medical curriculum, will allow the medical school to function effectively to deliver medical education, even during unprecedented times as presented by the current COVID-19 pandemia, to attract high achieving students (academic capital), as well as allow a more effective delivery of courses with access to limited infrastructure and human resources (economic capital). This will augment the ranking of the medical school, which has adopted the teaching framework (symbolic capital), as well as facilitate the school in applying and receiving more funding or emoluments (economic capital) in the field of medical education and health professions education research. These aspects will impact the medical school’s values, primacies and curricula (habitus). Furthermore, all the above will be reflected in students the medical school will attract and train (habitus).

Figure 4.

Bourdieu’s theory of practice. The figure elaborates on 3 intimately related concepts: field, capital, habitus. The text box in blue elaborates how Bourdieu’s Theory of Practice when applied to the current context demonstrates the benefit of the DL framework being adopted by a medical school. The concept of the figure was derived from Brosnan C. Making sense of differences between medical schools through Bourdieu’s concept of ‘field.’ Med Educ. 2010; 44:645–652.

While strategising our DL pedagogical approach, we endeavoured to map it with different learning outcome frameworks. In this regard, we adopted the Dundee 3-circle outcome model and the ‘Scottish Doctor’ framework (Figure 5). The ‘Scottish Doctor’ framework is based on 12 – domains which are integrated in a 3-circle model. ⁵⁰ The rationale for choosing this framework is that the different aspects of healthcare do not function in isolation in this healthcare framework. As shown in Figure 5, our pedagogical approach attested to several aspects in the ‘Scottish Doctor’ framework either fully or partially. As the different domains are interlinked, we firmly believe that adoption of our DL pedagogical approach in a competency-based medical curriculum will attest to augmentation of patient care in the long run.

Figure 5.

The Dundee 3-circle outcome model and how the DL framework attests to several of the 12 competencies identified in the model.

Although our DL pedagogical framework has overarching and specific benefits, there are specific limitations in our study that need to be addressed.

• (1) Firstly, in this study we have evaluated only the first 2 levels of Kirkpatrick’s framework. However, Levels 3 and 4 of the framework, corresponding to ‘Did the intervention bring about a change in behaviour?’ and ‘Did the intervention influence performance?’, respectively, still need to be evaluated (Figure 1). However, in order to pursue these evaluations, long-term studies are warranted, where the DL framework needs to be adopted across courses in both preclinical and clinical phases of curriculum, following which the effect of this intervention has to be assessed using suitable tools. For assessment of Level 3 of Kirkpatrick’s framework, behavioural analysis of the ward rounds of students, exposed to the DL framework across different courses in the curriculum, needs to be pursued. In this regard, the methodology of Sanson-Fisher et al can be employed. ⁵¹ Evaluation of Level 4, that is, effect of the intervention on clinical practice, can be pursued using a strategy analogous to Seeley and Harding. ⁵² Here, one group of students (the experimental group) will be exposed to the DL pedagogical approach in different courses in the curriculum. A second group of students (the control group) will attend courses delivered using traditional teaching methods. Dedicated multiple-choice question assessments and objective structured clinical examinations (OSCE) will be used to evaluate knowledge and skills. Results will indicate if the experimental group shows improved post-intervention clinical practice compared with the control group.

• (2) Secondly, although our DL framework has been able to provide an enriched learning experience to students in the FEB course, will it be able to do the same across all the courses, specifically in those courses which involve the delivery of practical skills through laboratory sessions, such as anatomy and physiology. To address this issue, we have implemented the DL framework in the delivery of structure-function courses, where precepts of anatomy and physiology are disseminated. Results from this study are awaited and will form the basis of future scholarly communications.

• (3) Thirdly, one of the reasons we were able to successfully implement our teaching approach within a limited frame of time can be attributed to the presence of well-structured e-learning and cyber resources at our university, which we have alluded to, in the methodology. However, ‘Can our framework be adopted effectively by medical schools with limited access to such resources?’, is a question that still needs to be addressed. One of the cost-effective strategies for medical schools with limited access to e-learning and cyber resources, will be to implement social media applications (SMA) such as YouTube channels and WhatsApp discussion groups in the delivery of courses through DL modality. In fact, our previous study indicates that these 2 SMA are regularly used by medical students in their learning process. ¹⁸ Additionally, instructors with limited access to e-learning resources can employ virtual classroom modules such as WizIQ (https://www.wiziq.com), which provide flexibility of pricing.

• (4) The fourth aspect, which one needs to consider, is the issue of proctoring assessments in courses delivered through the DL approach. We were able to circumvent this issue with ease as the FEB course, in which the DL approach was adopted, employed open book assessment, which does not require strict proctoring measures. However, for courses where other forms of assessments are required to assess competency, instructors can use dedicated DL proctoring modules such as Examity and ProctorU, as these modules can be integrated easily with the LMS and gives instructors the ability to self-manage the scheduling of proctored events. Also, these modules offer different levels of proctoring service depending on the needs of the instructor and the type of exam being given. These levels include, AA – automated authentication only; Level 0 – authentication by a live person; Level 1 – recording and random review of tests; Level 2 – recording and review of all tests; and Level 3 – live continuous proctoring.

• (5) Lastly, it needs to be assessed if our teaching framework will be effective in the delivery of courses involving patient exposure. One of the ways to address this aspect will be to integrate the principles of telemedicine in our framework. A fundamental strategy for healthcare surge control is ‘forward triage’ or the sorting of patients prior to their arrival in the emergency department. ⁵³ Direct-to-consumer (or on-demand) telemedicine, a 21st-century tactic to forward triage that permits patients to be competently screened, is equally patient-centred and advantageous to self-quarantine, and it safeguards patients, clinicians, medical students and the community from exposure. It allows physicians and patients to connect 24/7 using smart devices. Respiratory symptoms, which may be initial signs of COVID-19 infection are among the complaints generally appraised with this approach. Health care providers can effortlessly obtain complete travel and exposure histories. Automated screening algorithms are usually built into the intake process, and local epidemiologic information can be used to standardize screening and practice patterns across providers. In line, if precepts of telemedicine are integrated into our framework, especially in our virtual live online sessions, students will be able to interact safely with patients, even during the COVID-19 pandemic. However, this requires further investigation.

Conclusion

In conclusion, in this study we present a DL pedagogical framework which can be effectively adopted for the delivery of courses in a competency-based medical curriculum. Our DL approach is supported by well-established theories associated with distance education. Further, our approach integrated instructional design models, which assisted us to create a ‘simulacrum’ of live on-campus learning experience in our virtual off-campus live sessions. The success of the DL approach has been demonstrated, using the exemplar of the FEB course during the COVID-19 lockdown period. Evaluation of the DL pedagogical approach was pursued using Kirkpatrick’s evaluation framework, which showed that the DL approach was not only positively received by students, but also contributed successfully to their cognitive development. Hence, our study presents an adaptable DL teaching strategy to address students’ learning trajectories during unprecedented times such as that created by the current COVID-19 pandemic.