Experience of e-learning and online assessment during the COVID-19 pandemic at the College of Medicine, Qassim University

Abstract

Objectives

During the COVID-19 pandemic, academic institutions are promptly shifting all educational activities to the e-learning format. The present work describes concurrent procedures for online teaching and assessment performed at the College of Medicine, Qassim University, KSA. We also explored the impact of e-learning and assessment on the performance of students and faculty, and the challenges to their sustainability.

Methods

In this descriptive cross-sectional study, we recorded the number and duration of different online educational activities during the COVID-19 pandemic. Training sessions for various procedures of virtual classrooms and online assessments were organised for teachers and students. A newly established e-assessment committee arranged different online assessments. A comparison between the mean problem-based learning (PBL) grades of the same students was conducted either face-to-face or online. A student satisfaction survey and online staff focus group about the online learning experiences were conducted, and weekly staff perception reports were prepared. The results obtained were then analysed.

Results

A total of 620 virtual classrooms were successfully implemented over 994 h including theoretical lectures, PBL sessions, seminars, and tutorials. A significant increase in the mean PBL grades was observed for female students during the online sessions. Out of the basic year students and staff, 58.82% reflected their high satisfaction towards virtual classrooms, online assessment, and online workshops.

Conclusion

The present study elaborates on the benefits of e-learning and assessment. We observed higher student achievements and promising staff perceptions with obvious improvement in their technological skills. These findings support the shift towards future implementation of more online medical courses.

Introduction

The COVID-19 pandemic has led to the global disruption of medical education which necessitated working online.¹ Urgent response to the current situation required an increase in medical educators' awareness towards online teaching.² Several researches determined the effectiveness of digital technologies for life-long e-learning and continuous professional development.³ E-Learning has been established worldwide in response to the shortage of health educators and the need to switch into TEAL.^4,5 E-learning has several advantages, such as encouraging students for self-directed learning⁶ and updating the curricula.⁷

The College of Medicine at Qassim University, established in 2001, adopted the PBL system as an interactive educational strategy.⁸ The college shifted into digitalised PBL materials for one year, which was a good preparation for complete online PBL sessions. Recent technologies allowed the progressive innovation of e-learning.^9,10 Several studies have investigated the benefits of these technologies in medical education, especially the PBL system.^11,12 Official online platforms, mainly through the Blackboard learning management system (LMS) version 9.1 (Blackboard, Washington, DC), are used in Qassim university to conduct educational sessions including lectures, tutorials, PBL sessions, seminar presentations, and open discussion forums. Online formative assessments, through the Blackboard, were also performed. These assessments reflect the nature of online learning and give the students more responsibility for their learning.¹³ Online assessment allows the learners to demonstrate their capabilities in critical thinking and solving problems, which are the key benefits of shifting from traditional teaching to e-learning where the teacher is mainly a facilitator.¹⁴

The present study described the procedures performed to facilitate the urgent transition to e-learning and online assessment during the COVID-19 pandemic and to highlight its expected benefits and impact on student and staff satisfaction and performance. It also aimed to compare the scores of male and female students during both face-to-face and online PBL sessions, and to explore the expected challenges of this experience to sustain its future implementation after the COVID-19 pandemic.

Materials and Methods

Study design

This was a descriptive observational study conducted over 65 days. During this period, four weeks of PBL sessions were accomplished for the basic year students.

Study participants

The study collected data from all the undergraduate students, involved in this e-learning experience, of preclinical phase (n = 425) (male students = 252; female students = 173) and clinical phase (n = 249) (male students = 155; female students = 94), and 120 instructors (47 basic and 73 clinical). Data of 23 students were not included as they dropped out from both the basic and clinical phases. The aim and procedure of the study were clarified to the participating students and staff, and their consents were obtained. Students' identity will not be disclosed for ethical reasons. The confidentiality of the information obtained was maintained.

Setting

All theoretical activities, including lectures, PBL, tutorials, and seminar sessions, in College of Medicine at Qassim University were adapted to the e-learning modality through the Blackboard, version 9.1 (Blackboard, Washington, DC) and Zoom Cloud Meetings. The authors contributed to the rescheduling of timetables for all these activities. All practical or clinical sessions were postponed. Webinars about virtual classrooms and online assessments were presented for staff and students. An e-assessment committee was established for the first time consisting of thirteen members—nine basic scientists and four clinical staff. This committee formulated, revised, and uploaded the online assessments with their logistics and evaluated the results using item analysis.

Measures

The educational strategies for the live streaming sessions, during the COVID-19 pandemic, at the College of Medicine at Qassim University included lectures, PBL sessions, tutorials, and seminar presentations to achieve the course objectives. Comparisons of the mean PBL marks during control (face-to-face) and online (virtual) sessions were done for male and female students of the first and third years. Second-year students were excluded as they started a new block with the shift to e-learning. The students completed an online satisfaction survey on their perceptions of the e-learning experience, with a 5-point Likert scale. Weekly reports concerning staff perceptions on the effectiveness of live streaming activities were collected with a 3-point Likert scale.

The Cronbach's alpha test was used for testing the internal consistency and reliability of the students' and staff's perceptions. Kendall's tau B, a nonparametric measure of association that exists between two variables, was used to test the correlation of the items of the students' survey.

An online focus group for the staff, using the FocusGroupIt software (https://www.focusgroupit.com/), was performed. The questions were presented in the form of a SWOT analysis (Figure 1). The group was composed of one moderator, one observer, and seven participants (the supervisors of basic sciences departments and one radiology staff). The two-hour-long online synchronous focus group discussion was recorded. The analysed data were reviewed and interpreted by an independent investigator. The data were analysed using the Statistical Package for the Social Sciences (SPSS) software, version 21 (IBM Corp., Armonk, NY). Descriptive statistics (percentages, mean, and standard errors of the mean) were used to describe the quantitative variables with their analysis through paired-samples (to compare the male and female mean PBL marks) and independent t-tests (to compare the mean marks of the face-to-face and online PBL sessions). A p-value of <0.05 was considered significant.

Figure 1.

e-Learning and online assessment –- SWOT analysis.

Data collection

Weekly reports though the official learning management system—Blackboard—regarding the number, duration, and modality of different educational activities including the live streaming sessions and students' attendance rate were collected from the course organisers in coordination with the e-learning unit, phase coordination, and e-assessment committees. Evaluation of students during their e-learning experience was done based on their assessment during the online PBL sessions.

Results

Table 1, Table 2 present the details regarding the virtual classrooms, conducted mostly through Blackboard, of the preclinical and clinical phases. They represent the numbers of e-learning activities successfully implemented.

Table 1.

Virtual classrooms for the students of the preclinical phase.

Students' Year	Live streaming		Duration	Number of Students	Educational Activity
	Bb	Zoom	(hours)		Lecture	PBL	Seminar
First	114	27	231	3634	37	91	13
Second	108	7	216	3512	31	72	12
Third	81	34	189	1979	19	84	12
Total	303	68	636	9125	87	247	37

Bb: Blackboard.

Table 2.

Virtual classrooms for the students of the clinical phase.

Students'Year	Live streaming		Duration	Number of Students	Educational Activity
	Bb	Zoom	(hours)		Lecture	Tutorial	Seminar
Fourth	82	24	169	3805	79	27	0
Fifth	99	39	189	5031	159	0	17
Total	181	63	358	8836	238	27	17

Bb: Blackboard.

Significant increase in the mean PBL marks of the female students of both first and third years was observed during the online sessions than in the face-to-face sessions of the relevant year. Additionally, there was a significant increase in the mean PBL marks of the female students of the first and third years than that of the male students of the relevant year in both face-to-face and online sessions (Table 3, Table 4).

Table 3.

Comparing the mean PBL marks of the first year students.

PBL sessions	First Year Students
	Male	Female
N	84	56
Control PBL (Face-to-face)	4.41 ± 0.09	4.72 ± 0.07b
Online PBL (Virtual classes)	4.54 ± 0.08	4.94 ± 0.02a,c

^aSignificant to control female.

^bSignificant to control male.

^cSignificant to online male (paired and independent t-test).

Table 4.

Comparing the mean PBL marks of the third year students.

PBL sessions	Third Year Students
	Male	Female
N	78	59
Control PBL (Face-to-face)	4.39 ± 0.1	4.54 ± 0.09b
Online PBL (Virtual classes)	4.73 ± 0.08	4.88 ± 0.03a,c

^aSignificant to control female.

^bSignificant to control male.

^cSignificant to online male (paired and independent t-test).

Two hundred and fifty students of the preclinical phase (58.82%) completed the perception survey. Students' satisfaction towards the new modality of e-learning and online assessment was reported (Table 5). The items covered in the students' survey included the following: the success of e-learning in compensating for the urgent suspension of face-to-face teaching during the COVID-19 pandemic, efficiency of instructions announced before the online teaching, staff's resistance and experience in e-learning requirements, and effectiveness of online assessment in testing their knowledge and skills levels. The Cronbach's alpha test performed for all items of this survey resulted in an overall score of 0.67. Kendall's tau B was used to test the correlation of these items. The correlation coefficient ranged from 0.134 to 0.394.

Table 5.

Students' survey for e-learning and online assessment.

#	Question	N	SA		A		NAD		D		SD
			n	%	n	%	N	%	n	%	n	%
1	E-Learning compensated the suspension of face-to-face teaching due to the COVID-19 pandemic	250	92	36.8	103	41.2	30	12.0	19	7.6	6	2.4
2	Educational activities got enough time during the online teaching	250	75	30.0	97	38.8	43	17.2	25	10.0	10	4.0
3	Staff have enough experience in e-learning requirements	250	19	7.6	81	32.4	74	29.6	62	24.8	14	5.6
4	Interaction during online session was satisfactory	250	50	20.0	95	38.0	52	20.8	38	15.2	15	6.0
5	Announced instructions before quizzes are useful and sufficient	250	96	38.4	91	36.4	31	12.4	20	8.0	12	4.8
6	Online assessments are effective to test the knowledge level	250	66	26.4	84	33.6	41	16.4	34	13.6	25	10.0
7	Do you suggest online teaching for some theoretical courses?	250	149	59.6	49	19.6	13	5.2	14	5.6	25	10.0

SA: strongly agree, A: agree, NAD: neither agree nor disagree, D: disagree, SD: strongly disagree.

Open-ended comments were received from 225 out of 250 students (90%), which were classified into two categories—one for teaching activities and the other for online assessment. Eighty-seven per cent (n = 195) of the students reflected their enthusiasm towards the e-learning modality: for example, ‘Very amazing; online teaching experience is the future of learning’ and ‘Online lectures are super-satisfying’. They expressed their satisfaction in having the same staff delivering the lectures for both male and female students. E-Learning provided a chance for the shy students—as they could participate freely through online chatting— and motivated them for verbal discussions. They expressed satisfaction with the rescheduling of educational activities and their allocated time and duration; for example, ‘Modification of the timetable allows more time for educational and recreational activities, thus allowing to live like a normal human’. The students expressed that some of the staff lacked adequate experience in conducting open discussions during online teaching. Seventy per cent (n = 158) of the students reported that frequent online quizzes motivated them to study; for example, ‘Online quizzes are very helpful to improve my grades’. They appreciated the conduction of the mock quiz; for example, ‘Mock exam was very helpful for training before the online exams’. Electronic assessment ensured fewer errors carried out by the students while filling out bubbles in their answer sheets. Lastly, they recommended that some courses could be implemented online in the future. Staff satisfaction in virtual classrooms was also observed (Table 6, Table 7), which reflected that the live streaming sessions were very effective.

Table 6.

Staff satisfaction in virtual classrooms of the preclinical phase.

Students' Year	Number of educational activities	Very Effective		Somewhat Effective		Not Effective
		N	%	N	%	N	%
First	141	118	83.69	22	15.60	1	0.71
Second	115	104	90.43	9	7.83	2	1.74
Third	115	100	86.96	14	12.17	1	0.87
Total	371	322	86.79	45	12.13	4	1.08

Table 7.

Staff satisfaction in virtual classrooms of the clinical phase.

Students' Year	Number of educational activities	Very Effective		Somewhat Effective		Not Effective
		N	%	N	%	N	%
Fourth	106	89	83.96	15	14.15	2	1.89
Fifth	138	133	96.38	5	3.62	0	0.00
Total	244	222	90.98	20	8.20	2	0.82

The online focus group discussion was categorised into four themes based on the SWOT analysis. Concerning the strengths, most of the staff agreed that the new arrangement of educational activities efficiently compensated for the suspension of face-to-face teaching. They were satisfied with the university's efforts to improve staff awareness regarding virtual classrooms and online assessment through webinars. The majority of them appreciated the tremendous shift to implement online summative assessments; for example, ‘Online assessment assured the staff about students’ achievement of learning outcomes' and ‘Conduction of live oral exams based upon clear standardised checklist was highly effective’. The concerns expressed by the participants included some staff's limited online teaching experience and the insufficient number of IT technicians which interferes with proper digitalisation. Regarding the opportunities that could be gained, the staff acknowledged the constitution of the e-assessment committee in the focus group; ‘Such committee was an urgent requirement to tackle the full responsibility for online assessments’. They reflected the improvement of their expertise towards the novel pedagogical e-learning techniques and their enthusiasm to construct online courses; for example, ‘Adoption of electronic courses allows better arrangement of educational activities and gives more chances for students’ self-directing learning’. Two staff members reported the inconvenience of some students in taking the online quizzes, especially during the initial period, while the others specifically talked about those who live in distant rural areas with insufficient internet coverage. Four staff members emphasised the limited capabilities of the official version of Blackboard, particularly in conducting the morning sessions.

Discussion

Advanced technologies emerged during the COVID-19 pandemic to sustain world productivity. The Horizon 2020 Teaching and Learning report highlighted the role of advanced technology in medical education. The present study represents the major change in the educational culture. E-learning was highly beneficial for competent educators as it decreased the needs for in-class attendance.¹⁵ In agreement with McCoy et al.,¹⁶ students and staff reflected that live streaming lectures efficiently compensated the suspension of face-to-face teaching and provided more chances for open discussions. Nomination of the same staff to virtually present each topic for both male and female students encouraged more peer sharing and competition among the students. The rescheduled educational activities were more convenient to the students, as reflected in their attendance. Recent technologies resolved the lack of physical attendance and increased learning effectiveness.^17,18 The Horizon 2020 report highlighted the effectiveness of online teaching in overcoming the restrictions such as shortage of venues for large group lectures. However, one of the main challenges of e-learning, as reflected in the current work, is in teaching the psychomotor, practical, and clinical skills efficiently. Murphy¹⁹ recently reported that most medical schools suspended the clinical settings during the COVID-19 pandemic. This could be overcome by using virtual-reality simulators.^20,21

Unfortunately, most of the students were unsatisfied with how some staff members practiced e-learning. A previous study performed in a similar culture—United Arab Emirates—observed that the teachers felt worried about the shift into a new educational strategy. Psychological assurance was recommended to encourage them to deal with the unknown consequences. Multiple webinars about proper virtual classrooms and peer sharing of experiences between the staff members solved this problem. Goh et al.¹⁸ claimed that live streaming applications will improve the technological skills of the educators. PIVOT MedEd²² appreciated the free e-learning webinars for health professional educators worldwide and the national coordination between medical schools in sharing such training courses. Successful collaborative online learning demanded the support of IT technicians, as previously reported.^16,23 The Cronbach's alpha test implied that the survey tool had a good level of internal consistency and reliability for both the students' survey and staff's overall satisfaction in the live streaming experience. The correlation coefficient indicated that the items in the students' survey were well correlated. However, the application of Kendall's tau B test on the staff perception was non-feasible since it measures the association between two variables, unlike the current study which focused on the staff perception on the effectiveness of virtual classrooms.

The higher achievement level detected in the mean marks of online PBL compared with face-to-face sessions could be attributed to the easier access to the explanation of the phenomena. The students' assessment during the PBL sessions in this work was based upon their commitment, team spirit, interaction with peers and tutors, presentation skills, and ability for brain storming and analysing the phenomena. The students' assessment during the PBL sessions was based upon their performance during the session rather than their achievements through other summative assessment methods such as multiple choice questions which are associated with higher chances of cheating.²⁴

These findings are in agreement with the previous study²⁵ which observed that online PBL enhanced critical thinking and fulfilled the intended learning objectives. Therefore, online PBL could enhance the metacognitive skills, ability to solve problems, and team working. Collaborative interaction in the online environment helped enhance peer sharing.²⁶ After the SARS epidemic, one medical school in China adopted online PBL as an educational strategy for the subsequent years.¹ The higher PBL scores in female students could be attributed to the difference in the style of thinking, learning, and capabilities of problem-solving.²⁷ Makonye²⁸ detected that female students have more ability for knowledge perception and reflection of their own ideas, and a higher competitive attitude. Additionally, this gender variation in PBL scores could be due to a higher commitment of female students in attending different educational activities.²⁹ However, Ajai and Imoko³⁰ observed equal performance of male and female students in PBL sessions, and recommended further studies to examine the underlining causes.

Validity and reliability of assessment should be established to ensure students' achievement of the learning objectives.¹⁴ The ability to solve any technical obstacle met during the online mock exam helped the e-assessment committee to manage the subsequent exams appropriately.²⁵ Results of the online assessments and their item analysis represented evidence-based high-quality evaluation. This perception was supported by Jawaid et al.³¹ and Bandele et al.³² who reported that students expressed a more positive attitude towards online exams. Similar findings were also observed by Martin et al.³³ who emphasised that ideal assessment was based upon optimal evaluation strategies. The online tests were revised by the e-assessment committee, in coordination with the supervisors of the relevant departments, to estimate sufficient time for each exam. Redistribution of students' grades towards more objective assessment strategies such as PBL, seminar presentations, and oral assessments was kept in consideration. This minimised the subjectivity of grading, as expressed in the feedback gained from the focus group, and discrepancy in students' grades, in agreement with Ozden et al.³⁴. The procedures implemented during the online exams to minimise the chance of cheating and unauthorised collaboration with peers included scheduling of brief exams of not more than 30 min with a timer set for the whole exam and also for each individual question. Formulation of a considerable number of scenario-based questions was implemented focusing on higher-order critical thinking, following the Bloom's taxonomy.³⁵ Questions and answer choices were randomised; each question was presented on a separate page with forced completion of these questions—students were not allowed to return to the previously-submitted questions—in agreement with Fontanillas et al.³⁶. A backup version of the questions with the same difficulty index was prepared for those students who experienced technical difficulties during submission. The answers were not displayed to the students until the examiner's permission was obtained and all participants had completed answering.

The potential use of TEAL in medical education—the future learning era—is expected after the resolution of the COVID-19 pandemic. Analysis of the benefits of e-learning in the present study could help the decision-makers of educational policies and committees of curriculum reform to implement them in the future. Goh and Sandars¹¹ pointed out that the medical educators worldwide, being deeply involved in the current tremendous shift towards e-learning, have to adapt to the current major educational challenges. Appropriate e-learning resources should be available to guarantee the implementation of these enormous changes. More use of technological tools will enable the medical schools to establish the active process of e-learning.³⁷ The Horizon 2020 report focused on ‘learning engineering’ such as the virtual reality simulators to facilitate the rapid development of teaching and learning. Online courses form one of the key success factors to achieve the program learning outcomes of medical graduates.

Conclusion

The current work explored the benefits of the shift towards e-learning and online assessment, which is a promising strategy with great educational potentials, after the WHO's declaration of the COVID-19 pandemic. This successful digital learning environment was observed in terms of student and staff satisfaction, achievement, and improvement of technological educational skills.

Recommendations

The present study recommends a sustained monitoring and updating of the e-learning resources, particularly the official LMS, and availability of sufficient number of information technology personnel. Multiple webinars and workshops suggested increasing the student and staff awareness of online teaching and assessment via improving the current faculty development program. Furthermore, the adoption of teaching through complete and partially online courses, and a total shift from physical attendance for PBL sessions to online ones are encouraged in the future.

Availability of data and material

All data are available from the corresponding author upon reasonable request.

Source of funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Conflict of interest

The authors have no conflict of interest to declare.

Ethical approval

Ethical approval was taken from the ethical committee of College of Medicine, Qassim University (ethical approval number S7564829, dated 02/07/2020).

Consent

Written informed consents were obtained from all the participants.

Authors contributions

WAA conceived and designed the study, conducted research, provided research materials, and collected and organised the data. AE and AES analysed and interpreted the data. All authors wrote the initial and final drafts of the article, and provided logistic support. All authors have critically reviewed and approved the final draft and are responsible for the content and similarity index of the manuscript.