Evaluation of Curricular Adaptations Using Digital Transformation in a Medical School in Arabian Gulf during the COVID-19 Pandemic

Abstract

Background:

Several institutions adopted innovative approaches to ensure continued learning for their students during the COVID-19 pandemic. All curricular innovations should undergo curriculum evaluation; hence, the objective of this paper was to share the salient features of evaluation using faculty and student's feedback on curricular adaptations implemented through digital transformation in a Medical School in Arabian Gulf during the COVID-19 pandemic, using a structured questionnaire.

Methodology:

After getting informed consent, feedback about acceptability and limitations regarding various aspects of curricular adaptations was obtained from students and faculty, using a structured and validated questionnaire. The response rate from faculty and students was 90% and 60%, respectively. The qualitative responses were analyzed using thematic analysis.

Results:

About 97% agreed that Modular Object-Oriented Dynamic Learning Environment, ZOOM, and Examsoft platforms were effective for curriculum delivery and assessment. 85% agreed that they were able to maintain online interactivity and 92% conveyed their willingness to continue to use these digital innovations even after the end of pandemic. “Lack of interactivity,” “missed clinical training,” “live sessions were more engaging than recorded ones” were the prominent themes emerged out of thematic analysis. All faculty and students expressed concern over the lack of clinical training involving real patients. All of them expressed appreciation to the university and faculty for their enormous efforts.

Conclusion:

Innovative ways should be considered to start clinical teaching with real patients, during pandemic. The learning outcomes of digital learning should be validated across all institutions. New indicators related to “digital learning” should be considered for accreditation of medical schools.

INTRODUCTION

COVID-19 outbreak had imposed quarantine and social distancing around the world, exerting huge burden on education, tourism, entertainment, and travel sectors. UNICEF declared that academic institutions were closed in >180 countries leading to disruption in the education of 98% of the global student population and several examinations were canceled worldwide.[1] Medical education was worst affected due to the inherent nature of the curriculum that demands close interaction with health-care teams, patients, and their families.[2] Worldwide, numerous medical schools struggled to adopt alternative approaches to ensure continued learning for their students.[3] This sudden transformation of large magnitude was indeed a huge challenge, in particular for institutions whose students were spread across several countries, as in this medical school in Arabian Gulf. This institution, with the support of its vibrant leadership and committed faculty, has ensured uninterrupted learning for their students through new innovative approaches.

It is recommended that all curricular innovations should endure the process of curriculum evaluation (CE) to confirm that the changes introduced are meaningful and valid. CE is defined as a method of appraisal of the part or the whole of curriculum, in terms of its merits and demerits. CE enables the curriculum to be focused in the desired direction.[4] In addition, CE also allows innovations to evolve and become more effective so that the changing needs of students and society are met.[5]

The CE are of two types: (i) formative (during the process of implementation) or (ii) Summative (at the end of implementation).[6] There are many models proposed for CE, for example, “The Eight Year Study” Evaluation Model; Provus' Discrepancy Evaluation Model, Stake's Congruence – contingency Model; Stuffluebeam's Context, Input, Process, and Product Model and so on.[7,8,9,10]

Feedback from students and faculty, who are the most important stakeholders in curriculum delivery, is an extremely important aspect of CE, in guiding us with useful data that can be employed for modifying the existing innovations if needed.[11]

Questionnaires are one of the simplest methods to obtain anonymous feedback from students on various aspects of the curriculum, namely the acceptability of design, instructional approaches, teaching-methods, nature of assessments conducted, quality of teaching materials, etc.[6,12]

Therefore, the primary objective of this paper is to share the salient features of evaluation using faculty and student's feedback on curricular adaptations implemented through digital transformation in a Medical School in Arabian Gulf during the COVID-19 pandemic, using a structured questionnaire.

METHODOLOGY

This medical school follows a student-centered and community-based problem-based learning (PBL) curriculum spanning across 6 years comprising of three phases: Phase I (1 year-foundation course), Phase II/pre-clerkship phase (Pre CP) (3 years – PBL modules) and Phase III/CP (2 years-clinical rotations).[13] The process of curricular adaptations through digital transformation was a complex procedure with a sequence of interdependent stages, as described below.[14,15]

Preparedness and leadership

Even before COVID-19 pandemic, this medical school had invested judiciously in upholding its resources sufficiently, in terms of organized administration, scholarly faculty, robust technology and active e-learning unit. The primary task ahead was to train the faculty, staff and students on how to use the technology for distance learning.[16] This technological shift was relatively easier for pre-CP because all tutorial sessions of “PBL” were already transformed into an “in class” digital format using the UNIO application. CP posed unique challenges, probably due to the multifaceted and practical nature of the training involved in imparting clinical competencies.

The decentralized leadership, manifesting across all levels, resulted in swift response, which was indispensable for proper crisis management.[17] Ensuring the safety of faculty, staff, and students, the medical school was closed from February 25, 2020 until further notice. Regular meetings were held between the leadership and other stakeholders to discuss the future of educational activities, keeping in mind different possible scenarios. Based on the scientific literature and discussion with Deans of other regional medical schools in the Gulf region, it was decided to convert “face-to-face” learning to “distance learning.”[2,14,17,18]

Selection and customization of tools

Multiple platforms like Big Blue Button, Modular Object-Oriented Dynamic Learning Environment (MOODLE), WIZiQ, DISPRZ, Google Meet, Microsoft Teams[18] and YouTube Live were shortlisted and tested for content delivery.[19,20,21,22,23,24,25] After careful consideration of the advantages and limitations of all the available options, MOODLE, which had been used at the college, was chosen as the central learning management system , which was then customized to suit the needs of on-going distant learning curricular activities. Educational Institutional License (annual) for ZOOM-a cloud-based software for teleconferencing and distance education, was purchased for uninterrupted transmission of online sessions. Online assessment software “ExamSoft” was procured for conducting secure online examinations. 'EXAMSOFT” has been shown to deter cheating, reduce faculty workload, and provide powerful data analysis of student performance, which might guide the enhancement of curriculum and eventually ease the accreditation process.[26,27]

Constant communication

Continuous and unambiguous communication was maintained between all stakeholders throughout all phases of curricular transformation, which was inevitable for its success. Formal communications were shared through official E-mails, whereas several WhatsApp groups were created specifically for each activity to support additional discussion on academics and logistics among students, administrators, faculty, and staff.[28]

Changes in curriculum delivery and assessment

None of the components from the existing curriculum were removed, instead, they were reorganized with a proposed plan to conduct the intensive course for 2–4 weeks on hands-on training for professional skills, laboratories and clinical skills when students physically return to the classes.[29,30] In CP, the students were divided into groups of 20–40 (based on the number of available faculty and nature of training required) under one or two tutors who discussed the same case through live virtual sessions, with all the students, to ensure uniformity of content delivery. It was also recommended to fill the gaps in clinical training, if any, during internship. Online objective structured clinical examination (OSCE) with special focus on history taking, clinical reasoning, communication, diagnostics, and interpretation skills was conducted for the end of rotation exams.[31] In addition, department of Family Medicine introduced team-based learning (TBL) using ZOOM breakout rooms function.[32] Table 1 shows an overview of curricular adaptations introduced during the COVID-19 pandemic.

Table 1.

Overview of curricular adaptations implemented during COVID-19 pandemic

	Preclerkship phase		Clerkship phase
	Educational activity	New alternative	Educational activity	New alternative
Changes in curriculum delivery	PBL tutorial sessions	MOODLE using big blue button plug in	Small group bedside teaching	Virtual, live sessions, with emphasis on case-based discussions and clinical reasoning
	Resource sessions and lectures	Videos of live lectures/audio recorded PowerPoint presentations uploaded to MOODLE TBL through break out rooms using zoom		Role play by tutors wherever required
Changes in skills training	Professional skills	Videos on professional skills were uploaded parallel to the PBL sessions	Clinical skills	Live sessions using zoom and microsoft teams Live videos of certain important procedures MSSC conducted streaming live sessions for batches of 25 students each
Changes in assessment methods	Summative assessment: theory	“Examsoft”†,‡	Summative assessment: theory	“Examsoft”†,‡
	Summative assessment: OSPE	“Examsoft”†,‡	Summative assessment: clinical skills	Virtual OSCE using “zoom”
	Formative assessment	Active participation during online tutorials, live demonstrations of professional skills, seminars, timely submission of digital assignments	Formative assessment	Active participation during simulation sessions, DXR modules, virtual clinical sessions, seminars, timely submission of digital assignments and updated portfolios

^†Software for conducting exam, ^‡SAQs/OSPE replaced by higher order MCQs/case cluster MCQs, DXR: Diagnostic reasoning software. PBL: Problem-based learning, MOODLE: Modular Object-Oriented Dynamic Learning Environment, MSSC: Medical skills and simulation center, OSCE: Objective structured clinical examination, OSPE: Objective structured practical examination, TBL: Team-based learning, SAQ: Short answer question, MCQ: Multiple choice question

Capacity building of faculty and students

All faculty (both full time and adjunct) were given hands-on training at the college campus, on utilizing the new technological methods of curriculum delivery through multiple mock sessions.[33] This training was given by a qualified and experienced e-learning team during the last 2 weeks of March 2020. The whole exercise was done in small groups following all COVID-19 guidelines, including hand sanitizing, masks, adequate social distancing, etc. Faculty who were not confident were encouraged to take tutorial sessions at college campus with the support of e-learning team, till they became competent.

Apart from hands-on training, 22 instructional videos were created by the e-learning team to address all queries and technical difficulties encountered by faculty and students. All students who were absent for the initial sessions were contacted individually through E-mail and/or phone, enquiring if they were facing any problem in logging in, or participating in the sessions. This distinctive approach, coupled with the conduct of mock tutorial sessions, alleviated the anxiety among students and brought down the overall absenteeism to <1%. Online faculty development workshops were held for clinical faculty on conducting virtual OSCE examinations using ZOOM breakout rooms.[34,35] The comprehensive data of digital transformation during the COVID-19 pandemic are shown in Table 2.

Table 2.

Overview of digital transformation during COVID 19 pandemic

Parameters	Foundation year	Preclerkship			Clerkship						Others
	Year 1	Year 2	Year 3	Year 4	Year 5			Year 6			M.Sc			Ph.D MM
					Medicine	OG	Pediatrics	Psychiatry	GS	FM	PM	LM	FM
Total number of students attended online learning	194	196	187	168	97	45	50	37	158	40			41
Lectures on PPT with audio recording hosted on one drive	70	137	118	146	60	35	40	13	45	48	33	9	FC	6
Live audio video tutorial on big blue button hosted on MOODLE		707	631	686							1			1
Online clinical/live teaching/resource sessions		84	73		105	35	30	10	21	94
Live sessions by MSSC faculty using big blue button on MOODLE					10	10	10		7
Number of students participated in MSSC sessions					97	46	51		82
Faculty/staff training
Number of faculty training sessions	2	2	6	7	1	3	1	1	5	11			11
Number of faculty trained for zoom/MOODLE		19	19		6	15	23	12	23	29			29
Number of faculty trained for MSSC					1	2	1		1
Number of support staff trained for MSSC					3	3	3		3
Other online activities: Modification/introduction of new portfolio by most of the departments

OG: Obstetrics and gynecology, GS: General surgery, FM: Family medicine, PM: Personalized medicine, LM: Laboratory medicine, MM: Molecular medicine, FC: Full course, MSSC: Medical skills and simulation center, MOODLE: Modular Object-Oriented Dynamic Learning Environment, PPT: PowerPoint

Evaluation of curricular adaptations

Evaluation is the integral component of any curricular innovation, and it is critical to obtain feedback to facilitate further modifications and reforms.[11] After getting informed consent, feedback about acceptability and limitations regarding various aspects of curricular adaptations through digital transformation was obtained from students and faculty, using a structured and validated questionnaire. The response rate from faculty and students was 90% and 60%, respectively. The qualitative responses were analyzed using thematic analysis, comprising of three steps, namely open coding, axial coding, and selective coding.[36] This stepwise process helps in identifying certain patterns and themes in the text to offer meaningful interpretation of the responses relevant to the objective of the research.

RESULTS

The detailed description of student feedback on various parameters (expressed as mean ± standard deviation), classified according to different years, is shown in Table 3. All the faculty agreed that MOODLE/ZOOM were effective platforms for conducting online sessions and they were satisfied with the faculty training. About 96% agreed that the communication was very clear, and the technical team was very supportive. About 85% agreed that they were able to maintain online interactivity among students, 74% expressed that group dynamics among students were adequate, 92% felt that the platforms were user friendly and conveyed their willingness to continue to use these digital innovations even after the end of pandemic.

Table 3.

Feedback from students on curricular adaptations during COVID-19 pandemic

Contents of the questionnaire	Mean±SD
	Year 1 (n=194)	Preclerkship (n=196)	Clerkship (n=351)	Overall score
The course outline (including the knowledge and skills) was made clear to me	85±20	66±27	74±23	75±23
My instructor (s) were available during a scheduled appointment to help me	86±19	75±27	79±23	80±23
My instructor (s) were enthusiastic about what they were teaching	84±20	73±27	79±22	79±23
Learning materials were of up to date and useful. (texts, handouts, references, etc.)	84±19	67±28	73±23	75±23
I was encouraged to ask questions and develop my own ideas	84±20	68±27	77±22	76±23
“The links between different learning activities, in my total program were made clear to me”	83±21	69±28	77±20	76±23
The technology and equipments used were appropriate and functional	83±21	68±29	78±21	76±24
This learning experience helped me to improve my ability to think and solve problems	79±25	74±27	76±23	76±25
This learning experience helped me to develop my skills in working as a member of a team	73±27	72±27	74±23	73±26
This learning experience improved my ability to communicate effectively	77±25	71±27	74±23	74±25
The software used for online teaching are appropriate and support the distance-learning process	77±24	72±28	81±19	77±24
The software used for online teaching offered good sound quality and clear audio	79±23	69±27	80±19	76±23
The software used for online teaching offer good visual quality with clear images, videos, and presentations	79±23	72±27	80±20	77±23
The software used for online teaching was relatively easy to use	82±21	72±27	83±19	79±23
The software used for online teaching was available whenever I needed with stable connection	80±23	68±28	79±20	75±24
The learning material (slides, presentations, videos, etc.) were well prepared and of high quality	79±22	68±28	77±21	74±23
The instructor was comfortable in using the technology to deliver the distance learning sessions	81±21	71±27	78±20	77±23
Appropriate assessment methods were adopted for online platforms to evaluate student performance in the course	80±22	68±27	76±22	75±24
There was appropriate support for using online platforms when facing technical difficulties	81±22	73±26	80±19	77±22
Clear information was provided on how to access the online platforms	82±21	75±26	82±19	80±22
Schedules for distance learning, virtual sessions and assessments were shared in a timely manner	75±26	70±27	79±21	75±25
I receive notifications when new course material or assessment was made available in the online platform	74±28	65±29	77±24	72±27
In general, the distance learning experience was useful and enjoyable	72±26	67±28	74±22	71±25
Average score of distance learning evaluation	79±23	70±27	79±21	76±23

SD: Standard deviation

About 77% of students perceived that the software used for online teaching were appropriate with good video/audio and supported the distance-learning process. About 79% felt that the software were relatively easy to use. 74% agreed that the learning materials provided through online were well prepared and of high quality. 75% have expressed that appropriate assessment methods were adopted and 77% were happy with the technical support provided for using online. In general, the distance learning experience was useful and enjoyable for 71% students.

The thematic analysis showed that around one-third of faculty faced problems in maintaining interactivity, “some students don't participate in the discussion and it's hard to keep track”, “I am having difficulty with group interaction”. “I can't agree completely with the use of this new way of teaching unless there is a problem like what we are facing. Otherwise teaching in a real class is much help full and better for strong interaction.”

When asked about what they liked most, many of them articulated that it saved a lot of time for them, “It was a new experience for distant learning, very comfortable about the free time (no time wasted in traffic to get into or out of the university),” “Saving time for the student” “clear and straight forward without time wasting,” “more time to study.” They were appreciative of online assessments and the simplicity of the software, “The mcq exams that we used to solve after each discussion were very useful,” “The applications used were easily accessible and with good voice/video connection,” “Simple to use.”

Most of them were grateful to the university for the digital initiatives, “I liked how our university tried the best to stay in touch with students, while some universities just stopped everything,” “The university did a great job.” Most of them liked the TBL virtual sessions and they believed that this was going to be the way of learning in future, “I think that it shapes the future of lectures and the way teaching should be. I believe that all mandatory lectures should be on these types of application to make it much easier to attend!!”

When enquired about what they disliked most, almost all of them unanimously expressed that they missed clinical training involving real patients, ”not going to the hospital and meeting patients made it very weird,” “the online classes were nothing like the actual hospital experience,” “sadly it feels unreal,” “we did not observe any real patient consultation,” “the fact that we did not attend the hospital.”

Many of them also said that they could not practice the clinical skills, ”I didn't had the enough time to develop my clinical skills,” “we were not able to practice things we learned in real life,” “we did not practice the clinical part of it at all: (!!!”, “only had a single chance to take history by myself...not enough for my clinical skills.” Another important theme which emerged was lack of sufficient interaction with faculty, ”how we couldn't interact with our doctors,” “lack of other communication elements like eye contact makes me uncomfortable to talk.” Some of them also felt that the sessions were too long when compared to regular sessions, ”the very long lectures and tutorial,” “very long,” “tutorials were long,” “Online studying is hard sometimes.”

Students and faculty offered many suggestions for the improvement of the distant learning experience. More than two third of the respondents felt that live sessions were more engaging than recorded ones, so they recommended more live sessions in future, ”doctors should hold some live online classes other than the recorded ones,” they also suggested various means for interaction such as WhatsApp and E-mail, ”doctors are available to talk to through WhatsApp and E-mail.” Many of them supported video chat for better understanding. The students welcomed sessions with more clinical correlation and case-based discussion, ”loved the clinical-based discussion. Loved the cases and approach (Hx, ddx, investigations etc)!,” “to make some videos on the common diseases how to do physical examinations with the relative findings.” Some of them advised that the distant learning to be more organized, ”i think more organization was required,” “getting the lectures as scheduled and getting them all at one time!”

In general, both students and faculty expressed their appreciation to the efforts of the University, ”thank you all for the hard work that you are doing for us,” “thank you for the department and the doctors for their efforts,” “thank you for making this happen and giving me a chance to graduate” and also shared their willingness to participate actively in online learning, if it is going to be continued beyond pandemic, ”I would like lectures to stay in an online format, as it is comfortable for both students and lectures,” “I think e learning should be available for students even without pandemic in place,” “to use e-learning more in the coming units,” “it is amazing, seriously futuristic.”

DISCUSSION

COVID-19 has been a catalyst for sweeping changes in medical education, which might not have happened otherwise. In our experience, the trust between leadership, faculty, and students during COVID-19 emerged as one of the strong motivating factors for the implementation of new methods. This was in contrast to the experience of few medical schools placed in gulf region.[37] The involvement of all stakeholders in the process of planning and implementation brought in more accountability and ownership.

The COVID-19 experience has transformed the dynamics of medical education worldwide, resulting in uncovering new potentials and deficiencies instantaneously. Many faculty and students believe that more interactive and engaging online sessions should be blended as a supporting tool to the existing curriculum in the coming years, specifically aimed at initial years of medical training.[37]

The unprecedented shift also presented with set of inherent challenges associated with technology, students, and faculty. Rapid shift to online learning required more technical assistance; hence, secretarial staff were trained as technical support team. When MOODLE was overloaded with academic activities, all the contents were transferred to “One drive” - a personal cloud storage, with dedicated space allotted for each phase, unit, and discipline.

Some PowerPoints could not play the audio due to noncompatibility and this was addressed by sharing an instructional video. Poor Internet connectivity during live sessions, was rectified by switching over to high-speed broadband connectivity. The technical team was part of all “WhatsApp” groups to provide swift support.

Perceived monotony and possible lack of interaction emerged as major limiting factor which is endorsed by many other authors. This is attributed to the fact that social interaction cocreate the identities of students and facilitate a sense of belonging; therefore, it is vital to sustain maximum interaction wherever possible, during online sessions.[38,39]

In our experience, online sessions became interactive on its own over a period as students and faculty became more confident in using the technology. In addition to this, the chat facility was activated across all platforms for better interaction. Many students were apprehensive and stressed about online assessment and the cancellation of mid-term examinations. The uncertainty regarding the final assessment caused a lot of stress among the students and this finding is in accordance with Lee, who documented that the students had the highest stress levels due to cancellation of examinations.[40]

Very few faculty encountered technical problems which were immediately addressed by the e-learning team either in person or over the video call. Faculty found it challenging initially, to evaluate the students in an online medium.[41] Even though some faculty were skeptical about the effectiveness of distance learning, as observed by other authors as well, they unanimously agreed that this was the best solution available in the given circumstances.[41]

The greatest difficulty was to find alternative methods for physical examination skills, which formed the backbone of medical training. During the final 2 years of medical school, the students undergo personalized training with advanced clerkship rotations, which has been disturbed by COVID-19. This disruption is attributed to many factors such as fear of transmission of the virus, inadequate personal protective equipment and testing kits for COVID-19 in many countries; the cancelation of all elective procedures and regular appointments.[42]

However, few medical schools, including ours, tried implementing innovative methods of assessment during COVID 19, to assess various aspects of medical training, including history taking skills, interpretation skills, clinical judgment skills, communication skills and so on.[43,44]

Sustainability of digital transformation

Our medical school has evolved enormously to handle similar challenges in future with more clarity and confidence. The future curriculum will embed more of “flipped classrooms” and TBL. Faculty development programs will be designed to focus more on emerging trends in digital learning. Online assessment will see paradigm shift and serve as a tool for formative assessment, including clinical skills. Updated digital infrastructure likely to motivate more students and faculty, possibly leading to better learning outcomes. Archival of the e-contents will serve as a digital library.

Suggestions for enhanced preparedness for “similar crisis”

Innovative ways should be considered to start clinical teaching with real patients. The effect of digital transformation on the learning outcomes should be validated across all institutions. New indicators related to “digital learning” should be considered for accreditation of medical schools.[37] Faculty training on evolving digital methods should continue post-pandemic also to keep up the quality of medical training. Extensive faculty training is recommended in the field of online assessment, including virtual OSCE, virtual viva, and medical simulation.[37]

CONCLUSION

Innovative ways should be considered to start clinical teaching with real patients. It is recommended to pay attention to already established priorities that have been mapped in a scientific manner to enable rationalized planning of medical institutions for post-COVID 19 training.[45]

Medical education units and curriculum committees should play a proactive role during any crisis and work swiftly in response to the changing needs of medical education. It is also recommended that all Medical institutions invest more money and training on digital assessment strategies.

Limitations of the study

This study reflects the experience of medical school and the perceptions of its faculty and students. Therefore, it should be interpreted with caution and the ideas shared here, may not be extrapolated to a wider population.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.