Skip to main content
NCBI home page
Elsevier - PMC COVID-19 Collection logoLink to Elsevier - PMC COVID-19 Collection
. 2021 Oct 25;108:105183. doi: 10.1016/j.nedt.2021.105183

Application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic: A systematic review

Xiaonan Hao 1, Xin Peng 1, Xinxin Ding 1, Yuan Qin 1, Miaohua Lv 1, Jing Li 1, Kun Li 1,⁎
PMCID: PMC8545701  PMID: 34741918

Abstract

Background

Due to the rapid spread of coronavirus disease 2019 (COVID-19) around the world, the World Health Organization (WHO) declared it a global pandemic on March 11, 2020. This declaration had an unprecedented impact on health profession education, especially the clinical clerkship of nursing and medical students. The teaching hospitals had to suspend traditional bedside clinical teaching and switch to digital education.

Objective

To systematically synthesize the available literature on the application of digital education in undergraduate nursing and medical interns during the COVID-19 pandemic.

Design

A systematic review informed by PRISMA guidelines.

Data sources

Five electronic databases were systematically searched: PubMed, Embase, MEDLINE (OVID), CINAHL and the Cochrane Library.

Review methods

The retrieved articles were screened at the title, abstract, and full text stages. The Mixed-Methods Appraisal Tool (MMAT) was used to assess the quality of quantitative and mixed-method studies. Then, two reviewers extracted the quantitative data of the included studies.

Results

A total of 4596 studies were identified following a comprehensive search, and 16 studies were included after removing duplicates and screening, which focused on undergraduate nursing students (3 studies) and medical students (13 studies). We found that the standalone digital education modalities were as effective as conventional learning for knowledge and practice. Different educational technologies have different effects on the knowledge and practice of interns.

Conclusion

Digital education plays a significant role in distance training for nursing and medical interns both now and in the future. The overall risk of bias was high, and the quality of evidence was found to be variable. There is a need for further research designing more quasi-experimental studies to assess the effectiveness of standalone digital education interventions for the remote training of nursing or medical interns to be fully prepared for emergencies.

Keywords: Digital education, Undergraduate, Nursing students, Medical students, COVID-19, Clinical practicum

1. Introduction

Due to the rapid spread of coronavirus disease 2019 (COVID-19) around the world, the World Health Organization (WHO) declared it a global pandemic on March 11, 2020 (Dao et al., 2021). As of June 2021, there have been more than 3.8 million deaths and 170 million confirmed cases in 222 countries and territories. The pandemic has caused enormous economic and social upheaval internationally; furthermore, it has had an immeasurable impact on the delivery of health profession education worldwide (Hodgson et al., 2021; Seymour-Walsh et al., 2020a). Therefore, to protect students' education and wellbeing, the Ministry of Education of the People's Republic of China issued a notice postponing the start of the school year, and the Association of American Medical Colleges (AAMC) recommended that health professions schools pause all clinical practicums (Harries et al., 2021).

Clinical instructional experiences and exposure to real patients are essential elements of nursing education and an indispensable part of personal medical education and development (AlThiga et al., 2017; Mandan et al., 2016). However, with the rapid spread of COVID-19, nationwide lockdown restrictions have had a strong impact on medical and nursing students' access to patients as well as medical staff (Dost et al., 2020). The resulting passive form of the clinical practicum has led to issues in the quality of medical education. Thus, the faculty preceptors had to urgently fill the gaps in the clinical practicum and complete the mission of cultivating qualified future health care workers. Given that the pandemic is a moment to promote the culture of professionalism and altruism and can present a once-in-a-lifetime disaster response learning experience for medical and nursing students, some medical schools encouraged their students to work on the front line during the COVID-19 pandemic (Lee et al., 2020; Leigh et al., 2020). However, shortages of personal protective equipment (PPE), limited availability of COVID-19 testing and imperfect infection control schemes have all been major safety concerns (Wang et al., 2020). Thus, digital education without security risks may be an alternative to clinical education that can be implemented even during the lockdown (Park et al., 2020).

Digital education is the act of teaching and learning via digital technologies (Car et al., 2019), including diversified teaching methods such as offline learning, online learning, serious games, mobile learning or virtual reality (VR) (George et al., 2014; Kononowicz et al., 2019). Different digital education methods have different characteristics, mainly embodied in technical preparation, measurement tools, delivery settings and learning approaches (Kyaw et al., 2019). Over the past 50 years, digital education has been widely used in health professional education (Triola et al., 2012), and may provide a more flexible, accessible and affordable alternative to traditional learning (Tudor Car et al., 2019b). Numerous published studies have evaluated the potential benefits of digital education for health professional education (Cook et al., 2008), for instance, without the limitations of time, resources and geography (Brusamento et al., 2019). The World Health Organization (WHO) has also affirmed its significant contribution to filling the gap between low- and high-income regions in medical education (Semwal et al., 2019).

Health professional education has been severely influenced by the novel coronavirus, but this emergency has fortunately provided an impetus for the innovation of teaching formats (Dedeilia et al., 2020; Seymour-Walsh et al., 2020b). Distance training has undoubtedly been one of the feasible teaching reform measures. A previous review evaluated the effectiveness and potential benefits of digital education in health professional education (Curran et al., 2017), but it was conducted before the COVID-19 pandemic. Therefore, the authenticity of the results may inevitably be interfered with by traditional offline teaching. In addition, some reviews synthesized data on medical or surgical education during the period of COVID-19, but they never took nursing education into consideration (Dedeilia et al., 2020; Wilcha, 2020). To the best of our knowledge, no systematic review has examined digital education as an alternative to training medical and nursing interns during the novel coronavirus pandemic. Therefore, this review filled a gap in the literature. We not only assessed the effectiveness of digital education but also evaluated its impact on the knowledge, attitudes, skills, and satisfaction of nursing and medical students.

2. Methods

2.1. Design

This systematic review is a critical appraisal of all available evidence to evaluate the substitutability of digital education to bedside clinical practicum during the pandemic. In this study, the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) was adopted to inform reporting of the search and results, owing to its clarity and transparency (Moher et al., 2009).

2.2. Data sources

A systematic search of the following five electronic databases was performed in April 2021: PubMed, Embase, MEDLINE (OVID), CINAHL, and the Cochrane Library. The databases were searched for studies published between 2019 and April 2021. A combination of medical subject headings (MeSH) and free text terms were utilized to generate a systematic search strategy, which was developed and adjusted for use across databases (Kerr et al., 2020). Boolean connectors AND and OR were used to maximize saturation of terms searched (Cleary et al., 2018). The keywords used to search relevant literature included “nurs* undergraduate” OR “medical student*” AND “clinical clerkship” AND “SARS-CoV-2” and “digital*” OR “e-learn*”. An example of the search strategy for the PubMed database is shown in Supplementary File. Additionally, we adopted additional methods to improve the comprehensiveness of the retrieval, including checking the reference list of included studies and citation tracking (Wu et al., 2018).

2.3. Inclusion and exclusion criteria

We adopted specific screening criteria to ensure the relevance of the included articles. The inclusion criteria were as follows: (1) quantitative and mixed-method primary studies published from 2019 to April 2021; (2) the study samples were restricted to undergraduate nursing students or medical students participating in the clinical practicum; (3) studies describing digital education as a principal teaching alternative; (4) studies conducted during the COVID-19 pandemic; and (5) English language publications. The exclusion criteria were as follows: (1) the majority of the samples were not undergraduate nursing students or medical students; (2) papers in which the background was not relevant to the COVID-19 pandemic; and (3) qualitative articles, papers not written in English, and those in which only the abstract was available.

2.4. Critical appraisal

The Mixed Methods Quality Appraisal Tool (MMAT) revised by Pace et al. (2012) in 2011 was used to assess the risk of bias of studies with quantitative and mixed methods designs. MMAT is an effective and practical quality assessment tool developed for systematic mixed studies reviews and has been utilized in previous systematic reviews (Humphries et al., 2014; Ivanova et al., 2018; Pieper et al., 2013). This tool includes two screening questions for all research types and three to four questions for each type. The quality of the included studies was scored as “Yes”, “No”, or “Can't tell” according to whether the research met the criteria. The overall quality score is equal to the number of criteria met (Yes) divided by the total number of related questions and multiplied by 100 (Mącznik et al., 2015). Two researchers independently assessed the quality of the full texts, and any bifurcations were rigorously discussed until a consensus was reached. Given the limited number of studies, we decided to take an inclusive approach to include all eligible studies in this review.

2.5. Data extraction and synthesis

We extracted the quantitative data from the included studies into a self-developed data extraction form to gather details about the author, location, year of publication, study design, characteristics of the study population, specific teaching methods, assessment methods or instruments, main evaluation content and main findings. Meta-analysis was not appropriate given the heterogeneity of the digital teaching methods, study designs and outcome measurements identified. Therefore, systematic review findings are presented in narrative format.

3. Results

3.1. Included studies

A search of the selected databases yielded 4596 articles, with 16 remaining after the four-stage article screening according to the guidelines of Moher et al. (2009) (Fig. 1 ). Table 1 summarizes the main characteristics of the 16 included studies, which were conducted between 2020 and 2021. Studies were conducted in the United States of America (n = 7), China (n = 3), the UK (n = 1), Japan (n = 1), Korea (n = 1), Italy (n = 1), Arabia (n = 1), and Israel (n = 1). The sample sizes in the included papers ranged from 10 to 211 students. A total of 1174 participants were enrolled in this review, of whom 457 were undergraduate nursing students (3 studies) and 717 were medical students (13 studies). Only one of these studies contained a mixture of educational levels (He et al., 2021), such as undergraduate and PhD students in medicine. However, we only extracted data related to undergraduate students. The 16 papers used a variety of methodologies, including one cross-sectional study, two quantitative descriptive studies, two quasi-experimental studies, one prospective cohort study and the rest of the ten mixed-method studies. The publications were devoted to exploring the different modalities of digital education, including virtual reality-based simulation training (3 studies) (De Ponti et al., 2020; Kang et al., 2020; Weston and Zauche, 2021), teleconsultation and virtual rounds (3 studies) (Bala et al., 2021; Sukumar et al., 2021; Weber et al., 2021), web-based specialized skills learning (2 studies) (Alpert et al., 2021; Shahrjerdi et al., 2020), and multimodal online curriculums (8 studies) (Coffey et al., 2020; He et al., 2021; Kaliyadan et al., 2020; Kasai et al., 2021; Michener et al., 2020; Samueli et al., 2020; Williams et al., 2021; Zhou et al., 2020). Study outcomes were measured utilizing a series of homemade tools. For instance, standardized questionnaires, scales, anonymous surveys and Likert scales were used. In addition, academic performance was assessed in 3 studies in terms of examinations, and one study used specialized instruments with high reliability and validity to evaluate learning outcomes.

Fig. 1.

Fig. 1

Open in a new tab

Summary of the search results and selection process.

Table 1.

Basic characteristics and results of included studies.

Author, location (year) Study design Participants (undergraduate interns) Other participants Digital teaching methods Quantitative methods to assess teaching effect Evaluation contents Main findings
Weston and Zauche, US (2020) A cross-sectional study N = 186，traditional and second-degree nursing students (second semester) – virtual simulation (VS) ATI Nursing Care of Children examination The ATI score There were no significant differences in ATI scores between students who completed their pediatric clinical practicum in the clinical setting compared with virtually
Shahrjerdi et al., Hong Kong (China) (2020) A quantitative descriptive study N = 30, medical students, males: 14, famales:16 median age:23 (22–24) right-handed: 28 (93.4%) – A new web-based surgical skills learning (WSSL) A standardized questionnaire Ease of acquiring basic surgical skills; The degree of recommendation of the new web-based surgical teaching format 96.7% of the students believed that online demonstration was clear; Most students felt that WSSL was as difficult/easy as conventional face-to-face teaching; 90% of the students recommended the WSSL format
Samueli et al., Israel (2020) A mixed-method study N = 59, medical students (M3 and M4) – Whole slide images (WSI) An anonymous questionnaire Learning experience; Understanding of the content; Technical challenges The course was rated very favorably by the students; The online format seems to be preferable to students; The single worst disadvantage of the class were technical challenges in accessing the slides
Kaliyadan et al., Saudi Arabia (2020) A mixed-method study N = 45, medical students (M4 and M5) N = 4, faculty An online module (Powerpoint presentations, videos, quizzes, live interactive sessions in small groups) A structured questionnaire The general experience and satisfaction levels with the online teaching module Both students and faculty gave a lower score for practical skills training and assessment, and gave high scores in overall content coverage and technical aspects
He et al., China (2021) A quantitative descriptive study N = 55, medical students (clinical medicine, n = 47 and psychiatry, n = 8) N = 37, doctor of medicine Online neurology training course based on Small Private Online Course (SPOC) and blending learning mode An online questionnaire and the final score The interns' satisfaction; The recommendation of the incorporation of the online course into the future intern training mode after the crisis No significant difference was detected between the final scores of different groups of interns; All students were satisfied with the online course; The interns might have a better learning experience with a smaller learning group Size;
Roberto et al., Italy (2020) A mixed-method study N = 122, medical students (M6) – A virtual reality platform (21 patient-based clinical scenarios) A 12-item questionnaire The perceived quality of this training modality such as clarity, coherence, and relevance of items The perceived quality of this training modality was considered satisfactory; 85% of the students considered this training modality useful also in the absence of potential obstacles to traditional medical training
Zhou et al., China (2020) A quasi-experimental study N = 60, nursing students
IG: male: 1 females: 29 years: 22.06 ± 0.56
CG: male: 1 females: 28 years: 22.10 ± 0.63		 – IG: combined mode of massive open online course (MOOC) micro-video Examination of theoretical knowledge; Simulation examination of practice dialogue; An anonymous questionnaire The total score (theory and practice); Teaching satisfaction There was no significant difference between the two groups in terms of theory and practice; Compared with the CG, the interns' teaching satisfaction of IG was higher in multiple dimensions
Kang et al., Korea (2020) A quasi-experimental study N = 211, nursing students
Response rate: 91.0% (192)
males: 38 females: 154
years: 22.34 ± 1.88 (20−30)		 – Group 1 (G1): lecture and Vsim
Group 2 (G2): lecture and high fidelity
Group 3 (G3): lecture and Vism and high fidelity		 The knowledge scale of nursing care for children with asthma; Modified confidence tool for clinical performance; Modified clinical performance tool Knowledge level; Confidence in practice (CP); The ability to care for children with asthma Knowledge (*): G1 > G2, G3 > G2
CP (**): G1 > G2, G3 > G2
Total performance (**): G2 > G1, G3 > G1
Assessment (**) and education (**): G2 > G1, G3 > G1
Alpert et al., US (2020) A prospective cohort study N = 83, medical students
Response rate: 81.9% (68)
Cohort 1: n = 27 Cohort 2: n = 41		 – Cohort 1: Conventional in-person general diagnostic radiology course
Cohort 2: Remote course with Virtual Read-Out sessions (VRO)		 A five-point Likert scale of perceived frequency Students' sense of involvement in reviewing radiology exams; Technical limitations they encountered; Educational value of the learning experience Remote students reported a higher frequency of interaction than Conventional students; There were no technical limitations to remote learning; Conventional students reported a sense of boredom at a higher frequency than remote students;
Williams et al., US (2021) A mixed-method study N = 10, medical students N = 20, faculty A combination of asynchronous and synchronous courses A pre- and post-course questionnaire Assessing participants perceptions of urology By the end of the curriculum, median scores significantly improved in every topic area; At the completion of the course, 2 students reported a shift of specialty commitment to Urology
Weber et al., US (2021) A mixed-method study N = 64, medical students
Response rate: 45.3% (29)		 – Teleconsultation A questionnaire Experience gained in telehealth technologies; Trends in the number of patient encounters; The barriers in telehealth Students gained substantial experience with telehealth technologies; The percentage of patients successfully transitioned to a Doxy.me virtual encounter trending higher over time
Sukumar et al., US (2021) A mixed-method study N = 29, medical students
Response rate: 48% (14)		 N = 34, volunteer tele-instructors
Response rate:74% (25)		 Virtual Rounds A five-point Likert scale Student perception of changes to clinical abilities and confidence and evaluation of curricular elements; Tele-instructor perceptions of ability to teach, student learning and the personal impact The majority of students perceived improvement in their clinical abilities; Most students believed that virtual rounds curricular was useful; Over two-thirds of tele-instructors felt that their overall ability to teach on VR was better than during in-person rounds
Michener et al., US (2020) A mixed-method study N = 34, medical students
Response rate: 67.6% (23)		 – A multimodal virtual geriatrics elective An optional and anonymous survey The quality of virtual geriatrics elective; The influence of different course modalities on learning effects 95.6% of respondents agreed that the course was well organized and objectives were clear; The majority of respondents believed that the different course modalities could enhance learning
Kasai et al., Japan (2021) A mixed-method study N = 43, medical students (respiratory unit: n = 22; general medicine: n = 21)
Respond rate: 100%		 – Online education for clinical practice (online-sCP) (sECR, e-PBL, Online-VMIs) A questionnaire Clinical skills competence; Self-evaluation of participants' medical performance; The difference of self-study time in different stages; All students indicated improvement for all aspects of clinical performance; Students' satisfaction level with the online-sCP conducted by each department was acceptable; Self-study time was longer during online-sCP than during standard CCs or the period when CCs was postponed
Coffey et al., US (2020) A mixed-method study N = 132, medical students
Response rate: 73% (96)		 – Remote curriculum An anonymous, optional survey Student utilization of remote learning resources; Students' assessment of the quantity and structure of remote learning curriculum; Preparedness for subsequent stages of training;
Living Environments and Connectedness		 73% of the respondents felt that the number of remote learning resources was “about right.”
The majority of students felt that the quantity of required content and the degree of structure was appropriate; 27% of the students felt prepared for the shelf examination
Bala et al., UK (2021) A mixed-method study N = 11, medical students
Males: 2, females: 9		 N = 2, patients
Male:1 female: 1
N = 2, faculty		 Remote access teaching ward round An anonymous questionnaire The feasibility, acceptability and effectiveness of remote access teaching ward rounds All students agreed that MR technology enabled access to clinical teaching that was otherwise not feasible secondary to the COVID- 19 pandemic; The majority agreed that the quality of the holographic content was adequate; The majority of students agreed that they could interact with the clinician and have their questions answered
Open in a new tab

Note: ATI: Assessment Technologies Institute; M3: Students in their third year; M4: Students in their fourth year; M5: Students in their fifth year; M6: Students in their sixth year; IG: Intervention group; CG: Control group; Vsim: Virtual reality simulation; *: P < 0.05; **: P < 0.001; sECR: Simulated electronic health records; e-PBL: Electronic problem-based learning; Online-VMIs: Online virtual medical interviews; CCs: clinical clerkships; MR: Mixed reality.

3.2. Methodological quality of included studies

Methodological quality, as appraised by the MMAT exercise for the included 16 articles, ranged from 25% to 100%, with one study rated at 100% (Kang et al., 2020); two studies rated at 75% (Alpert et al., 2021; Weston and Zauche, 2021); five studied rated at 50% (Coffey et al., 2020; He et al., 2021; Kasai et al., 2021; Shahrjerdi et al., 2020; Zhou et al., 2020), and the remaining eight studies rated at 25% (Bala et al., 2021; De Ponti et al., 2020; Kaliyadan et al., 2020; Michener et al., 2020; Samueli et al., 2020; Sukumar et al., 2021; Weber et al., 2021; Williams et al., 2021) (Table 2 ).

Table 2.

Results of quality assessment based on the Mixed Methods Appraisal Tool (MMAT-2011) for qualitative, quantitative and mixed methods studies.

Author, year Study design S1 S2 1.1. 1.2. 1.3. 1.4. 3.1. 3.2. 3.3. 3.4. 4.1. 4.2. 4.3. 4.4. 5.1. 5.2. 5.3. Score
Weston et al., 2020 A cross-sectional study Y Y Y Y N Y 75%
Shahrjerdi et al., 2020 A quantitative descriptive study Y Y N N Y Y 50%
Samueli et al., 2020 A mixed-method study Y Y Y N N N N N Y N N N N 25%
Kaliyadan et al., 2020 A mixed-method study Y Y Y N N N N N Y UC N N N 25%
He et al., 2021 A quantitative descriptive study Y Y N N Y Y 50%
Roberto et al., 2020 A mixed-method study Y Y Y N N N N N Y Y N N N 25%
Zhou et al., 2020 A quasi-experimental study Y Y N Y N Y 50%
Kang et al., 2020 A quasi-experimental study Y Y Y Y Y Y 100%
Alpert et al., 2021 A prospective cohort study Y Y Y Y N Y 75%
Williams et al., 2021 A mixed-method study Y Y Y N N N N N Y Y N N N 25%
Weber et al., 2021 A mixed-method study Y Y Y N N N N N Y N N N N 25%
Sukumar et al., 2021 A mixed-method study Y Y Y Y N N N N Y Y N N N 25%

Michener et al., 2020 		A mixed-method study Y Y Y N N N N N Y Y N N N 25%
Kasai et al., 2021 A mixed-method study Y Y Y Y N N N N Y Y N Y N 50%
Coffey et al., 2020 A mixed-method study Y Y Y N N N N N Y Y N Y N 50%
Bala et al., 2021 A proof-of-concept study
A mixed-method research		 Y Y Y N N N N N Y Y N N N 25%
Open in a new tab

Note: Y: Yes; N: No; UC: Unclear; S1: Are there clear qualitative and quantitative research questions (or objectives*), or a clear mixed methods question (or objective*)? S2: Do the collected data allow address the research question (objective)? For score, the higher score indicates the better of the research quality. 1.1.: Are the sources of qualitative data (archives, documents, informants, observations) relevant to address the research question (objective)? 1.2.: Is the process for analyzing qualitative data relevant to address the research question (objective)? 1.3.: Is appropriate consideration given to how findings relate to the context, e.g., the setting, in which the data were collected? 1.4.: Is appropriate consideration given to how findings relate to researchers' influence, e.g., through their interactions with participants? 3.1.: Are participants (organizations) recruited in a way that minimizes selection bias? 3.2.: Are measurements appropriate (clear origin, or validity known, or standard instrument; and absence of contamination between groups when appropriate) regarding the exposure/intervention and outcomes? 3.3.: In the groups being compared (exposed vs. non-exposed; with intervention vs. without; cases vs. controls), are the participants comparable, or do researchers take into account (control for) the difference between these groups? 3.4.: Are there complete outcome data (80% or above), and, when applicable, an acceptable response rate (60% or above), or an acceptable follow-up rate for cohort studies (depending on the duration of follow-up)? 4.1.: Is the sampling strategy relevant to address the quantitative research question (quantitative aspect of the mixed methods question)? 4.2.: Is the sample representative of the population understudy? 4.3.: Are measurements appropriate (clear origin, or validity known, or standard instrument)? 4.4.: Is there an acceptable response rate (60% or above)? 5.1.: Is the mixed methods research design relevant to address the qualitative and quantitative research questions (or objectives), or the qualitative and quantitative aspects of the mixed methods question (or objective)? 5.2.: Is the integration of qualitative and quantitative data (or results*) relevant to address the research question (objective)? 5.3.: Is appropriate consideration given to the limitations associated with this integration, e.g., the divergence of qualitative and quantitative data (or results*) in a triangulation design?

3.3. The modality of digital education

3.3.1. Virtual reality-based simulation training

Three studies reported the use of virtual reality-based simulation training as an alternative to bedside clinical practice. Two of them focused on nursing students (Kang et al., 2020; Weston and Zauche, 2021), and one paid attention to medical students (De Ponti et al., 2020). Virtual simulation platforms such as i-Human® and Body Interact™ were sufficiently utilized for case-based learning to present students with an interactive medical patient experience. In addition, one quasi-experimental study included two additional intervention groups in which high-fidelity simulation (HFS) training and collaborative teaching combining HFS and virtual simulation training were conducted for nursing students (Kang et al., 2020). The three studies assessed the interns' Assessment Technologies Institute (ATI) scores, the perceived quality of this training modality, the level of knowledge, confidence in practice pre- and post-training and the aggregate performance. One study compared the students who were in clinical practice with those who participated in virtual simulation and reported that there were no significant differences in ATI scores (Weston and Zauche, 2021). The study focused on medical students indicated that more than half of them preferred simulation training with virtual platforms rather than online formal teaching, although technical troubles may inevitably emerge when accessing the online platform (De Ponti et al., 2020). The study included additional educational modalities, such as HFS and lectures, and showed that virtual reality-based simulation training could further enhance the knowledge level and confidence in clinical practice for the interns compared with high-fidelity simulation. However, the promoting effect of simulation training on overall clinical performance was slightly inferior (Kang et al., 2020).

3.3.2. Teleconsultation and virtual rounds

For the purpose of maintaining the continuity of the clinical practicum for medical students, three studies reconnected the interns with clinicians and patients through telemedicine, including teleconsultation and virtual rounds. The interns performed real-time interaction with patients scheduling a telemedicine appointment via Doxy.me video calls during the teleconsultation to recover the internship experience furthest (Weber et al., 2021). This study revealed that students obtained substantial experience with telemedicine technologies; for example, they could make a personal connection with patients and determine patients' perspective of their illness independently. The remaining two studies used Zoom video conference and Microsoft Dynamic 365 Remote Assist Application to interact with patients during virtual rounds. One study demonstrated that the clinical ability and professional confidence improved in the majority of the students and that the overall ability to teach tele-instructors in virtual rounds was better than that in in-person rounds (Sukumar et al., 2021). Beyond that, this study found that smaller groups were better for teaching and further affirmed the value of virtual rounds after the release of isolation. However, while another study using mixed reality (MR) technology suggested that virtual rounds were acceptable and effective, it denied its significance secondary to the COVID-19 pandemic (Bala et al., 2021).

3.3.3. Web-based specialized skills learning

Due to social distancing mandates, it is an urgent requirement for medical colleges to explore new approaches to learning clinical technical skills. Two studies were conducted in this general environment and mainly developed online web-based surgical skills learning (WSSL) and virtual read-out (VRO), which included learning groups composed of a faculty member and several students reviewing selective lists of imaging exams. One study used Zoom video conferencing for online presentation, focusing on surgical instruments and hand knots (Shahrjerdi et al., 2020). In this study, most of the students believed the new web-based surgical instruction was as difficult/easy as conventional face-to-face teaching, and they considered that this new teaching format was worthy of recommendation. Another study that performed remote radiology review by means of WebEx, a teleconferencing and screen sharing application, gauged the learning experience of two groups of interns (in-person group and VRO group) in reviewing radiology exams (Alpert et al., 2021). The results showed that remote training was more entertaining and that students tended to play an active role in reviewing radiology exams.

3.3.4. Multimodal online curriculums

A large number of medical colleges and their affiliated teaching hospitals developed multimodal online curricula to supersede face-to-face clinical internships during the novel coronavirus pandemic. In this review, 8 studies conducted online curriculums consisting of synchronous and asynchronous content and compared them with no controls. Asynchronous content included online videos, massive open online courses (MOOCs), discussion posts and so on. Synchronous content contained virtual conferences, impromptu role playing and lectures. Canvas, an online learning management system, was responsible for hosting asynchronous content. There were a variety of asynchronous platforms, such as Zoom, Tencent Class, and Blackboard®.

Three of the studies assessed the preference of the students for the different curriculum formats, and the results suggested that students had a greater interest in interactive discussion learning patterns and were less interested in passive teaching resources (Coffey et al., 2020; He et al., 2021; Michener et al., 2020). In addition, two of the three studies indicated that the majority of students were satisfied with multimodal online courses (He et al., 2021; Michener et al., 2020). One study found the same result as Sukumar et al. (2021), that a smaller learning group size might bring a better learning experience for interns (He et al., 2021). Additionally, this study also showed that an overwhelming majority of students recommended continuing the online curriculum after the pandemic. Another study found that interns gradually reconnected with school and their peers by taking online curricula (Coffey et al., 2020).

Two studies involved impromptu role-playing instruction, in which a faculty member played the role of a patient and a student acted as a doctor to simulate real clinical scenarios (Kaliyadan et al., 2020; Kasai et al., 2021). In addition, these two studies also separately included short presentations (PPT courseware), case-based teaching and other curriculum formats. One study indicated that both faculty and interns gave good feedback in terms of the overall content coverage and technical aspects of this online module, but they were less satisfied with practical skills training and assessment (Kaliyadan et al., 2020). Another study showed that interns had an acceptable satisfaction level with online simulated clinical practice (online-sCP); however, bedside practice was more useful for learning associated with physical examinations, medical interviews and humanistic qualities such as professionalism than online-sCP.

The remaining three studies varied in curriculum formats and assessment contents (Samueli et al., 2020; Williams et al., 2021; Zhou et al., 2020). One study training the diagnostic capability of medical students by browsing and reviewing slide sets of surgical biopsies found that the online format seemed to be preferable to students (Samueli et al., 2020). In addition, it is worth noting that the single worst disadvantage of the online curriculums was the technical challenges when accessing the slides. Another study with a massive open online course (MOOC) and microvideo as the major nursing teaching approach proved that there was no significant difference in the scores of the examination of theoretical knowledge and practice dialogue between interns who took face-to-face internships and those who participated in remote curriculums (Zhou et al., 2020). More interestingly, those who took the online curriculums reported higher levels of satisfaction with their internship. The last remaining study showed that interns' confidence in their urology practice and knowledge level increased significantly after the online curriculums (Williams et al., 2021).

4. Discussion

In this systematic review, we synthesized recent evidence on the effectiveness of thoughtful implementation of digital education for nursing and medical interns during the COVID-19 pandemic. Due to the limitations of the research design, only a few studies have compared digital education either with conventional face-to-face internships or with another form of educational technology. We found that for knowledge and practice, the standalone digital education modalities were as effective as conventional learning, which was consistent with previous research (Dunleavy et al., 2019). By comparing two different modalities of digital education, we found that different educational technologies have different effects on the knowledge and practice of interns. In addition, most studies examined in this review demonstrated the generally positive impact of digital education on the included interns in terms of skills, attitudes, satisfaction and other aspects. However, there was a high risk of bias in this part of the studies due to the deficiency of control groups.

There were some meaningful findings in terms of learning group size and teaching resources. First, a smaller learning group size might exert a better clinical teaching effect compared with a larger group size. A published study indicated that group size was a significant predictor of satisfaction with clinical experience for interns (Ofei-Dodoo et al., 2018), which was similar to our findings. Smaller groups help students dig deeper into what they are exploring and enable greater participation, collaboration and discussion (Bristol and Kyarsgaard, 2012; Burgess et al., 2017). Therefore, even though online teaching modalities allow a large number of users to attend a virtual space concurrently, the faculty should try to avoid centralized teaching where possible. Second, we found that interns had a greater interest in interactive training than passive access to resources. Similar findings have been found in previous reviews, indicating that interactive education was more effective than preaching education, which had little or no impact on professional practice (Bloom, 2005; Forsetlund et al., 2009). One of the possible reasons is that the motivation of learning can be heightened by an interactive style of teaching (Dolmans et al., 1998). In addition, the instantaneous feedback provided in interactive teaching helps students learn more effectively and internalize the material (Shapiro et al., 2014). Thus, it is a feasible and effective teaching reform measure to integrate interactive training into conventional didactic teaching to promote better learning after the COVID-19 epidemic is easing.

However, some deficiencies of digital education that played a significant role during the period of social isolation are worth noting. First, the technical challenges in distance access were the common shortcomings of most digital education modalities, which has also been confirmed in published studies (Hodgson and Hagan, 2020; Sleiwah et al., 2020). In this review, nearly half of the studies were conducted in developing countries, so costs such as computers and IT equipment might be one of the barriers to use. In addition, due to the outbreak of COVID-19, faculty members might have difficulty coping with the online delivery mode, so there was limited direction on how to implement such tools or programs (O'Doherty et al., 2018; Sahu, 2020). Therefore, a clear institutional strategy is recommended when implementing digital learning to reduce technical barriers to access (O'Doherty et al., 2018). Second, while most studies gave high praise for digital education and affirmed its value after the pandemic, they consistently believed that bedside clinical teaching was irreplaceable. Conventional clinical training and interaction with tutors was invaluable (De Ponti et al., 2020). Research has suggested that there might not be a competitive relationship between digital education and face-to-face teaching in some respects (Pei and Wu, 2019). Therefore, it may be worthwhile to combine the advantages of digital and face-to-face teaching methods to maximize the benefits (Pei and Wu, 2019).

The global scientific community has made great progress in combating COVID-19 (Chakravarty, 2021). With the advent of vaccines against COVID-19 and the implementation of social isolation measures worldwide, we have entered a period of relaxation of restrictions, during which the national economy and health education have been temporarily restored. In accordance with the teaching arrangements, we have resumed face-to-face teaching and ward rotation in a limited form (Khoo et al., 2021); additionally, digital education seems to continue to play a significant role in health professional education. However, there are still potential risks, such as gene mutation in SARS-CoV-2, which is one of the challenges that vaccine development has to face (Silveira et al., 2021). In addition, the popularization of vaccines also has insurmountable bottlenecks, including mass production and delivery of billions of doses to the worldwide population (Izda et al., 2021). As a result, the development of SARS-CoV-2 resistance to vaccines and unexpected outbreaks are highly possible in the near future (Dos Santos, 2021), which will lead to a new wave of digital learning in health profession education. Digital education is vital to remote training for interns both now and in the future. Nevertheless, it is worth noting that the ethical issues of patient privacy and data security are often ignored in the implementation of digital education, which may compromise the welfare of patients (Lall et al., 2019). To better apply it to clinical training, teaching hospitals should further standardize the specific implementation guidelines of digital education.

There are some limitations to the evidence evaluated in this review. First, there was a high risk of biases in many studies, such as attrition and selection bias. Second, the results of the included studies were too heterogeneous to be pooled. Third, non-English language articles were excluded due to lack of translation resources, which was likely to influence the findings of the results. Moreover, only four of the included studies (Alpert et al., 2021; Kang et al., 2020; Weston and Zauche, 2021; Zhou et al., 2020) contained controls, so the assessment of the effectiveness of digital education was relatively incomplete. Although randomized controlled trials that provide the highest level of evidence for the effectiveness of interventions are not appropriate for educational research (Tudor Car et al., 2019a), a quasi-experimental design can be used at least in future studies to explore the effectiveness of standalone digital education in health professions. Finally, almost all studies used homemade questionnaires to measure outcomes, which made it challenging to compare the use of digital education in different settings. Therefore, alternative methods such as the total time spent on digital learning or deeply qualitative analyses on the perceptions of digital education should be used to investigate interns' attitudes with digital education in future studies (Huang et al., 2019).

5. Conclusion

This is the first systematic review to synthesize the evidence relating to the application of digital education for nursing and medical interns during the COVID-19 pandemic. After comprehensive analysis, we found that digital education plays a significant role in distance training for nursing and medical interns both now and in the future. The overall risk of bias was high, and the quality of evidence was found to be variable. There is a need for further research adopting designs that provide a high level of evidence to assess the effectiveness of standalone digital education interventions for the remote training of nursing or medical interns. Additionally, it is significant for teaching hospitals to formulate systematic guidelines for the implementation of digital education to be fully prepared for emergencies.

Funding statements

This study was supported by the Higher Education Reform Research Project of Jilin Province (2019XZD068).

CRediT authorship contribution statement

Xiaonan Hao and Xin Peng: developing the review questions and drafting the protocol, developing a search strategy, conception and writing of the manuscript, updating the review.

Xinxin Ding and Yuan Qin: developing a search strategy, searching for articles, obtaining copies of articles, selecting which articles to include.

Miaohua Lv and Jing Li: extracting data from included articles, carrying out the analysis, assessing the risk of bias.

Kun Li: drafting the protocol, providing methodological expertise and advice, conception, writing and proof-reading of the manuscript, updating the review.

Xiaonan Hao and Xin Peng contributed equally to this work.

Declaration of competing interest

There is no competing financial interest in the research. The purpose of the research did not reflect the official policy or position of the organization and government.

Acknowledgements

None.

Footnotes

Appendix A

Supplementary data to this article can be found online at https://doi.org/10.1016/j.nedt.2021.105183.

Appendix A. Supplementary data

Search Strategies.

mmc1.docx (19.8KB, docx)

PRISMA 2020 Cheaklist.

mmc2.docx (36.6KB, docx)

References

  1. Alpert J.B., Young M.G., Lala S.V., McGuinness G. Medical student engagement and educational value of a remote clinical radiology learning environment: creation of virtual read-out sessions in response to the COVID-19 pandemic. Acad. Radiol. 2021;28:112–118. doi: 10.1016/j.acra.2020.09.011. [DOI] [PubMed] [Google Scholar]
  2. AlThiga H., Mohidin S., Park Y.S., Tekian A. Preparing for practice: nursing intern and faculty perceptions on clinical experiences. Med. Teach. 2017;39:S55–s62. doi: 10.1080/0142159X.2016.1254739. [DOI] [PubMed] [Google Scholar]
  3. Bala L., Kinross J., Martin G., Koizia L.J., Kooner A.S., Shimshon G.J., Hurkxkens T.J., Pratt P.J., Sam A.H. A remote access mixed reality teaching ward round. Clin. Teach. 2021;18:386–390. doi: 10.1111/tct.13338. [DOI] [PubMed] [Google Scholar]
  4. Bloom B.S. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int. J. Technol. Assess. Health Care. 2005;21:380–385. doi: 10.1017/s026646230505049x. [DOI] [PubMed] [Google Scholar]
  5. Bristol T.J., Kyarsgaard V. Asynchronous discussion: a comparison of larger and smaller discussion group size. Nurs. Educ. Perspect. 2012;33:386–390. doi: 10.5480/1536-5026-33.6.386. [DOI] [PubMed] [Google Scholar]
  6. Brusamento S., Kyaw B.M., Whiting P., Li L., Tudor Car L. Digital health professions education in the field of pediatrics: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/14231. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Burgess A., Bleasel J., Haq I., Roberts C., Garsia R., Robertson T., Mellis C. Team-based learning (TBL) in the medical curriculum: better than PBL? BMC Med. Educ. 2017;17:243. doi: 10.1186/s12909-017-1068-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  8. Car J., Carlstedt-Duke J., Tudor Car L., Posadzki P., Whiting P., Zary N., Atun R., Majeed A., Campbell J. Digital education in health professions: the need for overarching evidence synthesis. J. Med. Internet Res. 2019;21 doi: 10.2196/12913. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Chakravarty S. COVID-19: the effect of host genetic variations on host-virus interactions. J. Proteome Res. 2021;20:139–153. doi: 10.1021/acs.jproteome.0c00637. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cleary M., Kornhaber R., Thapa D.K., West S., Visentin D. The effectiveness of interventions to improve resilience among health professionals: a systematic review. Nurse Educ. Today. 2018;71:247–263. doi: 10.1016/j.nedt.2018.10.002. [DOI] [PubMed] [Google Scholar]
  11. Coffey C.S., MacDonald B.V., Shahrvini B., Baxter S.L., Lander L. Student perspectives on remote medical education in clinical core clerkships during the COVID-19 pandemic. Med. Sci. Educ. 2020:1–8. doi: 10.1007/s40670-020-01114-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  12. Cook D.A., Levinson A.J., Garside S., Dupras D.M., Erwin P.J., Montori V.M. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008;300:1181–1196. doi: 10.1001/jama.300.10.1181. [DOI] [PubMed] [Google Scholar]
  13. Curran V., Matthews L., Fleet L., Simmons K., Gustafson D.L., Wetsch L. A review of digital, social, and mobile technologies in health professional education. J. Contin. Educ. Heal. Prof. 2017;37:195–206. doi: 10.1097/CEH.0000000000000168. [DOI] [PubMed] [Google Scholar]
  14. Dao T.L., Hoang V.T., Gautret P. Recurrence of SARS-CoV-2 viral RNA in recovered COVID-19 patients: a narrative review. Eur. J. Clin. Microbiol. Infect. Dis. 2021;40:13–25. doi: 10.1007/s10096-020-04088-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. De Ponti R., Marazzato J., Maresca A.M., Rovera F., Carcano G., Ferrario M.M. Pre-graduation medical training including virtual reality during COVID-19 pandemic: a report on students' perception. BMC Med. Educ. 2020;20:332. doi: 10.1186/s12909-020-02245-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  16. Dedeilia A., Sotiropoulos M.G., Hanrahan J.G., Janga D., Dedeilias P., Sideris M. Medical and surgical education challenges and innovations in the COVID-19 era: a systematic review. In Vivo. 2020;34:1603–1611. doi: 10.21873/invivo.11950. [DOI] [PMC free article] [PubMed] [Google Scholar]
  17. Dolmans D.H., Wolfhagen I.H., van der Vleuten C.P. Motivational and cognitive processes influencing tutorial groups. Acad. Med. 1998;73:S22–S24. doi: 10.1097/00001888-199810000-00034. [DOI] [PubMed] [Google Scholar]
  18. Dos Santos W.G. Impact of virus genetic variability and host immunity for the success of COVID-19 vaccines. Biomed. Pharmacother. 2021;136 doi: 10.1016/j.biopha.2021.111272. [DOI] [PMC free article] [PubMed] [Google Scholar]
  19. Dost S., Hossain A., Shehab M., Abdelwahed A., Al-Nusair L. Perceptions of medical students towards online teaching during the COVID-19 pandemic: a national cross-sectional survey of 2721 UK medical students. BMJ Open. 2020;10 doi: 10.1136/bmjopen-2020-042378. [DOI] [PMC free article] [PubMed] [Google Scholar]
  20. Dunleavy G., Nikolaou C.K., Nifakos S., Atun R., Law G.C.Y., Tudor Car L. Mobile digital education for health professions: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/12937. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Forsetlund L., Bjørndal A., Rashidian A., Jamtvedt G., O'Brien M.A., Wolf F., Davis D., Odgaard-Jensen J., Oxman A.D. Continuing education meetings and workshops: effects on professional practice and health care outcomes. Cochrane Database Syst. Rev. 2009;2009 doi: 10.1002/14651858.CD003030.pub2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  22. George P.P., Papachristou N., Belisario J.M., Wang W., Wark P.A., Cotic Z., Rasmussen K., Sluiter R., Riboli-Sasco E., Tudor Car L., Musulanov E.M., Molina J.A., Heng B.H., Zhang Y., Wheeler E.L., Al Shorbaji N., Majeed A., Car J. Online eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J. Glob. Health. 2014;4 doi: 10.7189/jogh.04.010406. [DOI] [PMC free article] [PubMed] [Google Scholar]
  23. Harries A.J., Lee C., Jones L., Rodriguez R.M., Davis J.A., Boysen-Osborn M., Kashima K.J., Krane N.K., Rae G., Kman N., Langsfeld J.M., Juarez M. Effects of the COVID-19 pandemic on medical students: a multicenter quantitative study. BMC Med. Educ. 2021;21:14. doi: 10.1186/s12909-020-02462-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. He M., Tang X.Q., Zhang H.N., Luo Y.Y., Tang Z.C., Gao S.G. Remote clinical training practice in the neurology internship during the COVID-19 pandemic. Med. Educ. Online. 2021;26:1899642. doi: 10.1080/10872981.2021.1899642. [DOI] [PMC free article] [PubMed] [Google Scholar]
  25. Hodgson J.C., Hagan P. Medical education adaptations during a pandemic: transitioning to virtual student support. Med. Educ. 2020;54:662–663. doi: 10.1111/medu.14177. [DOI] [PubMed] [Google Scholar]
  26. Hodgson S.H., Mansatta K., Mallett G., Harris V., Emary K.R.W., Pollard A.J. What defines an efficacious COVID-19 vaccine? A review of the challenges assessing the clinical efficacy of vaccines against SARS-CoV-2. Lancet Infect. Dis. 2021;21:e26–e35. doi: 10.1016/S1473-3099(20)30773-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Huang Z., Semwal M., Lee S.Y., Tee M., Ong W., Tan W.S., Bajpai R., Tudor Car L. Digital health professions education on diabetes management: systematic review by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/12997. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Humphries S., Stafinski T., Mumtaz Z., Menon D. Barriers and facilitators to evidence-use in program management: a systematic review of the literature. BMC Health Serv. Res. 2014;14:171. doi: 10.1186/1472-6963-14-171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. Ivanova O., Rai M., Kemigisha E. A systematic review of sexual and reproductive health knowledge, experiences and access to services among refugee, migrant and displaced girls and young women in Africa. Int. J. Environ. Res. Public Health. 2018;15 doi: 10.3390/ijerph15081583. [DOI] [PMC free article] [PubMed] [Google Scholar]
  30. Izda V., Jeffries M.A., Sawalha A.H. COVID-19: a review of therapeutic strategies and vaccine candidates. Clin. Immunol. 2021;222 doi: 10.1016/j.clim.2020.108634. [DOI] [PMC free article] [PubMed] [Google Scholar]
  31. Kaliyadan F., ElZorkany K., Al Wadani F. An online dermatology teaching module for undergraduate medical students amidst the COVID-19 pandemic: an experience and suggestions for the future. Indian Dermatol. Online J. 2020;11:944–947. doi: 10.4103/idoj.IDOJ_654_20. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Kang K.A., Kim S.J., Lee M.N., Kim M., Kim S. Comparison of learning effects of virtual reality simulation on nursing students caring for children with asthma. Int. J. Environ. Res. Public Health. 2020;17:1–11. doi: 10.3390/ijerph17228417. [DOI] [PMC free article] [PubMed] [Google Scholar]
  33. Kasai H., Shikino K., Saito G., Tsukamoto T., Takahashi Y., Kuriyama A., Tanaka K., Onodera M., Yokoh H., Tatusmi K., Yoshino I., Ikusaka M., Sakao S., Ito S. Alternative approaches for clinical clerkship during the COVID-19 pandemic: online simulated clinical practice for inpatients and outpatients-a mixed method. BMC Med. Educ. 2021;21:149. doi: 10.1186/s12909-021-02586-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  34. Kerr D., Ostaszkiewicz J., Dunning T., Martin P. The effectiveness of training interventions on nurses' communication skills: a systematic review. Nurse Educ. Today. 2020;89 doi: 10.1016/j.nedt.2020.104405. [DOI] [PubMed] [Google Scholar]
  35. Khoo T., Warren N., Jenkins A., Turner J. Teaching medical students remotely during a pandemic - what can psychiatry offer? Australas. Psychiatry. 2021;29:361–364. doi: 10.1177/1039856220971931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  36. Kononowicz A.A., Woodham L.A., Edelbring S., Stathakarou N., Davies D., Saxena N., Tudor Car L., Carlstedt-Duke J., Car J., Zary N. Virtual patient simulations in health professions education: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/14676. [DOI] [PMC free article] [PubMed] [Google Scholar]
  37. Kyaw B.M., Posadzki P., Paddock S., Car J., Campbell J., Tudor Car L. Effectiveness of digital education on communication skills among medical students: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/12967. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. Lall P., Rees R., Law G.C.Y., Dunleavy G., Cotič Ž., Car J. Influences on the implementation of Mobile learning for medical and nursing education: qualitative systematic review by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/12895. [DOI] [PMC free article] [PubMed] [Google Scholar]
  39. Lee Y.M., Park K.D., Seo J.H. New paradigm of pediatric clinical clerkship during the epidemic of COVID-19. J. Korean Med. Sci. 2020;35 doi: 10.3346/jkms.2020.35.e344. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Leigh J., Bolton M., Cain K., Harrison N., Bolton N.Y., Ratcliffe S. Student experiences of nursing on the front line during the COVID-19 pandemic. Br. J. Nurs. 2020;29:788–789. doi: 10.12968/bjon.2020.29.13.788. [DOI] [PubMed] [Google Scholar]
  41. Mącznik A.K., Ribeiro D.C., Baxter G.D. Online technology use in physiotherapy teaching and learning: a systematic review of effectiveness and users' perceptions. BMC Med. Educ. 2015;15:160. doi: 10.1186/s12909-015-0429-8. [DOI] [PMC free article] [PubMed] [Google Scholar]
  42. Mandan J., Sidhu H.S., Mahmood A. Should a clinical rotation in hematology be mandatory for undergraduate medical students? Adv. Med. Educ. Pract. 2016;7:519–521. doi: 10.2147/AMEP.S112132. [DOI] [PMC free article] [PubMed] [Google Scholar]
  43. Michener A., Fessler E., Gonzalez M., Miller R.K. The 5 Mʼs and more: a new geriatric medical student virtual curriculum during the COVID-19 pandemic. J. Am. Geriatr. Soc. 2020;68:E61–E63. doi: 10.1111/jgs.16855. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Moher D., Liberati A., Tetzlaff J., Altman D.G. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339 [PMC free article] [PubMed] [Google Scholar]
  45. O'Doherty D., Dromey M., Lougheed J., Hannigan A., Last J., McGrath D. Barriers and solutions to online learning in medical education - an integrative review. BMC Med. Educ. 2018;18:130. doi: 10.1186/s12909-018-1240-0. [DOI] [PMC free article] [PubMed] [Google Scholar]
  46. Ofei-Dodoo S., Goerl K., Moser S. Exploring the impact of group size on medical Students' perception of learning and professional development during clinical rotations. Kansas J. Med. 2018;11:70–75. [PMC free article] [PubMed] [Google Scholar]
  47. Pace R., Pluye P., Bartlett G., Macaulay A.C., Salsberg J., Jagosh J., Seller R. Testing the reliability and efficiency of the pilot mixed methods appraisal tool (MMAT) for systematic mixed studies review. Int. J. Nurs. Stud. 2012;49:47–53. doi: 10.1016/j.ijnurstu.2011.07.002. [DOI] [PubMed] [Google Scholar]
  48. Park J., Park H., Lim J.E., Rhim H.C., Lee Y.M. Medical students' perspectives on recommencing clinical rotations during coronavirus disease 2019 at one institution in South Korea. Korean J. Med. Educ. 2020;32:223–229. doi: 10.3946/kjme.2020.170. [DOI] [PMC free article] [PubMed] [Google Scholar]
  49. Pei L., Wu H. Does online learning work better than offline learning in undergraduate medical education? A systematic review and meta-analysis. Med. Educ. Online. 2019;24:1666538. doi: 10.1080/10872981.2019.1666538. [DOI] [PMC free article] [PubMed] [Google Scholar]
  50. Pieper M.J., van Dalen-Kok A.H., Francke A.L., van der Steen J.T., Scherder E.J., Husebø B.S., Achterberg W.P. Interventions targeting pain or behaviour in dementia: a systematic review. Ageing Res. Rev. 2013;12:1042–1055. doi: 10.1016/j.arr.2013.05.002. [DOI] [PubMed] [Google Scholar]
  51. Sahu P. Closure of universities due to coronavirus disease 2019 (COVID-19): Impact on Education and mental health of students and academic staff. Cureus. 2020;12 doi: 10.7759/cureus.7541. [DOI] [PMC free article] [PubMed] [Google Scholar]
  52. Samueli B., Sror N., Jotkowitz A., Taragin B. Remote pathology education during the COVID-19 era: crisis converted to opportunity. Ann. Diagn. Pathol. 2020:49. doi: 10.1016/j.anndiagpath.2020.151612. [DOI] [PMC free article] [PubMed] [Google Scholar]
  53. Semwal M., Whiting P., Bajpai R., Bajpai S., Kyaw B.M., Tudor Car L. Digital education for health professions on smoking cessation management: systematic review by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/13000. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Seymour-Walsh A.E., Bell A., Weber A., Smith T. Adapting to a new reality: COVID-19 coronavirus and online education in the health professions. Rural Remote Health. 2020;20:6000. doi: 10.22605/RRH6000. [DOI] [PubMed] [Google Scholar]
  55. Seymour-Walsh A.E., Weber A., Bell A. Pedagogical foundations to online lectures in health professions education. Rural Remote Health. 2020;20:6038. doi: 10.22605/RRH6038. [DOI] [PubMed] [Google Scholar]
  56. Shahrjerdi P., Bhaiji S., Khawar S., Nwokedi E. Distant surgical teaching during COVID-19 - a pilot study on final year medical students. Surg. Pract. 2020;24:157–158. doi: 10.1111/1744-1633.12436. [DOI] [PMC free article] [PubMed] [Google Scholar]
  57. Shapiro M.C., Anderson O.R., Lal S. Assessment of a novel module for training dental students in child abuse recognition and reporting. J. Dent. Educ. 2014;78:1167–1175. [PubMed] [Google Scholar]
  58. Silveira M.M., Moreira G., Mendonça M. DNA vaccines against COVID-19: perspectives and challenges. Life Sci. 2021;267 doi: 10.1016/j.lfs.2020.118919. [DOI] [PMC free article] [PubMed] [Google Scholar]
  59. Sleiwah A., Mughal M., Hachach-Haram N., Roblin P. COVID-19 lockdown learning: the uprising of virtual teaching. J. Plast. Reconstr. Aesthet. Surg. 2020;73:1575–1592. doi: 10.1016/j.bjps.2020.05.032. [DOI] [PMC free article] [PubMed] [Google Scholar]
  60. Sukumar S., Zakaria A., Lai C.J., Sakumoto M., Khanna R., Choi N. Designing and implementing a novel virtual rounds curriculum for medical students' internal medicine clerkship during the COVID-19 pandemic. 2021;17:11106. doi: 10.15766/mep_2374-8265.11106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  61. Triola M.M., Huwendiek S., Levinson A.J., Cook D.A. New directions in e-learning research in health professions education: report of two symposia. Med. Teach. 2012;34:e15–e20. doi: 10.3109/0142159X.2012.638010. [DOI] [PubMed] [Google Scholar]
  62. Tudor Car L., Kyaw B.M., Dunleavy G., Smart N.A., Semwal M., Rotgans J.I., Low-Beer N., Campbell J. Digital problem-based learning in health professions: systematic review and meta-analysis by the digital health education collaboration. J. Med. Internet Res. 2019;21 doi: 10.2196/12945. [DOI] [PMC free article] [PubMed] [Google Scholar]
  63. Tudor Car L., Soong A., Kyaw B.M., Chua K.L., Low-Beer N., Majeed A. Health professions digital education on clinical practice guidelines: a systematic review by digital health education collaboration. BMC Med. 2019;17:139. doi: 10.1186/s12916-019-1370-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  64. Wang J.J., Deng A., Tsui B.C.H. COVID-19: novel pandemic, novel generation of medical students. Br. J. Anaesth. 2020;125:e328–e330. doi: 10.1016/j.bja.2020.05.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  65. Weber A.M., Dua A., Chang K., Jupalli H., Rizwan F., Chouthai A., Chen C. An outpatient telehealth elective for displaced clinical learners during the COVID-19 pandemic. BMC Med. Educ. 2021;21:174. doi: 10.1186/s12909-021-02604-z. [DOI] [PMC free article] [PubMed] [Google Scholar]
  66. Weston J., Zauche L.H. Comparison of virtual simulation to clinical practice for prelicensure nursing students in pediatrics. Nurse Educ. 2021;46:E95–E98. doi: 10.1097/NNE.0000000000000946. [DOI] [PMC free article] [PubMed] [Google Scholar]
  67. Wilcha R.J. Effectiveness of virtual medical teaching during the COVID-19 crisis: systematic review. JMIR Med. Educ. 2020;6 doi: 10.2196/20963. [DOI] [PMC free article] [PubMed] [Google Scholar]
  68. Williams C., Familusi O.O., Ziemba J., Lee D., Mittal S., Mucksavage P., Smith A., Kovell R.C. Adapting to the educational challenges of a pandemic: development of a novel virtual urology subinternship during the time of COVID-19. Urology. 2021;148:70–76. doi: 10.1016/j.urology.2020.08.071. [DOI] [PMC free article] [PubMed] [Google Scholar]
  69. Wu Y., Brettle A., Zhou C., Ou J., Wang Y., Wang S. Do educational interventions aimed at nurses to support the implementation of evidence-based practice improve patient outcomes? A systematic review. Nurse Educ. Today. 2018;70:109–114. doi: 10.1016/j.nedt.2018.08.026. [DOI] [PubMed] [Google Scholar]
  70. Zhou T., Huang S., Cheng J., Xiao Y. The distance teaching practice of combined mode of massive open online course micro-video for interns in emergency department during the COVID-19 epidemic period. Telemed. J. e-Health. 2020;26:584–588. doi: 10.1089/tmj.2020.0079. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Search Strategies.

mmc1.docx (19.8KB, docx)

PRISMA 2020 Cheaklist.

mmc2.docx (36.6KB, docx)

Articles from Nurse Education Today are provided here courtesy of Elsevier

RESOURCES