How Are Alternative Clinical Placements Performed Compared to Traditional Clinical Placements During the COVID-19 Pandemic? Sought Through a Systematic Review and Meta-Analysis

Abstract

Clinical placement is the essential method of learning in health professions education, but it has been the most disrupted by the COVID-19 pandemic. Institutions of higher education resorted to alternative learning such as telehealth, simulations, and blended-learning for clinical placement to ensure that educational activities continue without delay. However, this raises questions about student competency and necessitates making up for missed in-person hours. A thorough investigation of the effectiveness of alternative clinical placement learning is required. A systematic searching was conducted on ten electronic databases, and the quality of the included articles was assessed using the Medical Education Research Study Quality Instrument (MERSQI). A meta-analysis was conducted by pooling studies with examination mark outcomes. Twenty-four articles were included in the systematic review and nine were included in the meta-analysis. The average MERSQI score for included studies is 11.15. Outcomes on student performance favor alternative placement, whereas perceived-based outcomes have mixed results and are slightly prone to traditional clinical placement. Meta-analysis indicates that alternative learning is either more effective than traditional clinical placement or at least on par with it. There is a discrepancy between perceived outcome and performance assessment regarding the utility of alternative learning to conventional clinical placement. Nonetheless, objective measurement outcomes and the meta-analysis support alternative learning as a reliable learning strategy for clinical education. Alternative learning for clinical placement experience can be improved further by adding more synchronous sessions, and implementing various learning methods, learning activities based on strong instructional design, and at least a short real-setting attachment.

Introduction

COVID-19 has a wide-ranging impact on the field of health professions education. Immediate response to the pandemic entails shifting educational activities from physical to online and virtual, substituting clinical placement with alternative approaches, mitigating risk during patient contact, and providing faculty development to equip educators with immediate skills in new learning [1]. Student clinical learning and support were substantially disrupted, impacting personal and academic life and raising concerns for the mental health and well-being of students and staff [2–4]. In certain disciplines, such as the emergency department, the majority of the staff felt that students should not undergo clinical placement and were unwilling to teach students during the outbreak because they were overworked [5, 6]. Students have expressed negative perceptions regarding the altered educational situation, particularly concerning the shift in assessment methods from practical, such as the Objectively Structured Clinical Examination (OSCE), to non-practical, like written assignments. The students perceived that this change does not effectively gauge their competence, leading to a perceived decline in the quality of clinical learning. Moreover, they were also anxious about feeling less competent and are stressed about their future [7–10]. There is a suggestion that affected students supplement their clinical experience with additional placement [6], which could be cumbersome and result in an increased workload.

Although the COVID-19 pandemic impeded clinical placement and actual practice, it has also stimulated innovation in clinical and practical education. In health professions programs, clinical placement is regarded as crucial for learning opportunities. During the pandemic, numerous innovations in alternative clinical placement emerged by utilizing interactive and technological advancement and adapting pedagogical approaches, such as using virtual spaces, e-learning, telehealth, image repositories, live-streaming or pre-recorded videos, remote clinical visits, simulations, and restructuring clinical learning by integrating didactic lecture, blended-learning, student-centered learning, active learning activities, and games [11–14]. The use of modern technology in educational activities has the benefits of eliminating many barriers such as geographical and time constraints, making education more interactive and meaningful, and fostering independent learning. However, it also presents technical difficulties and a lack of human and real-world connection [10, 14, 15]. Recent research by Hao et al. [16] examined how the use of digital education during the COVID-19 pandemic can enhance students’ knowledge and practice. However, the evaluation only considered the population of nursing and medical students and did not investigate the effect of digital education on clinical placement and other alternative learning methods. A review by Jones et al. [17] found that the use of technology in clinical placement is plausible and advantageous. However, the review examined the use of technology as a supplementary learning material during clinical placement, and the included studies were dated and not intended for the COVID-19 period. A review by Jimenez et al. [18] suggested that simulation and other education interventions could replace a portion of clinical placement in medical radiation sciences education, but it is inconclusive, indicating that the optimal ratio of quality to quantity of learning activities warrants a more detailed investigation.

Developing an optimal clinical learning environment is a contentious issue in the field of health professions education. Clinical placement is regarded as the pinnacle of the learning opportunities offered by health professions programs. Clinical placement allows for the development of the learner’s personal characteristics and experiences, social relationships, organizational culture, and physical and virtual infrastructure [19]. Despite this, there is criticism that traditional clinical placement is unplanned and inefficient, and clinical staff such as practitioners, preceptors, and instructors are not well trained in education, preventing optimal learning [20–24]. Learners’ and educators’ readiness and acceptance for the use of technology as an alternative approach in teaching and learning practices are progressively increasing at present [15, 25, 26]. However, there is a perception that alternative clinical learning is insufficient and inferior to traditional clinical placement [11, 14, 27]. Consequently, traditional clinical placement is well-established and widely recognized as the “gold standard” in health professions education.

Though the replacement of a portion of clinical placement is supported by evidence, permitted, and recommended by several professional organizations, there is a widespread belief that alternative learning is inadequate and inferior to traditional clinical placement because it does not provide real-world experience. Therefore, the goal of our current review was to determine if the learners have acquired sufficient competence through alternative learning and if additional placement is required to supplement their education.

Methods

This systematic review followed the protocol for conducting a systematic review in medical education devised by Cook and West [28]. The protocol for this evaluation is registered with INPLASY (Registration Number: INPLASY202350110). Using the Population, Intervention, Comparison, and Outcome (PICO) method, the review question is formulated as follows: “How effective is alternative learning over traditional clinical placement for students in health professions education during the COVID-19?”.

Searching the Literature

Searches were conducted from May 3, 2023, to June 1, 2023, on Scopus database and Academic Search Complete, CINAHL, Cochrane Libraries, Health Business Elite, MEDLINE, Psychology and Behavioural Sciences Collection, and SPORTDiscus via EBSCOHost. The keywords were generated by searching for synonyms, referencing pertinent reviews and expert discussions. The search terms included healthcare (medicine, health sciences, nursing, dietetics, allied health, occupational therapy, physiotherapy, etc.), tertiary education students (e.g., university students, undergraduates, etc.), COVID-19 (novel coronavirus, SARS-CoV-2, etc.), and clinical learning (clinical placement, clinical fieldwork, clinical practicum, rotation, internship, etc.). Appropriate usage was made of Boolean operators, truncations, and parenthesis. Complete keywords and command operators used in this current review can be found in the registered protocol. The search was restricted to the years 2020 and later, as COVID-19 emerged in that timeframe. There was no manual search conducted.

Selection of Literature

The selection of articles was based on the following inclusion criteria: (i) the original research examined the efficacy of alternative clinical placement due to the COVID-19 outbreak, and (ii) the study population is composed of undergraduate or entry-level healthcare professions education academic programs. The article is excluded when (i) there is no full text available, (ii) the full text is not in English, (iii) it is gray literature (e.g., magazine, book, thesis, dissertation), (iv) it is not a research article (e.g., letter to the editor, brief communication), (v) it is not a primary study (e.g., review, audit study, protocol description), and (vi) there is no comparison aspect or control population/group with traditional clinical placement learning.

First, articles found through electronic searching were gathered and duplicates were removed using EndNote™ 20 software. Then, an Excel document was utilized to screen the articles. Initially, the first author (MHR) screened the titles and abstracts of the articles based on the criteria. Then, the eligible full texts were screened by the first author. Additionally, the full-text articles were randomly selected, divided equally, and independently screened by two other raters (SKA and SS). The ratings from the independent raters were compared with those of the first author, resulting in strong agreement (90% pre-consensus agreement). Any disputes regarding the status of an article were resolved through arbitrator (FWY) intervention to achieve a consensus.

Quality Appraisal

The Medical Education Research Study Quality Instrument (MERSQI) was used to evaluate the research quality of each included study [29]. MERSQI consists of ten items measuring six domains (i.e., study design, sampling, type of data, validity of the evaluation instrument, data analysis, and outcomes). The total score is the sum of the scores awarded to each item, which contribute to an aggregate score range of 5–18, with a higher score indicating better quality. There is consistency that a MERSQI score below 10 indicates lower quality [29, 30], and the average MERSQI score from multiple reviews pooled by Cook and Reed [31] was 11.3. In light of this, we chose a cutoff score of 11.5 for our review. The MERSQI is regarded as a feasible and credible instrument for evaluating health professions education literature due to its validity and reliability [29, 31, 32]. The first author was responsible for rating the MERSQI and the result was then discussed and validated by other authors.

Synthesis of Findings

The author, study objective, population studied, country of study, method of alternative clinical learning and traditional clinical placement, and selected findings related to the review’s objective were extracted from each included article and compiled in a matrix table. The narrative synthesis focuses on the alternative clinical learning procedure and summarizes the relevant findings. The narrative synthesis concerning the effectiveness of alternative learning outcomes is categorized into two sections: (i) objective outcomes, which evaluate students’ performance, including examination scores, and (ii) subjective outcomes, which are measured based on perceptions and questionnaire responses.

Meta-Analysis

The meta-analysis was performed using SPSS version 29.0. Only studies that employed objective measurement as the outcome were pooled since they are considered to be more credible and trustworthy [29, 31]. For any article with data that was absent, such as a standard deviation value and sample size, attempts were made to contact the author and request the missing information. The 50% threshold for I₂ analysis determines whether the data is homogeneous or heterogeneous, permitting the application of either a fixed-effect or random-effect model. Effect size is defined as small (d = 0.2), medium (d = 0.5), and large (d = 0.8) [33].

Result

The electronic search yielded a total of 1086 articles, but only 24 [34–57] were included in the final analysis for the systematic review (Fig. 1). The quality of the included studies, as measured by the MERSQI, has a mean score of 11.15 (SD 1.975) (Table 1), with only 11 studies deemed to be of good quality (score ≥11.5). The included studies consist of randomized controlled trial [53], quasi-experimental designs—such as non-randomized controlled trials [34, 38, 40, 47, 52], single-subject research [35, 41, 55], and one-group pre-post test [39], and observational designs—such as cross-sectional [37, 42, 43, 45], case–control [44, 46, 48–51, 54, 56, 57], and longitudinal studies [36, 54], respectively. The majority of the learner population are medical students (n = 10) [34, 35, 38, 40, 41, 46–48, 50, 57] and nursing students (n = 10) [36, 37, 39, 42, 43, 45, 52, 53, 55, 56] followed by students of other health sciences (e.g., biomedical science [49], medical radiation science [51], occupational therapy [54], and physiotherapy [44]) (n = 4). Table 2 provides information on each included study in detail.

Fig. 1.

Screening process. The screening process is illustrated in a flowchart where the initial searching found 1086 articles and end with 24 articles that were included in the systematic review synthesis, and 9 articles were included in the meta-analysis

Table 1.

MERSQI score evaluation on each included study

Author	Study design	Sampling		Type of data	Validity of instrument			Data analysis		Outcome	Total score
		Institutions	Response rate		Content	Internal structure	Relationship to variables	Appropriateness	Sophistication
Alpert et al. [34]	2	0.5	1.5	1	0	0	0	1	2	1	9
Arifin et al. [35]	2	0.5	1.5	1	0	0	0	1	2	1	9
Banjo-Ogunnowo et al. [36]	2	0.5	0.5	3	1	1	1	1	2	1.5	13.5
Brown et al. [37]	1	0.5	1.5	1	1	1	0	1	2	1	10
Daccache et al. [38]	2	1	1	1	1	0	0	1	2	1	10
Fung et al. [39]	1	0.5	1.5	1	1	1	0	1	2	1	10
Heitmann et al. [40]	2	0.5	1.5	1	1	0	0	1	2	2	11
Houghton et al. [41]	1.5	0.5	0.5	1	1	0	0	1	1	1	7.5
Kang and Hwang [42]	2	1.5	1.5	1	1	1	0	1	2	2	13
Kim and Roh [43]	1	1.5	1.5	1	1	1	0	1	2	1	11
Lawton et al. [44]	2	0.5	1.5	3	1	0	0	1	2	1	12
Leighton et al. [45]	1	1.5	1	1	1	1	0	1	2	1	10.5
Maar et al. [46]	2	0.5	1.5	1	0	0	0	1	2	1	9
McInerney et al. [47]	2	1	0.5	3	0	0	0	1	2	2	11.5
Nnamani Silva et al. [48]	2	0.5	1	3	1	0	1	1	2	1.5	12.5
Ooi et al. [49]	2	0.5	0.5	3	1	0	0	1	1	1.5	10.5
Redinger and Greene [50]	2	0.5	0.5	3	1	0	0	1	1	1.5	10.5
Sheilds et al. [51]	2	0.5	1.5	3	1	0	0	1	2	1.5	12.5
Son [52]	3	0.5	1.5	1	1	1	0	1	2	1	12
Su et al. [53]	3	0.5	1.5	3	1	1	1	1	2	2	16
Wan Yunus et al. [54]	2	0.5	1.5	3	1	0	0	1	2	2	13
Weston and Zauche [55]	3	0.5	1.5	3	1	0	0	1	2	1.5	13.5
Williams et al. [56]	2	1.5	0.5	1	0	0	0	1	1	1	8
Xiong et al. [57]	2	1.5	0.5	3	1	0	0	1	2	1	12

Table 2.

Characteristics of the included studies

Author	Study objective	Setting and participants	Procedure	Outcome measurement	Findings
Alpert et al. [34]	To determine the educational value of a new remote clinical radiology learning environment	Population: medical students Country: USA	Intervention: Virtual Read-Out (VRO) substituting exposure to clinical practice into reviewing curated imaging via teleconference, and classroom learning activities are transferred online Comparison: Conventional diagnostic radiology involves daily exposure to clinical radiology practice and classroom-based didactic, small-group, and interactive learning activities	Researcher-developed questionnaire on student’s perception on engagement and educational value with 8 items, 5-point Likert scale	Three of the eight items attained statistical significance, with two in favor of the remote group: “less boring” (p = 0.007) and “active role” (p = 0.01), and one in favor of the conventional group: “gaining confidence with PACS” (p < 0.001). Although not statistically significant, it is worth noting that “images are plainly seen” in one instance (p = 0.09), in favor of the conventional group. Other items are rated similarly and do not differ significantly (ps = 0.33–0.83)
Arifin et al. [35]	To evaluate usability of the remote contouring and virtual review (RECOVR) program	Population: 9 radiation oncology residents Country: Canada	Intervention: The RECOVR program contains organized pre-assigned cases and virtual online contour review conducted by the Microsoft Team Comparison: In-person contour review and no review	Perception of the value of education on a 5-point Likert scale	Increase the quantity of cases previously contoured by the RECOVR program. Overall educational value is perceived to be greater and substantially higher virtually than no review (p = 0.003), but the difference is not statistically significant than in-person (p = 0.993). Virtual feedback is perceived slightly higher but not significantly different from in-person feedback (p = 0.803), but substantially better than post hoc feedback (e.g., phone, email) (p < 0.001)
Banjo-Ogunnowo et al. [36]	To compare the HESI and NCLEX-RN outcome between traditional and virtual learning nursing students	Population: two cohorts of Associate Degree Nursing (I) students Country: USA	Group 1: Consists of 18 students in virtual setting (virtual classroom learning and i-Human virtual patient simulations) Group 2: Consists of 14 students in traditional setting (didactic classroom learning and in-hospital clinical)	Program examination by the Health Education System Inc. (HESI®) and the National Council Licensure Exam – Registered Nurse (NCLEX-RN®)	The HESI specialty exam reveals that the virtual group scored marginally higher than the traditional group, but the difference was not statistically significant (p = 0.69). The HESI exit exam was found to be marginally higher in the traditional group than in the virtual group, but this difference was not statistically significant (p = 0.430). No significant difference in NCLEX-RN® pass rates was observed between the two groups (p = 0.512)
Brown et al. [37]	To examine the efficacy of virtual simulation for educational purposes	Population: Clinical rotation nursing students (i.e., APRN/DNP and MSN) Country: USA	Three clinical learning simulation environments: traditional clinical experience (TCE), face-to-face virtual simulation (F2F), screen-based simulation by case study work (SBS) evaluated by 208 respondents	Clinical Learning Environment Comparison Survey (CLECS 2.0)	Finding on each CLECS domain indicated overall the TCE is statistically superior to F2F and SBS in terms of communication (p < 0.001), process (p < 0.001), holism (p < 0.001), and thinking (p = 0.002), but not self-efficacy (p = 0.486) and dyad (p = 0.988). Specifically, TCE is superior to SBS in each of the four domains, whereas F2F is superior to SBS in all domains except process
Daccache et al. [38]	To compare the effectiveness of a student-led social pediatrics clerkship to a conventional clerkship	Population: Medical students Country: Canada	Intervention (n = 7): Home visit with social medicine articles, virtual guest speakers, grand round presentations, and teleconference discussion for marginalized families Control (n = 12): previous cohort completed a 6-week core pediatric rotation	5-point Likert scale survey on 9 learning objectives as defined by Royal College of Physicians and Surgeons of Canada	The intervention group outperformed the control group on all nine learning objectives, but only six were statistically significant
Fung et al. [39]	To evaluate the effectiveness of virtual simulation combined with debriefing	Population: 188 nursing students Country: Hong Kong	Intervention: A combination of virtual simulation and debriefing sessions Comparison: Student’s previous experience on their traditional clinical practicum	Clinical Learning Environment Comparison Survey (CLECS 2.0)	On the communication, holism, and critical thinking CLECS domains, traditional clinical practicum is perceived to be superior to virtual simulation, whereas virtual simulation is perceived to be superior on the nursing process, self-efficacy, and dyads domains. However, only three domains were statistically significant: communication (p < 0.01), nursing process (p = 0.01), and critical thinking (p = 0.04). Holism (p = 0.56), self-efficacy (p = 0.72), and dyad (p = 0.23) were not
Heitmann et al. [40]	To compare the students’ competency between non-contact patient bedside teaching (non-bedside) with patient contact bedside teaching learning	Population: Medical students Country: Germany	Intervention group (n = 53): Pre-class preparation, small group in-person teaching, and peer practice Control group (n = 49): Pre-class preparation, small group in-person teaching, and attended bedside teaching	Perception on knowledge and skills on a 5-point scale	From the students’ perspective, there is no difference between the two groups in terms of knowledge and skills (ps > 0.05). The evaluation of lecturers, however, reveals a significant difference in knowledge (p = 0.011) and skills (p < 0.001) favoring non-bedside learning over bedside learning
Houghton et al. [41]	To compare student perceptions of digital versus face-to-face clinical placements	Population: First-time placement medical students Country: UK	After completing each placement, the students rated the digital and face-to-face perception evaluation tool. Only 29 (34.9%) responded to both placements	Researcher developed questionnaire based on the New World Kirkpatrick’s Model	Initially, 66% perceived digital placement prepared them for future placement, but this decreased to 48% after experienced face-to-face placement. Digital placement is more pleasurable than traditional (81%), but a 9-month follow-up reveals that face-to-face placement is perceived to be more enjoyable (68%) and challenging (74%) than digital placement. During the digital placement survey, 83% of respondents were in favor of incorporating digital placement into standard curriculum, and 76% were in favor of continuing digital curriculum alongside regular placement. However, in 9-month post survey, the number dropped to only 40%
Kang and Hwang [42]	To compare the statuses of newly graduated nursing students on their perception on nursing life	Participants: 178 newly graduated nursing students Country: South Korea	COVID-19 Cohort (n = 106): Graduates participated in substitutes clinical practicum through various online learning activities (e.g., skills laboratories, simulation laboratories, virtual simulation, or written assignments) during COVID-19 Non-COVID-19 Cohort (n = 72): Graduates participated in conventional clinical practicums prior to the COVID-19 pandemic	Questionnaire evaluating the perception of difficulties in nursing tasks, work readiness, reality shock, and organizational socialization	The clinical substitutes cohort perceived poorer than the conventional practicum cohort in all aspects, but only the difficulties in nursing tasks reached statistical significance (p = 0.020), whereas the others did not: work readiness (p = 0.769), reality shock (p = 0.707), and organizational socialization (p = 0.831)
Kim and Roh [43]	To compare the emotions of nursing students between alternative learning and clinical practicum	Population: 236 Nursing students from various institutions Country: South Korea	Intervention: Various alternative learning activities (e.g., e-Learning, simulations, skill laboratories) Comparison: Traditional clinical practicum of field practice in a clinical institution under the guidance of clinical instructors	Class-related Achievement Emotions Questionnaire	Positive achievement emotions of “Enjoyment” (p < 0.001), “Hope” (p < 0.001), and “Pride” (p = 0.001) were more prone to clinical practicum than alternative learning and reached a significant level. Negative achievement emotions are also prone to clinical practicum but only significant for “Anxiety” (p < 0.001) and “Shame” (p < 0.001) but not for “Anger” (p = 0.087) and “Hopelessness” (p = 0.056). However, “Boredom” is significantly associated with alternative learning (p = 0.023)
Lawton et al. [44]	To explore students’ perspective and performance achievement of the placement	Participants: Physiotherapy students Country: Australia	Intervention (n = 58): Online project-based placement in contemporary placement settings in corporate and sport facilities Comparison (n = 118): Traditional face-to-face of contemporary placement settings	• Assessment of Physiotherapy Practice (APP) in sport placements • Interprofessional Capability Assessment Tool (ICAT) in corporate placements	There is relatively no significant difference on the 20 outcomes in the APP except for two, on “Selects and measures relevant health indicators and outcomes” (p = 0.004), and “Conducts an appropriate client-centered interview” (p = 0.045), favoring the traditional placement. Only two of the three outcomes of the ICAT, the online group has fewer pupils in the advanced category, indicating a significant difference
Leighton et al. [45]	To compare students’ perception on clinical learning across three learning environments	Participants: 113 valid responses prelicensure nursing students Country: Canada, Japan, the USA	The analysis is grouped into three categories of traditional clinical experience (TCE), face-to-face simulation with facilitators (F2F), screen-based simulation using computer (SBS). No virtual environment involved	Clinical Learning Environment Comparison Survey (CLECS 2.0)	Overall, the TCE has the highest score while the SBS has the lowest and the difference is statistically significant for all items (ps < 0.05). The majority of significant differences were observed between TCE and SBS, while only a few were observed between F2F and SBS
Maar et al. [46]	To compare the perception of two students’ cohort between virtual and in-person simulated cultural communication scenarios (SCCS)	Population: pre-clerkship medical students Country: Canada	The simulation involved interaction with trained standardized patient from indigenous actor, and followed by a reflective discussion with a tutor to foster culturally safe interprofessional skills and understanding	Researcher developed outcome measure on five questions rated on a 5-point Likert scale	The virtual SCCS has lower learning experience than in-person SCCS on all questions with statistical significance (ps = 0.02–0.01)
McInerney et al. [47]	To investigate the effectiveness of simulation complementing the reduced clinical rotations experience	Population: Medical students Country: Ireland	Intervention (n = 35): Hybrid of simulation and clinical rotations Control (n = 31): Conventional clinical rotations	Competent assessment conducted via direct observation by non-academic clinicians of learner’s competence	Eight of the nine criteria observed indicate that the simulation group is better than the control; however, only three are statistically significant different which are on “history taking” (p = 0.004), appropriate laboratory (p < 0.001). and radiological investigation (p = 0.010)
Nnamani Silva et al. [48]	To determine the effectiveness of rapid adjustment on surgical clerkship	Population: Pre-clinical surgical students Country: USA	Traditional clerkship cohort: six 8-week surgical clinical block learning EMLR: Partial amount of traditional clerkship was substituted with a 3-week Extended Mastery Learning Rotation preparing the student to solidification of foundational and clinical knowledge consist of interactive and didactic learning Longitudinal clerkship: Partially substitution of intermittent surgical experience with telemedicine visit	Standard examination of either NBME surgery self-assessment examination or surgery shelf exam	The EMLR cohort’s NBME surgery self-assessment score is relatively similar to national average and not significantly different (p = 0.33). The longitudinal cohort shelf exam score is relatively higher than traditional clerkship but not statistically significant (p = 0.36), but statistically significantly lower than the previous national average shelf score (p = 0.001)
Ooi et al. [49]	To explore the implementation of a revolutionized e-practicum in biomedical science clinical learning	Population: Biomedical science students (number of samples not mentioned) Country: Malaysia	Intervention: Online teaching–learning and assessment activities including videos, live stream, reading materials, simulations, case-based studies, hands-on home laboratory, and virtual laboratory tool (e.g., Labster) Comparison: Previous cohort attended physical practicum	Final mark from course examinations and evaluations	There is no huge different on marks between online practicum learning cohort and physical practicum cohort whether on supervisor evaluation report and case study (20.9% vs. 23.8%), final internship report (33.3% vs. 34.3%), and logbook (21.1% vs. 25.9%), respectively
Redinger and Greene [50]	To explore the applicability and outcome of virtual emergency medicine clerkship during the COVID-19 pandemic	Population: Emergency medicine clerkship students Country: USA	Intervention: 56 students completed virtual rotation Comparison: 48 students previously completed traditional rotation	EM shelf exam composite score	There is no statistically significant difference between the exam scores of the two groups
Sheilds et al. [51]	To evaluate if the clinical alternative portfolio was sufficient and effective in increasing students’ knowledge and skills in nuclear medicine	Population: Medical Radiation Science students Country: Australia	Intervention: Limited physical skill attendance complemented with instructional videos, and completion of an alternative clinical portfolio Comparison: Physical laboratory skills attendance and 2 weeks’ clinical placement	Course’s examination mark	The clinical portfolio cohort has marginally higher radiopharmacy result (p = 0.11) and clinical placement result (p = 0.12) than the previous clinical placement cohort, but not statistically significant
Son [52]	To investigate the effect of S-PBL on nursing students’ attitudes, metacognition and critical thinking in comparison to the conventional clinical practicum	Population: Nursing students Country: South Korea	Experiment group (n = 47): Attending a 1-week Simulation Problem Based Learning (S-PBL) sessions using the case assignments and high-fidelity maternal birth simulator Control group (n = 31): Attended a 1-week maternity nursing clinical practicum	• Learning Attitude Measuring Scale (LAMS) • Modified Klein’s Metacognition Measure (m-KMM) • Yoon’s Critical Thinking Tool (YCTT)	The pre-test analysis for LAMS (p = 0.799), m-KMM (p = 0.901), and YCTT (p = 0.522) reveals no significant differences between the two groups. The within-group analysis revealed that the post-test performance of the experimental group improved significantly on LAMS (p < 0.001) and YCTT (p = 0.008), but not on m-KMM (0.063). On the other hand, neither LAMS (p = 0.319) nor m-KMM (p = 0.140) nor YCTT (p = 0.798) showed a significant improvement for the control group. No between or within-between analysis was reported for the post-test
Su et al. [53]	To investigate the effectiveness of structured learning compared to traditional learning in orthopedic nursing	Population: Undergraduate nursing students Country: China	Intervention group (n = 50): CDIO concept flipped-classroom, for theory, skills and clinical orthopedic department learning Control group (n = 50): Traditional teaching of didactic theory and skills teaching, and random clinical learning in the orthopedic department	• Theory examination • Mini-CEX • Self-directed learning ability evaluation scale • Critical Thinking Ability Assessment Scale • The teaching quality evaluation	Students in the experiment group perform significantly better than achievement those in control group in all parameters and all aspects (ps < 0.05). The teaching quality was also deemed to be favorable in the intervention group compared to the control group, reaching statistically significant (p < 0.001)
Wan Yunus et al. [54]	To explore the effectiveness of innovative clinical learning over conventional clinical practicum in psychiatric placement	Population: Undergraduate occupational therapy students Country: Malaysia	Innovation cohort (n = 17): Active and interactive online learning utilizing the SPICES model consist of case-based learning, team-based learning, and telehealth session with actual clients Conventional cohort (n = 19): Previous cohort attended traditional in-hospital clinical placement	Final marks based on examinations and other evaluations	The innovation cohort has better marks compared to conventional cohort and it achieved statistical significance (p = 0.006)
Weston and Zauche [55]	To compare the student examination outcome between two clinical practice methods of in-person and virtual among prelicensure nursing students	Population: Nursing students Country: USA	Intervention (n = 98): 5 weeks of virtual simulation learning using i-Human platform and online debriefing session Control (n = 88): 5 weeks of clinical practice with in-person simulation training and physical debriefing session	Assessment Technologies Institute (ATI) Nursing Care of Children examination	There is no significant different on the ATI exam score between the virtual and in-person clinical practice (p = 0.485)
Williams et al. [56]	To evaluate the students’ perception on placement experience	Population: nursing students Country: United Kingdom	Intervention: Simulation placement utilizing digital simulation and simulation laboratory delivered either flipped, in-person or remote Comparison: conventional clinical placement from previous cohorts	Institution’s survey on the feedback and perception on the placement	Positive responses were mixed across three disciplines (i.e., Adult, Child, Mental Health) for four cohorts (i.e., 2017, 2018, 2019, 2020), where the simulation cohort (i.e., 2020) is sometime better or poorer on some occasions
Xiong et al. [57]	To measure the success of flipped clinical rotations	Population: Medical students Country: USA	Intervention: Flipped clinical rotations where the students assigned to virtual knowledge-building activities followed by standardized exam prior to a briefer in-person patient care activities period Comparison: Traditional rotations where direct patient care activities fully integrated with teaching sessions and culminating in a standardized exam	Institution’s standardized exam on knowledge aspect	Students in the same cohort who attended flipped rotations performed better than those in traditional rotations in all clinical areas (e.g., Family Medicine, Internal Medicine, Neurology, Obstetrics and Gynecology, Pediatrics, Psychiatric, Surgery), but the differences were only statistically significant in three (Family Medicine, Internal Medicine, and Pediatrics). When compared to a previous cohort that participated in traditional rotations, the reversed rotations performed marginally better in all clinical areas with the exception of pediatrics and surgery, but none of the differences was statistically significant except for family medicine

Alternative Learning Activities for Clinical Placement

Several studies [36, 37, 39, 40, 45, 49, 52, 55] completely replace the clinical placement with non-real-world learning experiences, not even a virtual clinical visit. The majority of research on total replacement reveals unfavorable findings on alternative learning, while others [34, 35, 37, 38, 41–48, 50, 51, 53, 54, 56, 57] partially replace the clinical placement with alternative learning, and having a portion of actual clinical placement either physical or virtual. Research findings on partial clinical replacements are varied, with perceptions often fluctuating between alternative and actual placements.

Alternative learning found in the included studies involves the use of technology such as virtual simulation, online learning, and telehealth, as well as teaching methods employing non-real-world learning experiences through active learning activities including simulations, case-based learning, team-based learning, and didactic lectures.

The Effectiveness of Alternative Learning Activities for Clinical Placement Based on Objective Outcomes

Only the study by Su et al. [53] is regarded as the strongest evidence as it employed a randomized controlled trial design, and having the highest MERSQI score. In terms of both written examination and clinical placement scores, the study indicates that the alternative clinical learning method is preferable to traditional clinical placement. Other studies that use examination scores as an outcome majority indicate the alternative learning is superior to traditional clinical placement; however, the significance of the efficacy is varied, where only one study achieved statistical significance [54] and one with mixed results [57], while others did not [36, 48, 50, 51]. Several studies have concluded that alternative learning is inferior to conventional clinical placement; however, none of these studies is statistically significant [36, 55], and one study [49] did not conduct an inferential test.

The Effectiveness of Alternative Learning Activities for Clinical Placement Based on Subjective Outcomes

The included studies also explored the perceptions regarding alternative learning and traditional placement. However, only eight [37, 39, 42–45, 52, 53] studies utilized standardized measurement tools.

Among those studies that utilized standardized questionnaires, several indicate a stronger preference for traditional clinical placements. Three [37, 39, 45] studies that used the Clinical Learning Environment Comparison Survey (CLECS 2.0) as an outcome measure discovered that learners perceived traditional placement to be superior to any alternative learning, and the majority of the domains reached statistical significance. Another study [42] also discovered that alternative learning is inferior to traditional placement, but only in one of the four domains examined is statistically significant. A study [44] of physiotherapy students’ perceptions of their own competence found that they favored the traditional placement. One study [43] indicates that traditional clinical practicum is associated with greater emotional intensity, whether positive or negative, than alternative learning. Nonetheless, three [47, 52, 53] investigations found the opposite, favoring alternative learning. Son [52] found that the perception of learners’ learning attitude and critical thinking is marginally better, whereas metacognition is comparable to the control group but none of these differences is statistically significant. A hybrid of simulation and clinical rotations was found to be superior to traditional rotations in almost every competency but only three of the eight components are statistically significant [47]. Su et al. [53] discovered that participants in the intervention group perceived their self-learning ability and critical thinking more positively than those in the control group and attained statistical significance. The study [53] also found that the learners perceived the teaching quality of educators in the intervention group to be significantly superior to that of educators in the control.

Other remaining studies used self-developed outcome measures [34, 35, 38, 40, 41, 46, 56], for which the validity and reliability of the instrument are queried. Among the perception studies that utilized non-standardized outcome measures, one study [38] found a positive outcome where alternative learning is better than traditional placement in achieving the learning objectives; however, only six out of the nine learning objectives were statistically significant. According to another study [34], students perceive alternative learning to be interactive, but traditional practicum gives them greater confidence in their practice. Contradictory perceptions exist between learners and educators, where learners believe there is no difference in their knowledge and skills, whereas educators believe there is a significant difference, supporting the advantages of alternative learning over traditional bedside instruction [40]. Three studies utilizing basic measurement found that alternative learning is not as meaningful or beneficial as traditional placement; however, only one study [46] found significant value, another [35] did not, and one [41] had only descriptive data. While one study [56] reported mixed results regarding the perception of clinical placement between simulation and conventional methods, it ultimately found that the overall positive perception of clinical placement was significantly higher for simulation-based placements.

Meta-Analysis

There are nine [36, 48–51, 53–55, 57] studies that present the examination scores of students as the outcome considered for the meta-analysis. However, one of the two outcomes in Nnamani Silva et al. [48] is omitted due to a large sample size disparity between the institution sample and the national population, which may contribute to bias. The theoretical examination results from Sheild et al. [51] and Su et al. [53] were omitted from specific research investigations. HESI examination results specific to the Maternal Specialty in Banjo-Ogunnowo and Chisholm [36] were retained, but the HESI Exit examination score was omitted due to the implementation of alternative placements during that specialization period. Xiong et al. [57] present two sets of data comparing alternative learning to traditional clerkship from the same cohort or the previous cohort. It was decided to include only the preceding cohort as the placement is at its optimal level. In addition, Xiong et al. [57] data were reported separately according to area of practice and were therefore analyzed separately.

Other studies could not be pooled because their results were not comparable and did not utilize an objective outcome. Although it is possible to combine studies utilizing CLECS, the measurement instrument is self-administered by participants and is therefore subjective.

As the I₂ value is 82%, a model with random effects was implemented. The meta-analysis (Fig. 2) indicates that alternative clinical placement is more effective than traditional placement, with a medium effect size (d = 0.379) and statistical significance (p = 0.01). The funnel plot (Fig. 3) depicts a balanced distribution that falls largely within the confidence interval, indicating an acceptable publication bias.

Fig. 2.

Meta-analysis with forest plot pooling studies with examination mark as the outcome. Fifteen data from nine studies in a forest plot majority skewed to the right side of the graph, favoring alternative learning over traditional clinical placement, while the overall value and its confidence interval line (i.e., diamond shape with its whisker) placed on the right side without crossing the no-effect line (i.e., vertical dotted line) indicate significant effectiveness of alternative learning over traditional clinical placement

Fig. 3.

Funnel plot analysis for publication bias. The preponderance of publications falls within the triangle-shaped graph, indicating that publication bias is minimal and acceptable

Discussion

The technologies and approaches found in this review are comparable to those described in previous reviews [16–18], where technology use, particularly in relation to online and virtual learning, dominated alternative learning. According to the findings of the meta-analysis, alternative clinical placement is superior to or at least equivalent to traditional clinical placement, which is an objectively valuable conclusion. Despite variations in findings regarding subjective outcomes, substantial evidence supports alternative learning. Previous reviews show that alternative learning methods in health professions education, such as virtual reality [58], e-learning [59], adaptive e-learning environments [60], and technology-based learning in general education [61] enhance student outcomes in comparison to traditional learning. There is not much difference on outcomes between high-fidelity and low-fidelity alternative learning, but students have more positive perceptions on the high-fidelity one. This is similar to a previous study where students preferred more high-fidelity simulation and assumed to be more competent when learning through high-fidelity simulation [62]. High fidelity involves intricate and immersive learning opportunities that closely resemble real-world scenarios, where learners can receive realistic feedback and rich learning experiences. In contrast, low fidelity involves simpler, less complex methods of learning that cater to partial tasks with limited functionality to meet specific skill practice requirements; it lacks the realism of true-to-life experiences. It was noted in some cases that alternative learning is superior to traditional clinical placement as it can provide structured learning opportunities and accommodate uncommon situations encountered in practice [59, 63]. This review found that although clinical experience is reduced, learning outcomes are better. One reason they are better is the substitution of alternative clinical learning based on structured teaching and learning approaches, which makes the learning activity more targeted and meaningful. Appropriate instructional design is the key to enhancing student satisfaction [15, 64]. Without a theoretical base, knowledge cannot be effectively applied, and context and structure are lost [65]. The enhancement of the learning experience for the students facilitates the achievement of the learning objectives.

This analysis supports the recommendation that a portion of clinical placement can be substituted for alternative learning without compromising educational quality. Several organizations have suggested that 10 to 50% of clinical learning, including traditional clinical placement, can be safely replaced with alternative methods [66–69]. According to a Delphi study involving panels of nurses and allied health professionals, 11 to 30% of clinical placement could be replaced by alternative methods [70]. According to Larue et al. [71], alternative approaches ought not to supplant more than 25% of clinical placement hours. Nevertheless, actual clinical placement is unable to be denied as essential to provide genuine learning experience that cannot be replicated in alternative learning.

Real-world practice is crucial for workplace-based learning and doing clinical placement during a pandemic situation is feasible. According to reports, the majority of practitioners are prepared to continue supervising students physically or digitally during or after the pandemic [6, 72]. Stakeholders are crucial in ensuring that students successfully complete clinical placement in circumstances involving university preparation and communication, placement site and supervisor support, and students’ resilience and capacity to adapt to a changed work environment [9, 73]. The educational landscape should also be revolutionized by making learning more liberal, reforming education from conservative practice to contemporary practice with supported evidence, focusing on multiangle outcomes, expanding the assessment toolbox, and assisting the transition in career process [19, 74, 75]. These prepare future health professions education to embrace alternative clinical placement methods as the early preparations make students more equipped and ready for actual placement training.

This review indicates that perception and performance are at dissonance. Perception and performance do not always correspond, as the majority of students in this review perceive alternative learning to be inferior to traditional placement, despite performing better in alternative learning than in traditional placement. The low satisfaction with online learning during the COVID-19 pandemic is prevalent, particularly among medical students in comparison to non-medical students; however, the receptivity is improving over time [10, 76]. Despite the argument that perceptions such as emotion and satisfaction are associated with achievement, there are multiple factors that influence achievement as students become resilient, and manage and solve the problem [8, 10]. The disalignment is predicted as a psychological by-product, where humans are more comfortable with something they are familiar with, established, and widely practiced by the norms; thus, the learning during COVID-19 put learners as well as educators outside their comfort zone of common traditional clinical placement and increased anxiety and stress due to uncertainty of the situation as well as adapting to the new norms [77, 78]. This pledges for a substantial amount of alternative clinical learning to be incorporated into the curriculum of health professions education as a standard practice to make students more accustomed and accepting.

Limitation of the Current Review

A limitation of this review is that many other technologies and methods were not included because the studies did not meet the inclusion criteria. This review nevertheless provides evidence that alternative learning is a plausible method for substituting a portion of clinical placement in general. While this review enhances quality by selecting better-designed studies, it is worth noting that only a limited number of studies meet high-quality criteria. This is reflected in the fact that a significant amount of research in medical education is not rigorously designed and robustly reported [79]. Although it has been argued previously that design is not the most important aspect of medical education research [79], additional high-quality studies are required. In addition, the meta-analysis should be cautiously accepted as the I₂ value indicates high heterogeneity exists. However, it is important to note that this heterogeneity is inherent to the nature of research in the health professions education discipline [28, 80]. We became aware of the Modified MERSQI (MMERSQI) [81] after completing this review. While the MMERSQI represents an improvement over the original MERSQI, its psychometric evidence is limited, and its scoring interpretation remains ambiguous [82]. Nonetheless, we decided to remain with the original MERSQI due to our familiarity with it, as well as its widespread recognition and acceptance in mainstream academic journals.

Recommendation for Practice

While alternative learning offers benefits for clinical learning and placement, it is important to emphasize that rich and meaningful learning experiences can still be achieved. First, it is recommended that a substantial quantity of learning be conducted synchronously. Real-time interaction, whether with instructors, simulated patients, or actual clients, is essential to enhancing the learning experience [15, 17]. This will ensure instantaneous feedback and quick reasoning, optimizing the learning process. Second, it is essential to construct a realism in fidelity of simulation learning [83], which can be accomplished in a cost-effective manner. Technology development is constantly improving making it more affordable and having higher fidelity such as telehealth [11] and haptic technology to provide meaningful sensory feedback [84]. Third, learning activities should be varied to attract interest, and theoretical learning should be developed appropriately, supported by evidence-based instructional designs and learning techniques, such as case-based learning and team-based learning, and balanced between didactic and active learning [15, 17, 64]. Fourth, alternative learning is preparatory and should not stand alone; it should be followed by workplace-based learning—even briefly—to consolidate previously acquired knowledge and skills [85]. Students are better equipped and require less time to execute skills in practice due to the knowledge and abilities they have gained through alternative learning.

The application of technology in medical education encompasses every stage of the education process. The current situation necessitates the use of technology not only for learning, but also for pre-recruitment, assessment, and continuous learning. For instance, the use of virtual assessment such as electronic objectively structured practical examination (e-OSPE) is just as reliable as traditional OSPE and reveals that the virtual OSCE is beginning to be adopted in health professions education practice [86, 87]. It has been determined that the use of technology-enhanced simulations is advantageous for the professional development of healthcare practitioners [63]. Daniel et al. [88] also report an increase in the use of technology in the selection of candidates for educational programs in recent years. To make students more accustomed and receptive to technology use, it is essential that technology be incorporated into every stage of learning and assessment.

The use of technology in health profession education will become mainstream and standard in the future. Consequently, it is essential to prepare the next generation for the event of technology learning. Preceptors need training to be competent and prepared for future contemporary placements using virtual and remote methods [89, 90]. It is necessary for developing and low-income countries to prepare basic necessities and facilities, such as ICT equipment including computers, video cameras, and the internet, as they are feasible and the cost benefits are comparable to traditional placement [87, 91–94]. A wide range of technological strategies should be employed in clinical placement as they are beneficial to support better learning experience [17]. This review is relevant not only for addressing the challenges posed by COVID-19 but also for shaping the future educational landscape.

Conclusion

Educational institutions resorted to alternative learning to supplement the disrupted clinical placement during the critical period of the COVID-19 pandemic. Alternative learning does not compromise either the student’s knowledge or clinical abilities. The inferiority of alternate placement is due to perceptions rather than performance evaluation. Thus, the competence of students participating in alternative placement during the COVID-19 period is assured. Exploring the broader implementation of alternative learning in clinical placements beyond the pandemic is essential for ensuring the educational institution’s resilience and adaptability to future challenges.

Author Contribution

MHR’s major role was initiating the idea, involving in every aspect of searching, screening, disseminating, quality evaluation, extracting, and synthesizing the data, and writing the manuscript. FWY’s role was in screening and decision of included studies. SK’s and SS’s roles were in evaluating the quality and extracting the data. All authors involved in providing critical and significant intellectual feedback in writing the manuscript. All authors approved the final version of the article for publication.

Availability of Data and Materials

All of the data come from published articles with appropriate citations. The authors contacted the original author for data that was not included in the published article, and have received their response and permission to use the data in our meta-analysis.

Declarations

Ethical Approval

Not applicable. No human subjects were involved this study.

Informed Consent

NA.

Conflict of Interest

The authors declare no competing interests.