Review article: E‐learning in emergency medicine: A systematic review

Abstract

E‐learning (EL) has been developing as a medical education resource since the arrival of the internet. The COVID‐19 pandemic has minimised clinical exposure for medical trainees and forced educators to use EL to replace traditional learning (TL) resources. The aim of this review was to determine the impact of EL versus TL on emergency medicine (EM) learning outcomes of medical trainees. A systematic review was conducted according to the Preferred Reporting Items for Systematic Review and Meta‐analysis statement using articles sourced from CINAHL, Embase, OVID Medline and PubMed. Articles were independently reviewed by two reviewers following strict inclusion and exclusion criteria. Bias was assessed using the Cochrane Risk of Bias tool. The search yielded a total of 1586 non‐duplicate studies. A total of 19 studies were included for data extraction. Fifteen of the included studies assessed knowledge gain of participants using multiple‐choice questions as an outcome measure. Eleven of the 15 demonstrated no statistically significant difference while two studies favoured EL with statistical significance and two favoured TL with statistical significance. Six of the included studies assessed practical skill gain of participants. Five of the six demonstrated no statistical significance while one study favoured EL with statistical significance. This systematic review suggests that EL may be comparable to TL for the teaching of EM. The authors encourage the integration of EL as an adjunct to face‐to‐face teaching where possible in EM curricula; however, the overall low quality of evidence precludes definitive conclusions from being drawn.

E‐learning (EL) has been developing as a medical education resource since the arrival of the internet. The COVID‐19 pandemic has minimised clinical exposure for medical trainees and forced educators to use EL to replace traditional learning resources. The aim of this review was to determine the impact of EL versus traditional learning on emergency medicine learning outcomes of medical trainees.

Key findings.

• EL may be comparable to TL for learning outcomes of medical students and junior doctors in EM.

• The overall quality of the literature comparing EL versus TL is low which precludes definitive conclusions from being drawn.

• Investigation interpretation and life support skills represent topics in which multicentre randomised controlled trials that use a homogenous set of guidelines on how information is delivered and tested could allow for conclusive comparison between EL and TL.

Introduction

The COVID‐19 pandemic has caused significant changes to how medicine is both taught and learnt. Face‐to‐face teaching has been decreased or entirely disrupted for many medical students and junior doctors. ¹ To promote social distancing, measures to minimise teaching in clinical environments have been employed. Students have been largely removed from clinical placement; physical participation in workshops has ceased or been reduced and healthcare professionals have worked from home where students are unable to engage in their clinical encounters with patients. ² , ³ These changes to the education of medical trainees have forced educators to urgently adapt, shifting towards forums and modules, which can be accessed through e‐learning (EL). ¹

Although EL has been developing as an educational source since the arrival of the internet, ⁴ it has been shifted by the pandemic from a novel adjunct of traditional learning (TL) and face‐to‐face teaching to the primary source of education for many medical trainees. ¹ EL is defined as the gain of knowledge, skills or other competencies via electronic means (primarily referring to the internet). ⁵ It can delivered as either a purely online course, or as a part of a ‘blended learning’ approach to augment traditional learning methods. EL typically features at least one of three characteristics, an emphasis on communication and collaboration either between the learner and staff or between different learners, modalities that encourage direct interaction with the content or asynchronicity and flexibility allowing the learner to engage with the content in their own time and at their own pace. ⁶ EL is used synonymously with online learning, computer‐based instruction and internet learning.

EL offers unique advantages that TL cannot. Primarily, trainees can access EL at any time and at any location. ⁴ , ⁷ EL allows trainees to take control of their own learning by increasing the flexibility and ease with which they can access material. This lessens the impact of COVID‐19 on trainee learning in addition to reducing non‐COVID‐related problems such as travel time and other commitments of students (such as jobs, volunteering and recreation). Furthermore, EL can include platforms, which are more interactive to participants, transferring away from didactic models used in traditional, teacher‐centred resources. ⁸ , ⁹ Similarly, students can often access material at their own speed, adding to the participant‐centred style of teaching.

Conversely, EL faces some of its own distinctive problems. Practical skills may be difficult to develop over the internet and technical difficulties in EL resources may prohibit knowledge gain. ⁷ , ¹⁰ Moreover, while EL is often seen as an economic option, time and effort required by educators to create modules can be costly. ⁷

Emergency medicine (EM) presents challenges to doctors given the vast knowledge base required to stabilise, investigate, diagnose and manage a variety of patients. However, discrepancies across hospitals in terms of the scope of practise and educational resourcing of different EDs ensure that not all trainees are exposed to the same clinical teaching. ¹¹ Given the sometimes uniquely unpredictable nature of an ED, having a formal and structured EL platform which trainees can utilise at their own convenience may significantly improve access to information and allow precious face‐to‐face teaching to focus on other competencies such as interpersonal interaction or the application of practical skills. ⁷ Furthermore, an understanding of the efficacy of EL in EM for medical students is paramount given EDs are often among the first departments within hospitals to restrict student access during outbreaks of COVID‐19.

It is unlikely EL will ever be able to completely replace TL. However, there may be a place to supplement lectures and textbook learning with interactive EL modules in some areas. The aim of this review is to determine the impact of EL versus TL on the EM learning outcomes of medical students and junior doctors.

Methods

A systematic review of the literature was conducted assessing EL in the context of EM. This review followed the completion of a research protocol according to the Preferred Reporting Items for Systematic Review and Meta‐analysis (PRISMA) statement (Fig. 1). ¹²

Figure 1.

PRISMA flow chart.

Studies were included if they: applied EL as an intervention compared to a TL technique, assessed knowledge gain or practical skill gain as an outcome, were specific to medical students or junior doctors on the topic of EM and were quantitative studies with post‐intervention measures from all years. Studies assessing both medical students and junior doctors were included in this review because of the paucity of the literature, and the similar disruption in the clinical teaching in EM because of the pandemic. No restriction was applied on the length of the programme or how participants accessed the online content, whether it be via computer, mobile phone or tablet. For this review, EL was any intervention providing content, information or skill development over the internet. This included recorded asynchronous lecture programmes, web‐based modules and tutorials, online clinical vignettes and cases, Facebook‐based discussion groups and interactive case‐based simulations. TL was defined as tutorials, lectures or text‐based learning.

Studies were excluded if they: assessed EL only as an adjunct to TL, analysed non‐web‐based modalities such as CD‐ROM video lessons and simulated emergency scenarios with electronic dolls, were specific to any other participant cohort (e.g. nursing, paramedicine and allied health services), specific to other specialties and non‐emergency scenarios, provided no comparison or an incorrect comparator, were not written in English and had no available English translation, or presented incomplete outcome data. This precluded the use of conference abstracts in this review.

Articles were sourced from OVID Medline, CINAHL, PubMed and Embase. The final searches were completed on 4 October 2020. Search results were combined on Covidence®, a systematic review management tool. ¹³ The Boolean operator ‘AND’ was employed to split our search into two components. The first focussed on the intervention with search terms including ‘online learning’, ‘computer learning’ and ‘e‐learning’ linked by the Boolean operator ‘OR’. The second component focussed on the setting of EM, with search terms including ‘Emergency medicine’, ‘Emergency medical services’ and ‘Emergenc*’, again linked by the operator ‘OR’. The final search results were imported to Covidence where duplicates were removed.

The article titles and abstract were screened independently by two reviewers (AJS and PWM) and conflicts were discussed by three reviewers (AJS, PWM and TWM). Full texts were then screened independently by two reviewers (AJS and PWM) and conflicts were discussed by three reviewers (AJS, PWM and TWM). Final full texts were then collated for data extraction.

Studies were assessed independently for risk of bias by two reviewers (AJS and PWM) according to the guidelines set in the Cochrane ‘Risk of Bias’ tool. ¹⁴ Conflicts were discussed by three reviewers (AJS, PWM and TWM). Studies were assessed for level of evidence according to the Oxford Centre for Evidence‐Based Medicine: Levels of Evidence. ¹⁵ Where the study populations and designs, as well interventions, comparators and outcomes, were sufficiently consistent (i.e. not heterogeneous) to justify aggregating the results, a meta‐analysis would be conducted utilising a random effects model. If the heterogeneity of the references precluded the validity of conducting a meta‐analysis, the results of this review were to be summarised by table and description, according to the domains of the data extraction tool. The data extraction tool would have the following headings: specific topic of intervention, type of EL and TL platform utilised, population and sample size, study method and outcomes. Only knowledge and practical skill gain would be included as outcomes, even if the study contained data on other outcomes such as student enjoyment. Studies would then be aggregated into intuitive groups based on topic of intervention, population and outcome measure, for descriptive data analysis comparing those that did and did not demonstrate a statistically significant difference between EL and TL.

Results

Description of studies

As seen in the PRISMA flow chart (Fig. 1), 1999 references were initially imported for screening, with 413 of these removed as they were duplicates. This left 1586 studies for independent screening. A total of 1453 studies were deemed irrelevant, leaving 133 for full text screening. At full‐text screening, 114 articles were excluded and 19 were included for quality assessment plus data extraction and synthesis.

Table 1 provides the details of each included publication by design, intervention, comparator, participant type and number, and method.

TABLE 1.

Study designs

Author	Year	Article type (level of evidence)	Intervention topic	EL type	TL type	Population	Study method
Alnabelsi et al.	2015	RCT (level 1b)	ENT emergencies (e.g. epistaxis and stridor)	Streamed electronic lecture	In‐person lecture	Medical students	Pre‐test → access to EL/TL → post‐test
Armstrong et al.	2009	RCT (level 1b)	ABG interpretation	Multimedia based tutorials	Lectures	Medical students on EM rotation	Access to EL/TL → test
Barthelemy et al.	2017	RCT (level 1b)	ECG interpretation (e.g. ACS and electrolyte disturbances)	Interactive PowerPoint and quizzes	Lectures	EM residents	Pre‐test → access to EL/TL → post‐test
Berland et al.	2019	Prospective cohort study (level 2b)	Opioid overdose intervention	Online modules	Lecture	Medical students	Pre‐test → access to EL/TL → post‐test
Chenkin et al.	2008	RCT (level 1b)	Ultrasound	Online tutorials	Lectures	EM residents and physicians	Pre‐test → access to EL/TL → OSCE evaluation
Chien et al.	2015	RCT (level 1b)	Laceration repair	Video‐based learning module	Lecture and practice with feedback	Medical students	Universal training, access to EL/TL → practical skill testing 7 days later → second practical assessment 70 days after the first
Edrich et al.	2016	RCT (level 1b)	Ultrasound	PowerPoint presentation and an online portfolio programme	In‐person lecture and practical sessions	EM physicians and anaesthesia residents	Pre‐test → access to EL/TL → post‐test and practical assessment within 24 h Retention test later
Everson et al.	2020	RCT (level 1b)	Various EM topics	Online modules and in‐person teaching	In‐person teaching and practical sessions	Medical students	Access to EL/TL → OSCE evaluation
Farrar et al.	2008	RCT (level 1b)	Paediatric emergencies (e.g. seizures)	Online modules	In‐person facilitated case discussion	Paediatric residents	Pre‐test → access to EL/TL → post‐test Retention test 3 months later
Jordan et al.	2013	Prospective cohort study (level 2b)	Various EM topics	Online modules	In‐person lectures	Medical students	Pre‐test → access to EL/TL → post‐test Retention test 65 days later
Kho et al.	2018	RCT (level 1b)	Emergency airway management	Online modules	In‐person lectures and practical session	Junior doctors in ED	Pre‐test → access to EL/TL → post‐test and practical assessment
Khoshbaten et al.	2014	Prospective cross‐sectional study (level 2b)	Advanced cardiac life support	Electronic software	In‐person lectures	Interns in ED	Pre‐test → access to EL/TL → post‐test
Montassier et al.	2016	RCT (level 1b)	ECG interpretation (e.g. ACS and pericarditis)	E‐learning course	Lecture‐based course	Medical students	Pre‐test → access to EL/TL → post‐test
Platz et al.	2010	RCT (level 1b)	EFAST	Web‐based didactic teaching and an in‐person practical session	In‐person lectures and an in‐person practical session	Doctors in ED	Pre‐test → access to EL/TL → post‐test Retention testing 8 weeks later following further practical training for both groups
Platz et al.	2011	RCT (level 1b)	EFAST	Narrated lectures on computers	Traditional lectures	EM and general surgery residents	Pre‐test → access to EL/TL → post‐test
Pourmand et al.	2013	RCT (level 1b)	Various EM topics	Online modules	In‐person lectures	EM residents	Access to EL/TL → test
Soleimanpour et al.	2017	RCT (level 1b)	Cardiac arrest	E‐learning course	Lecture‐based learning	EM residents	Pre‐test → access to EL/TL → post‐test
Xiao et al.	2007	RCT (level 1b)	Sterile technique for central venous catheters	Online modules	Paper version of the same course	EM, surgical and paediatric residents	Access to EL/TL → practical assessment
Ziabari et al.	2019	Prospective cohort study (level 2b)	Basic life support	Telegram software and online PowerPoint presentation	PowerPoint presentation	Medical interns in EM educational programme	Pre‐test → access to EL/TL → post‐test

Design

Of the included studies, 15 were randomised controlled trials (level 1b evidence), while the remaining four were prospective cohort studies (level 2b evidence).

Risk of bias

Studies were analysed for risk of bias (Figs 2, 3). All studies were analysed according to the Cochrane Risk of Bias tool. ¹⁴

Figure 2.

Risk of bias graph. (), Low risk of bias; (), unclear risk of bias; (), high risk of bias.

Figure 3.

Risk of bias summary.

Selection bias (random sequence generation and allocation concealment)

The studies overall were deemed to be of high or unclear risk of selection bias. Most studies used inadequate randomisation processes or were simply not randomised at all. Only eight studies were determined to have performed randomisation in an acceptable manner: with six using computer software for randomisation. ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²¹ Of the studies deemed to be of unclear risk, several claimed to be randomised but with no description of how this was achieved while one study ²² opted not to allocate participants to intervention or control groups, but rather the topics being taught. Six studies reported a method for allocation concealment.

Performance and detection bias

All studies were deemed to be at low risk of performance bias. Due to the nature of the intervention, none of the participants were able to be blinded; however, the manner in which assessment was performed makes it unlikely that this would have influenced the outcomes of any of the studies. Most studies were at low risk for detection bias as they used objective measures of performance such as multiple choice questions (MCQs) with one correct answer. Where studies used practical assessments such as objective structured clinical examinations (OSCEs) to assess learning, five blinded assessors while one study ²³ was deemed to be at high risk for detection bias as they did not blind assessors.

Attrition bias

Of the 19 studies included, three had significant dropouts or incomplete data. However, none of these studies were deemed to be at high risk of attrition bias, only being considered of unclear risk. ²⁴ , ²⁵ , ²⁶ This is due to most studies having proportionate dropouts between study groups, and clearly accounting for each of the dropouts. For the three studies deemed to be at unclear risk, the effect of dropouts could not be interpreted, as the proportions of dropouts for different reasons between the groups were not mentioned within the study.

Reporting bias

In general, most of the included articles were deemed to be at low risk for reporting bias, with the majority reporting all pre‐outlined endpoints and complete sets of results. The studies classified as unclear risk for selective reporting were labelled as such for failing to publish complete details of their results, such as missing P‐values or selectively reporting some secondary outcomes. ¹⁹ , ²³ , ²⁷

Other bias

Five studies were determined to be at high risk of other biases because of confounding or the study's structure. Two studies ²² , ²⁴ were at high risk for a ‘ceiling effect’ where students in the EL group performed statistically significantly better than those in the TL group in the pre‐test (i.e. before exposure to the learning resource). This may have decreased the improvement that further teaching would have for this group.

Summary of bias

Ultimately, the included studies were generally assessed to be at high risk of bias, most notably regarding selection bias. This may affect the generalisability of the results. Only five of the included studies were deemed to be at low risk of bias.

Participants

There were seven studies that recruited medical students to assess their respective research questions, while 12 studies evaluated junior doctors with a range of experience across specialties including anaesthetics, general surgery, paediatrics and EM.

Intervention and comparators

There were 13 studies of a pre‐post design while six studies conducted a post‐intervention evaluation only. ²³ , ²⁴ , ²⁵ , ²⁸ , ²⁹ , ³⁰

Outcome measures

As per Table 2, for outcome measures, 15 out of the 19 studies assessed knowledge gain through MCQ examinations, and six studies investigated performance on OSCEs or practical skill tests such as sterile technique compliance during central venous catheter insertion. Additionally, five studies analysed retention of knowledge. ²² , ²⁵ , ²⁶ , ²⁷ , ³¹ Retention testing assesses the ability of participants to retain knowledge after a specified time, thereby analysing the long‐term efficacy of the interventions. Timing of retention testing between studies varied, ranging from several weeks to several months. It should be noted that several studies analysed a combination of these outcomes within the same study.

TABLE 2.

Results

Author	Sample size	Outcome measure and endpoint
Alnabelsi et al.	25 TL 25 EL	15 MCQs – mean grade improvement: EL = 38.4% ± 4.5. TL = 32.8% ± 7.8 (P = 0.168)
Armstrong et al.	12 TL 9 EL	5 MCQs – mean grade: EL = 68.89% ± 10.54. TL = 73.33% ± 17.75 (P = 0.54)
Barthelemy et al.	20 TL 19 EL	10 MCQs – mean grade: Pre‐test: EL = 42.1% ± 7.3. TL = 37.5% ± 6.7 (P = 0.42) Post‐test: EL = 59.5% ± 7.7. TL = 51% ± 8.6 (P = 0.14) No significant difference between control and study group
Berland et al.	132 TL 129 EL	11 MCQs – mean grade: Pre‐test: EL = 4.7 ± 2.0. TL = 3.7 ± 1.6 (P = NS) Post‐test: EL = 9.4 ± 1.5. TL = 9.5 ± 1.7 (P = NS) No P‐values provided; however, results were reported to have no statistically significant differences
Chenkin et al.	10 TL 11 EL	Four OSCEs – mean scores: EL = 75.0%. TL = 77.8%, absolute difference −2.8% (−9.3, 3.8) (P = NS)
Chien et al.	20 TL 20 EL	Practical assessment grade intervention versus control: EL = 18.21 (17.3–19.0). TL = 18.59 (17.6–19.3) (P = 0.549)
		Retention testing (same practical assessment 70 days post‐first assessment): EL = 17.75 (16.6–19.0). TL = 17.87 (16.6–19.1) (P = 0.8979)
Edrich et al.	56 TL (class group was TL for purposes of analysis) 54 EL	10 MCQs – mean score improvement: EL = 29.3 ± 5.6. TL = 23.4 ± 6 (P = 1.00) Practical skill testing: EL group improved more than TL group (P = NS)
		Retention testing (same MCQ test 28 days post‐first post‐test): EL = 15.2 ± 6.5. TL = 12.3 ± 6.7 (P = 1.00)
Everson et al.	24 TL 24 EL	OSCE – mean score: EL = 19.58. TL = 20.86 (P = 0.705)
Farrar et al.	33 TL 33 EL	10 MCQs – mean grade: Pre‐test: EL = 49.394. TL = 50.303 (P = NS) Post‐test: EL = 63.912. TL = 54.821 (P < 0.01)
		Retention testing (same MCQ test 3 months post‐first post‐test): EL = 67.3030. TL = 65.6 (P > 0.05)
Jordan et al.	44 (specific group sizes not stated)	MCQs – score improvement intervention versus control: EL = 9.93% ± 23.22. TL = 28.39% ± 18.06 (P = 0.0001) However: pre‐test mean scores TL: 39.75% EL: 62% *therefore, ceiling effect may have prevented EL learning from achieving a similar knowledge increase*
		Retention testing (same MCQ test 65 days post‐first post‐test) – post‐test to retention test score change: EL = −17.61% ± 17.12. TL = −14.94% ± 18.73 (P = 0.399)
Kho et al.	15 TL 15 EL	MCQs – mean score improvement: EL = 18. TL = 19 (P = 0.992) Practical assessment – mean score improvement: EL = 11. TL = 10 (P = 0.461)
Khoshbaten et al.	43 TL 41 EL	21 MCQs – mean score improvement: EL = 11.88 ± 3.66. TL = 10.44 ± 3.68 (P = 0.49)
Montassier et al.	49 TL 49 EL	10 MCQs – mean score: Pre‐test: EL = 9 ± 3.0. TL = 9 ± 3.0 (P = NS) Post‐test: EL = 15.1. TL = 15.0 (P = NS) No P‐values provided; however, results were reported to have no statistically significant differences
Platz et al. (2010)	24 TL (class group was TL for purposes of analysis) 24 EL	29 MCQs – mean grade improvement: EL = 14.7% ± 4.5. TL = 18.0% ± 5.5 (P = NS)
		Retention test (same MCQ test 56 days post‐first post‐test) – score difference between EL and TL: −0.3% (95% CI −3.9% to 3.3%) (P = 0.57)
Platz et al. (2011)	22 TL 22 EL	20 MCQs – mean grade: Pre‐test: EL = 63.2%. TL = 58.0% (P < 0.05) Post‐test: EL = 81.6%. TL = 85.9% (P < 0.05) Score improvement: EL = 18.4% (SD 11.3). TL = 28% (SD 8.0) (P < 0.05) Analysis of variance framework there was significant interaction between didactic group (computer vs classroom) and training (prior training vs no prior training)
Pourmand et al.	257 TL 138 EL	MCQs – mean score above baseline: EL = 32% (26, 37). TL = 27% (22, 32) (P = NS)
Soleimanpour et al.	21 TL 23 EL	19 MCQs – mean grade: Pre‐test: EL = 8.04 ± 2.72. TL = 7.67 ± 2.29 SD Post‐test: EL = 16.17 ± 0.58. TL = 16.52 ± 1.54 SD The difference between groups was not statistically significant (P = 0.977)
Xiao et al.	14 TL (text group was TL for purposes of analysis) 14 EL Note: some participants performed more than 1 CVC.	Practical skill testing – compliance rates: EL = 73.7%. TL = 38.7% The full compliance rate in the video group was significantly higher (P = 0.003) than that in the paper and control groups with an odds ratio of 6.1 (95% CI 1.96–22.03)
Ziabari et al.	50 TL 50 EL	20 MCQs – mean improvement: EL = 3.44 ± 1.48. TL = 1.16 ± 1.51 (P < 0.0001)

MCQ, multiple choice question; NS, not significant.

Effects of intervention

Knowledge gain

Table 2 describes the study results for knowledge gain. Of the 15 studies assessing knowledge gain, each utilised their MCQs in different ways. ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ , ²² , ²⁴ , ²⁶ , ²⁷ , ²⁸ , ²⁹ , ³¹ , ³² , ³³ , ³⁴ For example, many studies employed a pre‐intervention MCQ test – access to intervention/control – post‐intervention MCQ test study design but had differing numbers of questions. One study used weekly MCQ quizzes following each week of content to compare EL to TL, and then collated their results at the end of their study. ²⁹

Of the studies analysing MCQ performance, 11 of the 15 demonstrated no statistically significant difference between EL and TL. A further two studies reported EL statistically significantly improved MCQ performance, ³¹ , ³⁴ while two studies yielded significant findings that TL outperformed EL on MCQ examinations. ²² , ³³

Four studies assessed retention of knowledge over time, with no statistically significant differences between EL and TL. ²² , ²⁶ , ²⁷ , ³¹

Practical skill testing and OSCE evaluation

Table 2 describes the study results for practical skill gain. Out of the six studies that analysed either practical skills or OSCE performance, five studies demonstrated no statistically significant difference that EL yielded greater performance than TL. ¹⁸ , ²¹ , ²³ , ²⁵ , ²⁷ One study showed significant improvement (P = 0.003) for EL compared to the TL, which involved a paper copy of the same course, surrounding the topic of sterile technique for central venous catheter insertion. ³⁰

One study assessed retention of practical skill gain over time, with no statistical significance demonstrated between EL and TL. ²⁵

Life support skills

There were three articles that compared EL and TL for education regarding life support skills. ²⁰ , ³² , ³⁴ Life support skills taught were advanced cardiac life support, basic life support and one for a general approach to ‘cardiac arrest’. Within this intuitive group, there was still significant heterogeneity between studies. Two of the three studies demonstrated no statistically significant difference between EL and TL. One study demonstrated a statistically significant difference, favouring EL over TL for basic life support education (P < 0.0001). ³⁴

Investigation interpretation

There were seven articles that compared EL and TL for education regarding investigation interpretation. ¹⁷ , ¹⁹ , ²¹ , ²⁶ , ²⁷ , ²⁸ , ³³ Investigations taught were ABG and ECG interpretation as well as ultrasound and EFAST skills. There was marked heterogeneity between sub‐topics as well as how information was delivered and tested. Six of the seven studies demonstrated no statistically significant difference between EL and TL. One study demonstrated a statistically significant difference, favouring TL over EL for EFAST training (P < 0.05). ³³

Medical students

Of the seven articles that compared EL and TL for medical students, six demonstrated no statistically significant difference ¹⁶ , ¹⁹ , ²³ , ²⁴ , ²⁵ , ²⁸ while one study favoured TL with statistical significance. ²²

Junior doctors

Of the 12 articles that compared EL and TL for junior doctors, eight demonstrated no statistically significant difference. ¹⁷ , ¹⁸ , ²⁰ , ²¹ , ²⁶ , ²⁷ , ²⁹ , ³² Three studies favoured EL with statistical significance ³⁰ , ³¹ , ³⁴ while one study favoured TL with statistical significance. ³³

Meta‐analyses

The marked heterogeneity of the study settings, designs, variable definitions and outcomes measured precluded the conduct of a meta‐analysis.

Discussion

The purpose of this systematic review was to assess the impact of EL versus TL on the EM learning outcomes of medical students and junior doctors. This systematic review suggests that EL may be comparable to TL. Nonetheless, given the overall low quality and heterogenous nature of studies, there is insufficient evidence to suggest either learning resource is superior to the other.

Effective education is vital for students and junior doctors to progress in their medical careers. At the very least, teaching should be consistent and reliable, and not disadvantageous to learners across different sites at different times. Other beneficial aspects of teaching are interactive platforms, low cost of delivery, flexibility for students, enjoyable content as well as dependable and valid assessments, which can be used to inform further teaching. ³⁵ , ³⁶ During the COVID‐19 pandemic, it is even more important that teaching can be performed at a distance and that this teaching has at least comparable learning outcomes to TL, which occurred prior.

EM represents a field in which EL may be particularly important. Due to the nature of EM scheduling, where junior doctors often work in shifts that can be at any time, teaching in person may be difficult. COVID‐19 and social distancing measures mean that time in the hospital may be further limited, especially for medical students. Given EM doctors are at the frontline of hospitals, the integration of EL can save precious in‐person time in the hospital to deliver care.

This is the first systematic review of its kind, focusing on the implications of EL in EM. Some systematic reviews and meta‐analyses have been conducted, exploring the efficacy of EL in medical education as a whole. ³⁷ , ³⁸ These have shown little difference between EL and TL resources regarding learning outcomes. Few studies have explored the efficacy of EL within specialities. One systematic review in orthopaedic surgery demonstrated that EL outperformed TL across a range of subtopics; however, the heterogenous nature of included studies precluded generalisable conclusions as in this review. ³⁹ From previous literature, it is already clear that EL is a viable form of teaching and is invariably better than no teaching. ³⁷

Of all the included studies in this review, there were appreciably more that analysed EL for knowledge gain rather than practical skill gain. This aligns with our current understanding of the uses of EL in healthcare. ³⁷ Of the studies assessing the power of EL to improve practical skills, EL and TL were alike in their efficacy.

While necessarily comparing EL to TL, the studies analysed in this review demonstrate the diversity of EL. These included online modules, multimedia platforms, interactive PowerPoints, video‐based learning modules in addition to narrated and streamed lectures. Similarly, just as the ED demands a variety of skills and knowledge from physicians, EL platforms were used for a multitude of different topics within EM. Examples include acute lifesaving interventions such as airway management and ACLS, and investigation interpretation such as ECG, ABG and ultrasound. The multiplicity of these studies underlines the prospect for EL to deliver content to students in a field, which necessitates variation.

Studies were deemed to be of the highest quality if they were a randomised controlled trial and had: a moderate size cohort of 30 or more participants, a pre‐test to assess prior knowledge between groups and no unclear or high risk of bias. Five studies fit within this subgroup, all of which demonstrated no statistically significant differences between EL and TL. ¹⁶ , ¹⁷ , ¹⁸ , ¹⁹ , ²⁰ This supports the suggestion that the two learning resources may be comparable but the heterogeneity within this subgroup precludes any strong conclusions from being drawn.

It is important to consider the generalisability of these results. Many of these studies were randomised controlled trials, which provide good insight into the differences between EL and TL for students and junior doctors who took their respective courses. Nonetheless, it is necessary to bear in mind which courses are more attractive to participants. Pourmand et al. compared EM residents who had attended their lectures with those who were absent and watched the streamed lecture online. ²⁹ Absentees who accessed these streamed lectures performed better than residents who had been present although without statistical significance. This allowed residents to gain vital coursework knowledge while still allowing them the flexibility to attend clinics and other hospital priorities or take vacation and sick leave.

There are several limitations to this review. Firstly, the included studies were generally of low quality and were heterogenous, making direct comparisons difficult. This is due to a diverse range of EL platforms, intervention topics and outcome endpoints. Hence, it is challenging to make generalisations about any specific EL modality. Secondly, while demonstrating the diverse opportunities that EL allows, the heterogeneity of the analysed studies also precludes the use of quantitative meta‐analysis. Thirdly, while medical students and junior doctors were both included in the present study because of paucity of literature, they represent distinct cohorts with slightly differing learning outcomes, adding to the heterogeneity of the analysed studies. Fourthly, this research is only preliminary, particularly with regards to practical skill development in the ED. Finally, as explained by Koens et al., knowledge assessments via MCQs may not be the best way to assess the efficacy of an intervention; however, this was the most commonly used outcome measure among the included articles. ⁴⁰ The ability of learners to apply knowledge to a clinical situation may be a more useful outcome measure when comparing EL with TL.

In addition to the limited generalisability of studies because of bias, generalisations of study outcomes onto the broad topic of EM are further limited by the narrow scope that studies on individual learning topics offer. There were numerous studies assessing the role of EL regarding education on investigation interpretation such as ABG and EFAST. This represents a topic in which multicentre randomised controlled trials that use a homogenous set of guidelines on how information is delivered and tested could allow for conclusive comparison between EL and TL. Similar suggestions can be made regarding EL versus TL on life support education. Further investigation should also seek to apply a more nuanced approach to comparisons between EL and TL. The authors encourage the implementation of the model presented by Koens et al. to compare education resources. ⁴⁰

Conclusion

Ultimately, EL platforms are developing technologies whose necessity and practicality for teaching purposes has been revealed by the recent COVID‐19 pandemic. Convenient, interactive and repeatable by nature, EL may be comparable to TL for learning outcomes of medical students and junior doctors in EM. However, given the low quality of evidence, further research should be conducted with a view to compare the efficacy of different EL platforms for specific subtopics using a homogenous set of guidelines on how information is delivered and tested. The authors encourage the integration of EL as an adjunct to face‐to‐face teaching where it is possible in EM curriculum of all levels.

Data availability statement

The data that supports the findings of this study are available in this article.