Skip to main content
PMC home page
Journal of Medical Internet Research logoLink to Journal of Medical Internet Research
. 2020 Aug 10;22(8):e16504. doi: 10.2196/16504

Blended Learning Compared to Traditional Learning in Medical Education: Systematic Review and Meta-Analysis

Alexandre Vallée 1,✉, Jacques Blacher 1, Alain Cariou 2, Emmanuel Sorbets 1
Editor: Gunther Eysenbach
Reviewed by: Yves Lecarpentier, Matthaeus Grasl
PMCID: PMC7445617  PMID: 32773378

Abstract

Background

Blended learning, which combines face-to-face learning and e-learning, has grown rapidly to be commonly used in education. Nevertheless, the effectiveness of this learning approach has not been completely quantitatively synthesized and evaluated using knowledge outcomes in health education.

Objective

The aim of this study was to assess the effectiveness of blended learning compared to that of traditional learning in health education.

Methods

We performed a systematic review of blended learning in health education in MEDLINE from January 1990 to July 2019. We independently selected studies, extracted data, assessed risk of bias, and compared overall blended learning versus traditional learning, offline blended learning versus traditional learning, online blended learning versus traditional learning, digital blended learning versus traditional learning, computer-aided instruction blended learning versus traditional learning, and virtual patient blended learning versus traditional learning. All pooled analyses were based on random-effect models, and the I2 statistic was used to quantify heterogeneity across studies.

Results

A total of 56 studies (N=9943 participants) assessing several types of learning support in blended learning met our inclusion criteria; 3 studies investigated offline support, 7 studies investigated digital support, 34 studies investigated online support, 8 studies investigated computer-assisted instruction support, and 5 studies used virtual patient support for blended learning. The pooled analysis comparing all blended learning to traditional learning showed significantly better knowledge outcomes for blended learning (standardized mean difference 1.07, 95% CI 0.85 to 1.28, I2=94.3%). Similar results were observed for online (standardized mean difference 0.73, 95% CI 0.60 to 0.86, I2=94.9%), computer-assisted instruction (standardized mean difference 1.13, 95% CI 0.47 to 1.79, I2=78.0%), and virtual patient (standardized mean difference 0.62, 95% CI 0.18 to 1.06, I2=78.4%) learning support, but results for offline learning support (standardized mean difference 0.08, 95% CI –0.63 to 0.79, I2=87.9%) and digital learning support (standardized mean difference 0.04, 95% CI –0.45 to 0.52, I2=93.4%) were not significant.

Conclusions

From this review, blended learning demonstrated consistently better effects on knowledge outcomes when compared with traditional learning in health education. Further studies are needed to confirm these results and to explore the utility of different design variants of blended learning.

Keywords: blended learning, virtual patients, online learning, computer-aided instruction, traditional learning, meta-analysis

Introduction

Background

New types of learning, such as e-learning, have become popular in medical education [1,2] since the emergence of the internet [2]. These new models allow learning to transcend boundaries of space and time; they improve collaborative and individualized learning effectiveness and are more convenient [3-5]. Nevertheless, e-learning presents some disadvantages, including high cost multimedia materials, high costs for platform maintenance, and often user training is required. In parallel, traditional learning presents several limitations, including requiring the physical presence of students and teachers at a specific time and place [6].

Blended learning is characterized by the combination of traditional face-to-face learning and asynchronous or synchronous e-learning [7]. Blended learning is a promising alternative for medical education because of its advantages over traditional learning. In academia, this learning format has had a rapid growth and is now widely used [8].

Increased research on blended learning has been reported since the 1990s [9-11]. Synthesis of these studies may inform students and teachers on the effectiveness of blended learning [12]. Previous systematic reviews have reported that blended learning has the potential to improve clinical training among medical students [13] and undergraduate nursing education [14]. In parallel, many reviews have summarized the potential of blended learning in medical education [15,16]. A meta-analysis [12] showed that blended learning was more effective than nonblended learning but with a high level of heterogeneity.

Nevertheless, these reviews were limited to only some areas of health education, and few have used quantitative synthesis in the evaluation of the effectiveness of blended learning; therefore, the purpose of this study was to quantitatively synthesize the studies that evaluated the efficacy (using knowledge outcomes) of blended learning for health education (with students, postgraduate trainees, or practitioners).

Objective

The objective of this review was to evaluate the effectiveness of blended learning for health education on knowledge outcomes assessed with subjective (eg, learner self-report) or objective evaluations (eg, multiple-choice question knowledge test) of learners’ factual or conceptual understanding of the course in studies where blended learning was compared with traditional learning.

Methods

Comparison Categories and Definitions

Blended learning was compared with traditional learning, overall, and after stratification by type of learning support, the following comparisons were made: offline blended learning versus traditional learning, online blended learning versus traditional learning, digital blended learning versus traditional learning, computer-aided instruction blended learning versus traditional learning, and virtual patient blended learning versus traditional learning.

Offline learning was defined as the use of personal computers or laptops to assist in delivering stand-alone multimedia materials without the need for internet or local area network connections [17]. These could be supplemented by videoconferences, emails, and audio-visual learning materials kept in either magnetic storage (CD-ROM, floppy disk, flash memory, multimedia cards, external hard disks) as long as the learning activities did not rely on this connection [18].

Online support was defined as all online materials used in learning courses.

Digital education was a broad construct describing a wide range of teaching and learning strategies that were exclusively based on the use of electronic media and devices as training, communication, and interactions tools [19]. These aspects could pertain to educational approaches, concepts, methods, or technologies. Moreover, these concepts facilitated remote learning, which could help address the shortage of health professionals in settings with limited resources by reducing the time constraints and geographic barriers to training.

Computer-assisted instruction was defined as the use of interactive CD-ROM, multimedia software, or audio-visual material to augment instruction including multimedia presentations, live synchronous virtual sessions offered via a web-based learning platform, presentations with audio-visuals, and synchronous or asynchronous discussion forums to enhance participation and increase engagement [20,21].

Virtual patients were defined as interactive computers simulations of real-life clinical scenarios for health professional training, education, or assessment. This broad definition encompassed a variety of systems that used different technologies and addressed various learning needs [22].

Traditional learning, in this paper, was used to describe all nonblended learning such as nondigital and not online, but also only online, only e-learning, or other single support educational methods (lectures, face-to-face, reading exercises, group discussion in classroom).

Reporting Standards

We conducted and reported our study according to PRISMA guidelines [23] and Cochrane systematic review guidelines [24].

Eligibility Criteria

Inclusion criteria for studies were based on the PICOS (population, intervention, comparison, outcome, and study design) framework.

Studies were included if they were conducted among health learners, used a blended learning intervention in the experimental group, involved a comparison of blended learning with traditional learning, included quantitative outcomes with respect to knowledge assessed with either subjective or objective evaluations, and were randomized controlled trials or nonrandomized studies (which are widely used in health education). Only studies published in English were included.

Data Sources

To identify relevant studies, we conducted a search of citations published in MEDLINE between January 1990 and July 2019. Key search terms included delivery concepts (blended, hybrid, integrated, computer-aided, computer assisted, virtual patient, learning, training, education, instruction, teaching, course), participant characteristics (physician, medic*, nurs*, pharmac*, dent*, health*), and study design concepts (compar*, trial*, evaluat*, assess*, effect*, pretest*, pre-test, posttest*, post-test, preintervention, pre-intervention, postintervention, post-intervention). Asterisks were used as a truncation symbol for searching. Multimedia Appendix 1 describes the complete research strategy.

Study Selection

Using the eligibility criteria, AV and ES independently screened all articles and abstracts and reviewed the full text of potentially eligible abstracts.

Data Extraction

AV and ES independently extracted relevant characteristics related to participants, intervention, comparators, outcome measures, and results from the studies that were found to be eligible using a standard data collection form. Any disagreements were resolved through discussion with a third research team member until agreement was reached.

Risk of Bias Assessment

During the data extraction process, researchers independently assessed the risk of bias for each study using the Cochrane Collaboration’s risk of bias tool [25]. Evaluation criteria included the following: random sequence generation, allocation concealment, blinding of students and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, or other which included publication bias. Funnel plots were used to evaluate publication bias. Risk of bias for each criterion was rate as low, high, or unclear according to the Cochrane risk of bias instructions.

Data Synthesis

Analyses were performed for knowledge outcomes using SAS software (version 9.4; SAS Institute). The standardized mean difference (standard mean difference; Hedges g effect size), converted from means and standard deviation from each study, was used [15]. When the mean was available, but the standard deviation was not, we used the mean standard deviation of all other studies. Since the overall scores of included studies were not the same and standard mean difference could eliminate the effects of absolute values, we adjusted the mean and standard deviation so that the average standard deviation could replace the missing value of standard deviation. We employed a random-effects model for the meta-analysis (statistically significant if P<.05).

The I2 statistic was used to quantify heterogeneity across studies [26]. When the estimated I2 was equal to or greater than 50%, this indicated a large amount of heterogeneity. As the studies were functionally different and involved different study designs, participants, interventions, and settings; a random-effects model that allowed more heterogeneity was used. Forest plots were created to display the meta-analysis findings. To explore publication bias, funnel plots were created and Begg tests were performed (statistically significant if P<.05). To explore potential sources of heterogeneity, multiple meta-regression and subgroup analyses based on the study design were performed. Sensitivity analyses to test the robustness of findings were also performed.

Results

Study Selection

The search strategy identified 3389 articles from MEDLINE. After scanning the titles and abstracts, 93 articles were found to be potentially eligible, and their full texts were read for further assessment. Of these, 56 articles were included [9-11,22,27-78] (Figure 1). All articles that were included had been published in peer-reviewed journal.

Figure 1.

Figure 1

Open in a new tab

Study flow diagram.

Type of Participants

In the 56 articles, 9943 participants were included. In 30 out of 56 participant subgroups, participants were from the field of medicine [11,31-34,36,38,39,41,43,44,46-50,53,64-67,69-74, 76,78,79]. The participant subgroups from fields other than medicine were as follows: 16 studies in nursing [9,10,12,27,29,35,37,40,51,52,57,58,61-63,75], 1 in pharmacy [37], 3 in physiotherapy [12,30,45], 5 in dentistry [10,42, 54,55,59], and 4 interprofessional education [56,60,68,77].

Of the 56 studies, 47 were conducted in high-income countries: 14 were from the United States [9,10,29,31,36,38,43,50, 53-55,59,61,73], 2 from Canada [47,58], 5 from Germany [39,41,46,57,76], 3 from the United Kingdom [42,56,75], 3 from Spain [30,45,48], 1 from France [74], 1 from Greece [34], 1 from Sweden [67], 1 from the Netherlands [37], 1 from Korea [40], 1 from Poland [79], 1 from Serbia [70], 1 from Croatia [64], 1 from Turkey [32], 2 from Taiwan [28,51], 1 from Japan [69], and 7 from Australia [44,49,63,65,66,68,78]. Of the 56 studies, 9 studies were conducted in low- or middle-income countries: 2 from Thailand [52,62], 1 from China [77], 1 from Malaysia [72], 2 from Iran [27,71], 1 from Jordan [35], 1 from South Africa [11], and 1 from Uruguay [60].

The technical characteristics of the blended learning systems, topics of educational content, applied design methods, and other information on the validity of outcome measurements can be found in Multimedia Appendix 1.

Effects of Interventions

Blended Learning Versus Traditional Learning

The pooled effect size reflected a significantly large effect on knowledge outcome (standard mean difference 1.07, 95% CI 0.85 to 1.28, z=9.72, n=9943, P<.001). A significant heterogeneity was observed among studies (I2=94.3%). Figure 2 shows details of the main analysis. The test of asymmetry funnel plot (Figure 3) indicated publication bias among studies (Begg test P=.01). The trim and fill method indicated that the effect size changed to 0.41 (95% CI 0.16 to 0.66, P<.001) after adjusting for publication bias, which suggested that blended learning was more effective than traditional learning.

Figure 2.

Figure 2

Open in a new tab

Forest plot of blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval; SMD: standard mean difference.

Figure 3.

Figure 3

Open in a new tab

Funnel plot of blended learning versus traditional learning.

Offline Blended Learning Versus Traditional Learning

Of the 3 studies [27-29] comparing offline blended learning to traditional learning, in 2 studies [27,28], the groups with blended resources scored better than their corresponding control groups with significant positive standard mean differences. The other study did not show a statistically significant difference in knowledge outcome (standard mean difference 0.08, 95% CI –0.63 to 0.79) [29]. The pooled effect for knowledge outcomes suggested no significant effects from offline blended learning over traditional education alone (standard mean difference 0.67, 95% CI –0.50 to 1.84, I2=87.9%, n=327) (Figure 4).

Figure 4.

Figure 4

Open in a new tab

Forest plot of offline blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval.

Online Blended Learning Versus Traditional Learning

In studies comparing online blended learning to traditional learning, 26 [34-37,39-41,43,46,50,53-55,58,60,64,69-72, 74,75,77,78] of the 41 studies [34-47,50,51,53-55,57,58,60,62, 64,68-72,74,75,77,78] showed that groups with blended learning had better scores than those of their corresponding control groups. The pooled effect for knowledge outcomes was a standard mean difference of 0.73 (95% CI 0.60 to 0.86, n=6976) (Figure 5). There was a substantial amount of heterogeneity in the pooled analysis (I2=94.9%).

Figure 5.

Figure 5

Open in a new tab

Forest plot of online blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval; SMD: standard mean difference.

Digital Learning Versus Traditional Learning

Only 3 [32,33,63] of 7 studies [30-33,56,63,76] comparing digital learning to traditional learning presented a better score than the control group. The pooled effect for knowledge outcomes suggested no significant effects between blended and traditional learning (standard mean difference 0.04, 95% CI –0.45 to 0.52, I2=93.4%, n=1093) (Figure 6).

Figure 6.

Figure 6

Open in a new tab

Forest plot of digital blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval.

Computer-Assisted Instruction Blended Learning Versus Traditional Learning

Of the studies focusing on computer-assisted instruction blended learning, 5 [10,11,48,49,73] of the 8 studies [9-11,38,48, 49,52,73] showed significantly higher scores than those of traditional learning. Only 1 study [38] showed a significant negative effect compared to traditional learning (standard mean difference –0.68, 95% CI –1.32 to –0.04). The other studies showed no significant difference [9,52]. The pooled effect for knowledge outcomes suggested a significant improvement of computer-assisted instruction blended with traditional education over traditional education alone (standard mean difference 1.13, 95% CI 0.47 to 1.79, I2=78.0%, n=926) (Figure 7).

Figure 7.

Figure 7

Open in a new tab

Forest plot of computer-assisted instruction blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval.

Virtual Patient Blended Learning Versus Traditional Learning

In 4 [59,65,66,79] of the 5 studies [59,65-67,79] on knowledge outcomes when using virtual patients as a supplement to traditional learning, the groups with supplementary virtual patient learning support scored better than their corresponding control groups. Only 1 study with virtual patients did not show a statistically significant difference in knowledge outcomes (standard mean difference 0.13, 95% CI –0.30 to 0.56) [67]. The pooled effect for knowledge outcomes suggested significant effects for virtual patient blended learning (standard mean difference 0.62, 95% CI 0.18 to 1.06, I2=78.4%, n=621) (Figure 8).

Figure 8.

Figure 8

Open in a new tab

Forest plot of virtual patient blended learning to traditional learning comparison for knowledge outcomes. df: degree of freedom; CI: confidence interval.

Sensitivity Analyses

None of the subgroup analyses that were initially planned explained the heterogeneity of the results. Among many analyzed aspects, we considered the differences regarding the efficiency of learning with blended learning between the health professions disciplines. Most of the studies involved students of medicine as participants (30/56, 54%).

When analyzing knowledge outcomes in medicine, nursing, and dentistry, some differences were apparent. The pooled effect of medicine studies showed a standard mean difference of 0.91 (95% CI 0.65 to 1.17, z= 6.77, I2=95.8%, n=3418, P<.001) (Figure 9), nursing studies showed a standard mean difference of 0.75 (95% CI 0.26 to 1.24, z=2.99, I2=94.9%, n=1590, P=.008) (Figure 10), and dentistry studies showed a standard mean difference of 0.35 (95% CI 0.17 to 0.53, z=3.78, I2=37.6%, n=1130, P=<.001) (Figure 11). Dentistry studies included 3 online blended learning studies (standard mean difference 0.37, 95% CI 0.14 to 0.64, z=2.63, I2=58.3%, n=879), 1 virtual patient learning study, and 1 computer-assisted instruction learning study.

Figure 9.

Figure 9

Open in a new tab

Forest plot of blended learning to traditional learning comparison for knowledge outcomes for medical students. df: degree of freedom; CI: confidence interval.

Figure 10.

Figure 10

Open in a new tab

Forest plot of blended learning to traditional learning comparison for knowledge outcomes for nurses as students. df: degree of freedom; CI: confidence interval.

Figure 11.

Figure 11

Open in a new tab

Forest plot of blended learning to traditional learning comparison for knowledge outcomes for dentistry students. df: degree of freedom; CI: confidence interval.

Additional interest was observed for offline blended learning in nursing compared to traditional learning (standard mean difference 1.28, 95% CI 0.25 to 2.31, z=2.43, I2=86.2%, n=249), and in computer-assisted instruction (standard mean difference 0.53, 95% CI 0.17 to 0.90, z=2.84, I2=23.9%, n=174), but not for online blended learning (standard mean difference 0.68, 95% CI –0.07 to 1.45, z=1.76, I2=96.7%, n=1091).

Additional interest was observed for digital blended learning compared to traditional learning in medicine (standard mean difference 0.26, 95% CI 0.07 to 0.45, z=2.71, I2=95.6%, n=417) [31-33,76], in virtual patient (standard mean difference 0.71, 95% CI 0.14 to 1.28, z=2.45, I2=85.8%, n=416) [65-67,79], in online (standard mean difference 1.26, 95% CI 0.81 to 1.71, z=5.49, I2=96.1%, n=1879) [34,36,38,39,41,43,44,46,47,50,53, 64,69-72,74,78], and in computer-aided-instruction (standard mean difference 2.1, 95% CI 0.68 to 3.44, z=2.91, I2=97.9%, n=706) [11,38,48,49,73] suggesting more positive effects of blended learning over traditional learning alone for learning in medicine.

Risk of Bias

Risk of bias is shown in Figure 12. The risk of bias of evaluators was avoided in several studies by using automated assessment instruments. Thus, we rated the risk as low in 50 of the 56 studies. Nevertheless, it was still unclear whether the instruments had been correctly validated. Attribution bias was within acceptable levels in some studies (low risk in 24 of the 56 studies), but this did not exclude voluntary bias and its influence on the estimated effect. Reporting bias was considered low in 28 of the 56 studies.

Figure 12.

Figure 12

Open in a new tab

Risk of bias summary (+ low risk of bias; - high risk of bias; ? unclear risk of bias).

We cannot consider allocation bias as a significant problem in this review because, if studies described an adequate randomization method or an unclear description, it was assumed that randomization was unlikely be defective. Performance bias on traditional learning may be a problem, but it is impossible to avoid in this type of research. It is possible to blind participants in blended learning design comparisons, but these studies are still rare in the literature. We cannot reliably estimate publication bias given the high degree of heterogeneity of the included studies.

Discussion

Principal Findings

This meta-analysis provided several findings. First, blended learning had a large consistent positive effect (standard mean difference 1.07, 95% CI 0.85 to 1.28) on knowledge acquisition in comparison to traditional learning in health professions. A possible explanation could be that, compared to traditional learning, blended learning allowed students to review electronic materials as often as necessary and at their own pace, and this likely enhanced learning performance [80]. The trim and fill method showed that the pooled effect size changed to 0.41 (95% CI 0.16 to 0.66), meaning that blended learning remained more effective than traditional learning. The strength of this meta-analysis was that it reinforced previous results [12]; however, a large heterogeneity was observed across the studies. The participant subgroup analyses partially explained these differences.

The effectiveness of blended learning is complex and dependent on how well the evaluation fits, since it occurs before the implementation of any innovation as well as allowing planners to determine the needs, considering participant characteristics, analyzing contextual matters, and gathering baseline information [81]. Some interventional studies have highlighted the potential of blended learning to increase course completion rates, improve retention, and increase student satisfaction [82]. Nevertheless, comparisons between blended learning environments and the disparity between academic achievement or grade dispersions have been studied; no significant differences were observed [83]. The effectiveness of blended learning may be dependent on student characteristics, design features, and learning outcomes. Learner success is dependent on the ability to cope with technical difficulty, technical skills, and knowledge in computer operations and internet navigation. Thus, the success of blended learning is highly dependent on experience with the internet and computer apps. Some studies have observed that the success of blended learning was largely associated on the capability to participate in blended course. A previous study [84] showed that high motivation among blended learning led to persistence in their courses. Moreover, time management is a crucial effectiveness factor for successful online learning [85].

Second, offline blended learning did not show a positive pooled effect compared to traditional learning; however, 2 of the 3 studies were in nursing. These results were consistent with a previous meta-analysis on offline digital education [86]. Nevertheless, potential benefits of offline education such as unrestrained knowledge transfer and enriched accessibility of health education have previously been suggested [87]. These interventions could be focused on an interactive, associative, and perceptual learning experience by text, images, audio-video, or other components [88,89].

Third, the effect of digital learning on knowledge outcomes presented inconsistent effects according to the group or subgroup analysis. Overall, the 8 digital blended learning studies showed a nonsignificant effect compared to traditional learning whereas in the medicine subgroup, digital learning had a positive effect (standard mean difference 0.26, 95% CI 0.07 to 0.45). Previous studies [18,90] have shown similar results. Nevertheless, George et al [18] showed the effectiveness of digital learning for undergraduate health professionals compared to traditional learning.

Fourth, in the 10 studies related to computer-assisted instruction, we observed a significant difference in knowledge acquisition outcomes. Furthermore, the difference was higher in the medicine subgroup. This finding must be interpreted with caution because of the high level of heterogeneity (all computer-assisted instruction: I2=78.0%; medicine computer-assisted instruction: I2=97.9%). Previous studies showed that computer-assisted instruction was equally as effective as traditional learning [91]. Nevertheless, the results of these studies also had high levels of heterogeneity and require cautious interpretation. We believe that a comparative approach focusing on the differences in intervention design, sample characteristics, and context of learning is needed to better understand the effectiveness of computer-assisted instruction. Computer-assisted instruction could be perceived negatively by some students and impact outcomes.

Fifth, the participants in Al-Riyami et al’s study [92] reported difficulties accessing the course because of network difficulties with university’s server and internet; therefore, the asynchronous features of the discussion boards were not used to their full potential in this study. Both problems could have emerged regardless of the online course. In traditional learning, students may choose not to engage in discussions, and internet connectivity issues can happen anywhere. This supports the contention above that local conditions, rather than a general effect, may render one or the other mode of instruction preferable to the other.

Sixth, virtual patient blended learning had an overall positive pooled effect when compared to traditional learning on knowledge outcomes; this was also found in a similar meta-analysis [93]. Our observations also supplement the evidence in previous reviews [94,95] which included studies since 2010. Nevertheless, virtual patient simulations predominantly affect skill rather than knowledge outcomes. This could explain the low number of studies and the low added value of virtual patient in comparison to traditional learning. Virtual patients have greater impact in skills training, in applying problem solving, and when direct patient contact is not possible [93]. As proposed by Cook and Triola [96], virtual patients can be said to be a modality for learning in which learners actively use and train their clinical reasoning and critical thinking abilities before bedside learning [96]. Nevertheless, some exceptions can be noted. A need for more guidance within virtual patient simulations may appear in studies with different instructional methods where narrative virtual patient design was better than more autonomous problem-oriented designs [97]. Feedback given by humans in a virtual patient system was better than an animated backstory in increasing empathy [98], but no feedback had no more positive result on the outcomes than learning from a virtual patient scenario [99]. This reminds us that presenting realistic patient scenarios with a great degree of freedom may not be an excuse for neglecting guidance in relation to learning objectives [100].

Strengths and Limitations

This meta-analysis had numerous strengths. An evaluation of the effectiveness of blended learning for health professions is timely and very important for both health educators and learners. We intentionally kept our scope broad in terms of learning topic and included all studies with learners from health professions.

The samples used in this study consisted in various health professionals (in medicine, nursing, dentistry, and others) across a wide variety of health care disciplines. Although, these observations could explain the high level of heterogeneity, we found moderate or large effects for the pooled effects sizes of almost all subgroup analyses exploring variations in participant types. Thus, these results could suggest that health care learning should use blended learning in several and various disciplines of health learning.

However, some limitations must be considered. The systematic literature search encompassed one database (MEDLINE) with few exclusion criteria. The quality of the analyses was dependent on the quality of data from the included studies. Although the standard deviation of some interventions was not available due to poor reporting, we used the average standard deviation of the other studies and imputed effect sizes with concomitant potential for error. Results of subgroup analyses should be interpreted with caution because of the absence of a priori hypotheses in some cases, such as study design, country socioeconomic status, and outcome assessment. In addition, since variability of study interventions, assessment instruments, circumstances were not assessed and could be potential sources of heterogeneity, this should also be cause for cautious interpretation of results. Finally, publication bias was also found.

Conclusions

This study has implications for research on blended learning in health professions. Even though conclusions could be weakened by heterogeneity across studies, the results of this synthesis reinforced that blended learning may have a positive effect on knowledge acquisition related to health professions. Blended learning could be promising and worthwhile for further application in health professions. The difference in effects across subgroup analyses of health population indicated that different methods of conducting blended courses could demonstrate differing effectiveness. Therefore, researchers and educators should pay attention to how to implement a blended course effectively. This question may be answered successfully through studies directly comparing different blended instructional methods.

Appendix

Multimedia Appendix 1

Supplemental appendix.

Footnotes

Authors' Contributions: AV performed the design and conceived the original idea. AV and ES read the articles and selected the articles. AV performed the statistical analyses. AV wrote the manuscript. ES, AC, and JB participated in the writing of the manuscript.

Conflicts of Interest: None declared.

References

  • 1.Bediang G, Stoll B, Geissbuhler A, Klohn AM, Stuckelberger A, Nko'o S, Chastonay P. Computer literacy and e-learning perception in Cameroon: the case of Yaounde Faculty of Medicine and Biomedical Sciences. BMC Med Educ. 2013;13:57. doi: 10.1186/1472-6920-13-57. http://www.biomedcentral.com/1472-6920/13/57. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Choules AP. The use of elearning in medical education: a review of the current situation. Postgrad Med J. 2007 Apr;83(978):212–6. doi: 10.1136/pgmj.2006.054189. http://europepmc.org/abstract/MED/17403945. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Yu S, Yang K. Attitudes toward web-based distance learning among public health nurses in Taiwan: a questionnaire survey. Int J Nurs Stud. 2006 Aug;43(6):767–74. doi: 10.1016/j.ijnurstu.2005.09.005. [DOI] [PubMed] [Google Scholar]
  • 4.Peng Y, Wu X, Atkins S, Zwarentein M, Zhu M, Zhan XX, Zhang F, Ran P, Yan WR. Internet-based health education in China: a content analysis of websites. BMC Med Educ. 2014;14:16. doi: 10.1186/1472-6920-14-16. http://www.biomedcentral.com/1472-6920/14/16. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Moreira IC, Ventura SR, Ramos I, Rodrigues PP. Development and assessment of an e-learning course on breast imaging for radiographers: a stratified randomized controlled trial. J Med Internet Res. 2015;17(1):e3. doi: 10.2196/jmir.3344. http://www.jmir.org/2015/1/e3/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Kemp N, Grieve R. Face-to-face or face-to-screen? Undergraduates' opinions and test performance in classroom vs. online learning. Front Psychol. 2014;5:1278. doi: 10.3389/fpsyg.2014.01278. doi: 10.3389/fpsyg.2014.01278. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Kim K, Bonk CJ, Oh E. The present and future state of blended learning in workplace learning settings in the United States. Perf. Improv. 2008 Sep;47(8):5–16. doi: 10.1002/pfi.20018. [DOI] [Google Scholar]
  • 8.Dziuban C, Graham CR, Moskal PD, Norberg A, Sicilia N. Blended learning: the new normal and emerging technologies. Int J Educ Technol High Educ. 2018 Feb 15;15(1):15–3. doi: 10.1186/s41239-017-0087-5. [DOI] [Google Scholar]
  • 9.Mangione S, Nieman LZ, Greenspon LW, Margulies H. A comparison of computer-assisted instruction and small-group teaching of cardiac auscultation to medical students. Med Educ. 1991 Sep;25(5):389–95. doi: 10.1111/j.1365-2923.1991.tb00086.x. [DOI] [PubMed] [Google Scholar]
  • 10.Rouse DP. The effectiveness of computer-assisted instruction in teaching nursing students about congenital heart disease. Comput Nurs. 2000;18(6):282–7. [PubMed] [Google Scholar]
  • 11.Mars M, McLean M. Students' perceptions of a multimedia computer-aided instruction resource in histology. S Afr Med J. 1996 Sep;86(9):1098–102. [PubMed] [Google Scholar]
  • 12.Liu Q, Peng W, Zhang F, Hu R, Li Y, Yan W. The effectiveness of blended learning in health professions: systematic review and meta-analysis. J Med Internet Res. 2016;18(1):e2. doi: 10.2196/jmir.4807. http://www.jmir.org/2016/1/e2/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Rowe M, Frantz J, Bozalek V. The role of blended learning in the clinical education of healthcare students: a systematic review. Med Teach. 2012;34(4):e216–21. doi: 10.3109/0142159X.2012.642831. [DOI] [PubMed] [Google Scholar]
  • 14.McCutcheon K, Lohan M, Traynor M, Martin D. A systematic review evaluating the impact of online or blended learning vs. face-to-face learning of clinical skills in undergraduate nurse education. J Adv Nurs. 2015 Feb;71(2):255–70. doi: 10.1111/jan.12509. [DOI] [PubMed] [Google Scholar]
  • 15.Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Internet-based learning in the health professions: a meta-analysis. JAMA. 2008 Sep 10;300(10):1181–96. doi: 10.1001/jama.300.10.1181. [DOI] [PubMed] [Google Scholar]
  • 16.Jwayyed S, Stiffler KA, Wilber ST, Southern A, Weigand J, Bare R, Gerson LW. Technology-assisted education in graduate medical education: a review of the literature. Int J Emerg Med. 2011;4:51. doi: 10.1186/1865-1380-4-51. http://www.intjem.com/content/4//51. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Rasmussen K, Belisario JM, Wark PA, Molina JA, Loong SL, Cotic Z, Papachristou N, Riboli-Sasco E, Tudor CL, Musulanov EM, Kunz H, Zhang Y, George PP, Heng BH, Wheeler EL, Al SN, Svab I, Atun R, Majeed A, Car J. Offline eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health. 2014 Jun;4(1):010405. doi: 10.7189/jogh.04.010405. http://europepmc.org/abstract/MED/24976964. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.George PP, Papachristou N, Belisario JM, Wang W, Wark PA, Cotic Z, Rasmussen K, Sluiter R, Riboli-Sasco E, Tudor CL, Musulanov EM, Molina JA, Heng BH, Zhang Y, Wheeler EL, Al SN, Majeed A, Car J. Online eLearning for undergraduates in health professions: a systematic review of the impact on knowledge, skills, attitudes and satisfaction. J Glob Health. 2014 Jun;4(1):010406. doi: 10.7189/jogh.04.010406. http://europepmc.org/abstract/MED/24976965. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Martinengo L, Yeo NJY, Tang ZQ, Markandran KD, Kyaw BM, Tudor Car L. Digital education for the management of chronic wounds in health care professionals: protocol for a systematic review by the Digital Health Education Collaboration. JMIR Res Protoc. 2019 Mar 25;8(3):e12488. doi: 10.2196/12488. http://www.researchprotocols.org/2019/3/e12488/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Bandla H, Franco RA, Simpson D, Brennan K, McKanry J, Bragg D. Assessing learning outcomes and cost effectiveness of an online sleep curriculum for medical students. J Clin Sleep Med. 2012 Aug 15;8(4):439–43. doi: 10.5664/jcsm.2042. doi: 10.5664/jcsm.2042. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Viljoen CA, Scott Millar R, Engel ME, Shelton M, Burch V. Is computer-assisted instruction more effective than other educational methods in achieving ECG competence among medical students and residents? Protocol for a systematic review and meta-analysis. BMJ Open. 2017 Dec 26;7(12):e018811. doi: 10.1136/bmjopen-2017-018811. http://bmjopen.bmj.com/cgi/pmidlookup?view=long&pmid=29282268. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 22.Kononowicz AA, Zary N, Edelbring S, Corral J, Hege I. Virtual patients - what are we talking about? a framework to classify the meanings of the term in healthcare education. BMC Med Educ. 2015;15:11. doi: 10.1186/s12909-015-0296-3. http://www.biomedcentral.com/1472-6920/15/11. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 23.Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. J Clin Epidemiol. 2009 Oct;62(10):e1–34. doi: 10.1016/j.jclinepi.2009.06.006. http://linkinghub.elsevier.com/retrieve/pii/S0895-4356(09)00180-2. [DOI] [PubMed] [Google Scholar]
  • 24.Higgins J, Green S. editors. Cochrane Handbook for Systematic Reviews of Interventions Internet. Chichester, UK: John Wiley & Sons, Ltd; 2008. [Google Scholar]
  • 25.Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JAC, Cochrane BMG, Cochrane SMG. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ. 2011;343:d5928. doi: 10.1136/bmj.d5928. http://europepmc.org/abstract/MED/22008217. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Higgins JPT, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ. 2003 Sep 6;327(7414):557–60. doi: 10.1136/bmj.327.7414.557. http://europepmc.org/abstract/MED/12958120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Ebadi A, Yousefi S, Khaghanizade M, Saeid Y. Assessment competency of nurses in biological incidents. Trauma Mon. 2015 Nov;20(4):e25607. doi: 10.5812/traumamon.25607. http://europepmc.org/abstract/MED/26839862. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 28.Liu W, Chu K, Chen S. The development and preliminary effectiveness of a nursing case management e-learning program. Comput Inform Nurs. 2014 Jul;32(7):343–52. doi: 10.1097/CIN.0000000000000050. [DOI] [PubMed] [Google Scholar]
  • 29.Bayne T, Bindler R. Effectiveness of medication calculation enhancement methods with nurses. J Nurs Staff Dev. 1997;13(6):293–301. [PubMed] [Google Scholar]
  • 30.Fernández-Lao C, Cantarero-Villanueva I, Galiano-Castillo N, Caro-Morán E, Díaz-Rodríguez L, Arroyo-Morales M. The effectiveness of a mobile application for the development of palpation and ultrasound imaging skills to supplement the traditional learning of physiotherapy students. BMC Med Educ. 2016 Oct 19;16(1):274. doi: 10.1186/s12909-016-0775-1. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0775-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Johnston R, Hepworth J, Goldsmith M, Lacasse C. Use of iPod™ technology in medical-surgical nursing courses: effect on grades. Int J Nurs Educ Scholarsh. 2010;7:Article43. doi: 10.2202/1548-923X.2092. [DOI] [PubMed] [Google Scholar]
  • 32.Küçük S, Kapakin S, Göktaş Y. Learning anatomy via mobile augmented reality: effects on achievement and cognitive load. Anat Sci Educ. 2016 Oct;9(5):411–21. doi: 10.1002/ase.1603. [DOI] [PubMed] [Google Scholar]
  • 33.Kulier R, Gülmezoglu AM, Zamora J, Plana MN, Carroli G, Cecatti JG, Germar MJ, Pisake L, Mittal S, Pattinson R, Wolomby-Molondo J, Bergh A, May W, Souza JP, Koppenhoefer S, Khan KS. Effectiveness of a clinically integrated e-learning course in evidence-based medicine for reproductive health training: a randomized trial. JAMA. 2012 Dec 5;308(21):2218–25. doi: 10.1001/jama.2012.33640. [DOI] [PubMed] [Google Scholar]
  • 34.Kavadella A, Tsiklakis K, Vougiouklakis G, Lionarakis A. Evaluation of a blended learning course for teaching oral radiology to undergraduate dental students. Eur J Dent Educ. 2012 Feb;16(1):e88–95. doi: 10.1111/j.1600-0579.2011.00680.x. [DOI] [PubMed] [Google Scholar]
  • 35.Sowan AK, Jenkins LS. Use of the seven principles of effective teaching to design and deliver an interactive hybrid nursing research course. Nurs Educ Perspect. 2013;34(5):315–22. doi: 10.5480/1536-5026-34.5.315. [DOI] [PubMed] [Google Scholar]
  • 36.Lancaster JW, Wong A, Roberts SJ. 'Tech' versus 'talk': a comparison study of two different lecture styles within a Master of Science nurse practitioner course. Nurse Educ Today. 2012 Jul;32(5):e14–8. doi: 10.1016/j.nedt.2011.09.018. [DOI] [PubMed] [Google Scholar]
  • 37.Dankbaar MEW, Storm DJ, Teeuwen IC, Schuit SCE. A blended design in acute care training: similar learning results, less training costs compared with a traditional format. Perspect Med Educ. 2014 Sep;3(4):289–99. doi: 10.1007/s40037-014-0109-0. http://europepmc.org/abstract/MED/24477971. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Shomaker TS, Ricks DJ, Hale DC. A prospective, randomized controlled study of computer-assisted learning in parasitology. Acad Med. 2002 May;77(5):446–9. doi: 10.1097/00001888-200205000-00022. [DOI] [PubMed] [Google Scholar]
  • 39.Mahnken AH, Baumann M, Meister M, Schmitt V, Fischer MR. Blended learning in radiology: is self-determined learning really more effective? Eur J Radiol. 2011 Jun;78(3):384–7. doi: 10.1016/j.ejrad.2010.12.059. [DOI] [PubMed] [Google Scholar]
  • 40.Sung YH, Kwon IG, Ryu E. Blended learning on medication administration for new nurses: integration of e-learning and face-to-face instruction in the classroom. Nurse Educ Today. 2008 Nov;28(8):943–52. doi: 10.1016/j.nedt.2008.05.007. [DOI] [PubMed] [Google Scholar]
  • 41.Woltering V, Herrler A, Spitzer K, Spreckelsen C. Blended learning positively affects students' satisfaction and the role of the tutor in the problem-based learning process: results of a mixed-method evaluation. Adv Health Sci Educ Theory Pract. 2009 Dec;14(5):725–38. doi: 10.1007/s10459-009-9154-6. [DOI] [PubMed] [Google Scholar]
  • 42.Lowe CI, Wright JL, Bearn DR. Computer-aided Learning (CAL): an effective way to teach the Index of Orthodontic Treatment Need (IOTN)? J Orthod. 2001 Dec;28(4):307–11. doi: 10.1093/ortho/28.4.307. [DOI] [PubMed] [Google Scholar]
  • 43.Hilger AE, Hamrick HJ, Denny FW. Computer instruction in learning concepts of streptococcal pharyngitis. Arch Pediatr Adolesc Med. 1996 Jun;150(6):629–31. doi: 10.1001/archpedi.1996.02170310063011. [DOI] [PubMed] [Google Scholar]
  • 44.Ilic D, Hart W, Fiddes P, Misso M, Villanueva E. Adopting a blended learning approach to teaching evidence based medicine: a mixed methods study. BMC Med Educ. 2013;13:169. doi: 10.1186/1472-6920-13-169. http://www.biomedcentral.com/1472-6920/13/169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 45.Arroyo-Morales M, Cantarero-Villanueva I, Fernández-Lao C, Guirao-Piñeyro M, Castro-Martín E, Díaz-Rodríguez L. A blended learning approach to palpation and ultrasound imaging skills through supplementation of traditional classroom teaching with an e-learning package. Man Ther. 2012 Oct;17(5):474–8. doi: 10.1016/j.math.2012.04.002. [DOI] [PubMed] [Google Scholar]
  • 46.Raupach T, Münscher C, Pukrop T, Anders S, Harendza S. Significant increase in factual knowledge with web-assisted problem-based learning as part of an undergraduate cardio-respiratory curriculum. Adv Health Sci Educ Theory Pract. 2010 Aug;15(3):349–56. doi: 10.1007/s10459-009-9201-3. http://europepmc.org/abstract/MED/19774475. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Carbonaro M, King S, Taylor E, Satzinger F, Snart F, Drummond J. Integration of e-learning technologies in an interprofessional health science course. Med Teach. 2008 Feb;30(1):25–33. doi: 10.1080/01421590701753450. [DOI] [PubMed] [Google Scholar]
  • 48.Pereira J, Palacios M, Collin T, Wedel R, Galloway L, Murray A, Violato C, Lockyer J. The impact of a hybrid online and classroom-based course on palliative care competencies of family medicine residents. Palliat Med. 2008 Dec;22(8):929–37. doi: 10.1177/0269216308094561. [DOI] [PubMed] [Google Scholar]
  • 49.Devitt P, Smith JR, Palmer E. Improved student learning in ophthalmology with computer-aided instruction. Eye (Lond) 2001 Oct;15(Pt 5):635–9. doi: 10.1038/eye.2001.199. [DOI] [PubMed] [Google Scholar]
  • 50.Kiviniemi MT. Effects of a blended learning approach on student outcomes in a graduate-level public health course. BMC Med Educ. 2014;14:47. doi: 10.1186/1472-6920-14-47. http://www.biomedcentral.com/1472-6920/14/47. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 51.Hsu L, Hsieh S. Effects of a blended learning module on self-reported learning performances in baccalaureate nursing students. J Adv Nurs. 2011 Nov;67(11):2435–44. doi: 10.1111/j.1365-2648.2011.05684.x. [DOI] [PubMed] [Google Scholar]
  • 52.Kaveevivitchai C, Chuengkriankrai B, Luecha Y, Thanooruk R, Panijpan B, Ruenwongsa P. Enhancing nursing students' skills in vital signs assessment by using multimedia computer-assisted learning with integrated content of anatomy and physiology. Nurse Educ Today. 2009 Jan;29(1):65–72. doi: 10.1016/j.nedt.2008.06.010. [DOI] [PubMed] [Google Scholar]
  • 53.Kumrow DE. Evidence-based strategies of graduate students to achieve success in a hybrid web-based course. J Nurs Educ. 2007 Mar;46(3):140–5. doi: 10.3928/01484834-20070301-10. [DOI] [PubMed] [Google Scholar]
  • 54.Howerton WB, Enrique PRT, Ludlow JB, Tyndall DA. Interactive computer-assisted instruction vs. lecture format in dental education. J Dent Hyg. 2004;78(4):10. [PubMed] [Google Scholar]
  • 55.Gadbury-Amyot CC, Singh AH, Overman PR. Teaching with technology: learning outcomes for a combined dental and dental hygiene online hybrid oral histology course. J Dent Educ. 2013 Jun;77(6):732–43. http://www.jdentaled.org/cgi/pmidlookup?view=long&pmid=23740910. [PubMed] [Google Scholar]
  • 56.Perkins GD, Fullerton JN, Davis-Gomez N, Davies RP, Baldock C, Stevens H, Bullock I, Lockey AS. The effect of pre-course e-learning prior to advanced life support training: a randomised controlled trial. Resuscitation. 2010 Jul;81(7):877–81. doi: 10.1016/j.resuscitation.2010.03.019. [DOI] [PubMed] [Google Scholar]
  • 57.Strickland S. The effectiveness of blended learning environments for the delivery of respiratory care education. J Allied Health. 2009;38(1):E11–6. [PubMed] [Google Scholar]
  • 58.Gagnon M, Gagnon J, Desmartis M, Njoya M. The impact of blended teaching on knowledge, satisfaction, and self-directed learning in nursing undergraduates: a randomized, controlled trial. Nurs Educ Perspect. 2013;34(6):377–82. doi: 10.5480/10-459. [DOI] [PubMed] [Google Scholar]
  • 59.Boynton JR, Green TG, Johnson LA, Nainar SMH, Straffon LH. The virtual child: evaluation of an internet-based pediatric behavior management simulation. J Dent Educ. 2007 Sep;71(9):1187–93. http://www.jdentaled.org/cgi/pmidlookup?view=long&pmid=17761625. [PubMed] [Google Scholar]
  • 60.Llambí L, Esteves E, Martinez E, Forster T, García S, Miranda N, Arredondo AL, Margolis A. Teaching tobacco cessation skills to Uruguayan physicians using information and communication technologies. J Contin Educ Health Prof. 2011;31(1):43–8. doi: 10.1002/chp.20100. [DOI] [PubMed] [Google Scholar]
  • 61.Sherman H, Comer L, Putnam L, Freeman H. Blended versus lecture learning: outcomes for staff development. J Nurses Staff Dev. 2012 Jul;28(4):186–90. doi: 10.1097/NND.0b013e31825dfb71. [DOI] [PubMed] [Google Scholar]
  • 62.Gerdprasert S, Pruksacheva T, Panijpan B, Ruenwongsa P. Development of a web-based learning medium on mechanism of labour for nursing students. Nurse Educ Today. 2010 Jul;30(5):464–9. doi: 10.1016/j.nedt.2009.10.007. [DOI] [PubMed] [Google Scholar]
  • 63.Farrell MJ, Rose L. Use of mobile handheld computers in clinical nursing education. J Nurs Educ. 2008 Jan;47(1):13–9. doi: 10.3928/01484834-20080101-03. [DOI] [PubMed] [Google Scholar]
  • 64.Taradi SK, Taradi M, Radic K, Pokrajac N. Blending problem-based learning with web technology positively impacts student learning outcomes in acid-base physiology. Adv Physiol Educ. 2005 Mar;29(1):35–9. doi: 10.1152/advan.00026.2004. http://advan.physiology.org/cgi/pmidlookup?view=long&pmid=15718381. [DOI] [PubMed] [Google Scholar]
  • 65.Lehmann R, Thiessen C, Frick B, Bosse HM, Nikendei C, Hoffmann GF, Tönshoff B, Huwendiek S. Improving pediatric basic life support performance through blended learning with web-based virtual patients: randomized controlled trial. J Med Internet Res. 2015 Jul 02;17(7):e162. doi: 10.2196/jmir.4141. http://www.jmir.org/2015/7/e162/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 66.Succar T, Zebington G, Billson F, Byth K, Barrie S, McCluskey P, Grigg J. The impact of the Virtual Ophthalmology Clinic on medical students' learning: a randomised controlled trial. Eye (Lond) 2013 Oct;27(10):1151–7. doi: 10.1038/eye.2013.143. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 67.Wahlgren C, Edelbring S, Fors U, Hindbeck H, Ståhle M. Evaluation of an interactive case simulation system in dermatology and venereology for medical students. BMC Med Educ. 2006;6:40. doi: 10.1186/1472-6920-6-40. http://www.biomedcentral.com/1472-6920/6/40. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 68.Ilic D, Nordin RB, Glasziou P, Tilson JK, Villanueva E. A randomised controlled trial of a blended learning education intervention for teaching evidence-based medicine. BMC Med Educ. 2015 Mar 10;15:39. doi: 10.1186/s12909-015-0321-6. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-015-0321-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 69.Shimizu I, Nakazawa H, Sato Y, Wolfhagen IHAP, Könings KD. Does blended problem-based learning make Asian medical students active learners?: a prospective comparative study. BMC Med Educ. 2019 May 15;19(1):147. doi: 10.1186/s12909-019-1575-1. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-019-1575-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 70.Milic NM, Trajkovic GZ, Bukumiric ZM, Cirkovic A, Nikolic IM, Milin JS, Milic NV, Savic MD, Corac AM, Marinkovic JM, Stanisavljevic DM. Improving education in medical statistics: implementing a blended learning model in the existing curriculum. PLoS One. 2016;11(2):e0148882. doi: 10.1371/journal.pone.0148882. http://dx.plos.org/10.1371/journal.pone.0148882. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 71.Sadeghi R, Sedaghat MM, Sha Ahmadi F. Comparison of the effect of lecture and blended teaching methods on students' learning and satisfaction. J Adv Med Educ Prof. 2014 Oct;2(4):146–50. http://europepmc.org/abstract/MED/25512938. [PMC free article] [PubMed] [Google Scholar]
  • 72.Kho MHT, Chew KS, Azhar MN, Hamzah ML, Chuah KM, Bustam A, Chan HC. Implementing blended learning in emergency airway management training: a randomized controlled trial. BMC Emerg Med. 2018 Jan 15;18(1):1. doi: 10.1186/s12873-018-0152-y. https://bmcemergmed.biomedcentral.com/articles/10.1186/s12873-018-0152-y. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 73.Jarrett-Thelwell FD, Burke JR, Poirier J, Petrocco-Napuli K. A comparison of student performance and satisfaction between a traditional and integrative approach to teaching an introductory radiology course on the extremities. J Chiropr Educ. 2019 Mar;33(1):21–29. doi: 10.7899/JCE-17-26. http://europepmc.org/abstract/MED/30444635. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 74.Marchalot A, Dureuil B, Veber B, Fellahi J, Hanouz J, Dupont H, Lorne E, Gerard J, Compère V. Effectiveness of a blended learning course and flipped classroom in first year anaesthesia training. Anaesth Crit Care Pain Med. 2018 Oct;37(5):411–415. doi: 10.1016/j.accpm.2017.10.008. [DOI] [PubMed] [Google Scholar]
  • 75.McCutcheon K, O'Halloran P, Lohan M. Online learning versus blended learning of clinical supervisee skills with pre-registration nursing students: A randomised controlled trial. Int J Nurs Stud. 2018 Jun;82:30–39. doi: 10.1016/j.ijnurstu.2018.02.005. https://linkinghub.elsevier.com/retrieve/pii/S0020-7489(18)30048-8. [DOI] [PubMed] [Google Scholar]
  • 76.Noll C, von Jan U, Raap U, Albrecht U. Mobile augmented reality as a feature for self-oriented, blended learning in medicine: randomized controlled trial. JMIR Mhealth Uhealth. 2017 Sep 14;5(9):e139. doi: 10.2196/mhealth.7943. https://mhealth.jmir.org/2017/9/e139/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 77.Zhan X, Zhang Z, Sun F, Liu Q, Peng W, Zhang H, Yan W. Effects of improving primary health care workers' knowledge about public health services in rural China: a comparative study of blended learning and pure e-learning. J Med Internet Res. 2017 May 01;19(5):e116. doi: 10.2196/jmir.6453. https://www.jmir.org/2017/5/e116/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 78.Stewart A, Inglis G, Jardine L, Koorts P, Davies MW. A randomised controlled trial of blended learning to improve the newborn examination skills of medical students. Arch Dis Child Fetal Neonatal Ed. 2013 Mar;98(2):F141–4. doi: 10.1136/archdischild-2011-301252. [DOI] [PubMed] [Google Scholar]
  • 79.Kononowicz AA, Krawczyk P, Cebula G, Dembkowska M, Drab E, Frączek B, Stachoń AJ, Andres J. Effects of introducing a voluntary virtual patient module to a basic life support with an automated external defibrillator course: a randomised trial. BMC Med Educ. 2012 Jun 18;12:41. doi: 10.1186/1472-6920-12-41. https://bmcmededuc.biomedcentral.com/articles/10.1186/1472-6920-12-41. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 80.Wu J, Tennyson RD, Hsia T. A study of student satisfaction in a blended e-learning system environment. Computers & Education. 2010 Aug;55(1):155–164. doi: 10.1016/j.compedu.2009.12.012. [DOI] [Google Scholar]
  • 81.Kintu MJ, Zhu C, Kagambe E. Blended learning effectiveness: the relationship between student characteristics, design features and outcomes. Int J Educ Technol High Educ. 2017 Feb 6;14(1):1–20. doi: 10.1186/s41239-017-0043-4. [DOI] [Google Scholar]
  • 82.Garrison D, Kanuka H. Blended learning: Uncovering its transformative potential in higher education. The Internet and Higher Education. 2004 Apr;7(2):95–105. doi: 10.1016/j.iheduc.2004.02.001. [DOI] [Google Scholar]
  • 83.Kazu I, Demirkol M. Effect of blended learning environment model on high school students' academic achievement. Turk Online J Educ Technol. 2014 Jan 01;13(1):78–87. http://www.tojet.net/articles/v13i1/1318.pdf. [Google Scholar]
  • 84.Lim DH, Kim H. Motivation and learner characteristics affecting online learning and learning application. Journal of Educational Technology Systems. 2016 Jul 21;31(4):423–439. doi: 10.2190/0LW0-KE8X-MDYH-X27F. [DOI] [Google Scholar]
  • 85.Song L, Singleton ES, Hill JR, Koh MH. Improving online learning: student perceptions of useful and challenging characteristics. The Internet and Higher Education. 2004 Jan;7(1):59–70. doi: 10.1016/j.iheduc.2003.11.003. [DOI] [Google Scholar]
  • 86.Posadzki P, Bala MM, Kyaw BM, Semwal M, Divakar U, Koperny M, Sliwka A, Car J. Offline digital education for postregistration health professions: systematic review and meta-analysis by the digital health education collaboration. J Med Internet Res. 2019 Apr 24;21(4):e12968. doi: 10.2196/12968. http://www.jmir.org/2019/4/e12968/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 87.Greenhalgh T. Computer assisted learning in undergraduate medical education. BMJ. 2001 Jan 6;322(7277):40–4. doi: 10.1136/bmj.322.7277.40. http://europepmc.org/abstract/MED/11141156. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 88.Adams AM. Pedagogical underpinnings of computer-based learning. J Adv Nurs. 2004 Apr;46(1):5–12. doi: 10.1111/j.1365-2648.2003.02960.x. [DOI] [PubMed] [Google Scholar]
  • 89.Brandt MG, Davies ET. Visual-spatial ability, learning modality and surgical knot tying. Can J Surg. 2006 Dec;49(6):412–6. http://www.canjsurg.ca/vol49-issue6/49-6-412/ [PMC free article] [PubMed] [Google Scholar]
  • 90.Dunleavy G, Nikolaou CK, Nifakos S, Atun R, Law GCY, Tudor Car L. Mobile digital education for health professions: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019 Feb 12;21(2):e12937. doi: 10.2196/12937. http://www.jmir.org/2019/2/e12937/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 91.Tomesko J, Touger-Decker R, Dreker M, Zelig R, Parrott JS. The effectiveness of computer-assisted instruction to teach physical examination to students and trainees in the health sciences professions: a systematic review and meta-analysis. J Med Educ Curric Dev. 2017;4:2382120517720428. doi: 10.1177/2382120517720428. http://europepmc.org/abstract/MED/29349338. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 92.Al-Riyami S, Moles DR, Leeson R, Cunningham SJ. Comparison of the instructional efficacy of an internet-based temporomandibular joint (TMJ) tutorial with a traditional seminar. Br Dent J. 2010 Dec 11;209(11):571–6. doi: 10.1038/sj.bdj.2010.1085. [DOI] [PubMed] [Google Scholar]
  • 93.Kononowicz AA, Woodham LA, Edelbring S, Stathakarou N, Davies D, Saxena N, Tudor Car L, Carlstedt-Duke J, Car J, Zary N. Virtual patient simulations in health professions education: systematic review and meta-analysis by the Digital Health Education Collaboration. J Med Internet Res. 2019 Jul 02;21(7):e14676. doi: 10.2196/14676. https://www.jmir.org/2019/7/e14676/ [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 94.Cook DA, Erwin PJ, Triola MM. Computerized virtual patients in health professions education: a systematic review and meta-analysis. Acad Med. 2010 Oct;85(10):1589–602. doi: 10.1097/ACM.0b013e3181edfe13. [DOI] [PubMed] [Google Scholar]
  • 95.Xiao Z, Wang C, Li L, Tang X, Li N, Li J, Chen L, Gong Q, Tang F, Feng J, Li X. Clinical efficacy and safety of aidi injection plus docetaxel-based chemotherapy in advanced nonsmall cell lung cancer: a meta-analysis of 36 randomized controlled trials. Evid Based Complement Alternat Med. 2018;2018:7918258. doi: 10.1155/2018/7918258. doi: 10.1155/2018/7918258. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 96.Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009 Apr;43(4):303–11. doi: 10.1111/j.1365-2923.2008.03286.x. [DOI] [PubMed] [Google Scholar]
  • 97.Bearman M, Cesnik B, Liddell M. Random comparison of 'virtual patient' models in the context of teaching clinical communication skills. Med Educ. 2001 Sep;35(9):824–32. doi: 10.1046/j.1365-2923.2001.00999.x. [DOI] [PubMed] [Google Scholar]
  • 98.Foster A, Chaudhary N, Kim T, Waller JL, Wong J, Borish M, Cordar A, Lok B, Buckley PF. Using virtual patients to teach empathy: a randomized controlled study to enhance medical students' empathic communication. Simul Healthc. 2016 Jun;11(3):181–9. doi: 10.1097/SIH.0000000000000142. [DOI] [PubMed] [Google Scholar]
  • 99.Tolsgaard MG, Jepsen RMHG, Rasmussen MB, Kayser L, Fors U, Laursen LC, Svendsen JH, Ringsted C. The effect of constructing versus solving virtual patient cases on transfer of learning: a randomized trial. Perspect Med Educ. 2016 Feb;5(1):33–8. doi: 10.1007/s40037-015-0242-4. http://europepmc.org/abstract/MED/26754313. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 100.Edelbring S, Wahlström R. Dynamics of study strategies and teacher regulation in virtual patient learning activities: a cross sectional survey. BMC Med Educ. 2016 Apr 23;16:122. doi: 10.1186/s12909-016-0644-y. https://bmcmededuc.biomedcentral.com/articles/10.1186/s12909-016-0644-y. [DOI] [PMC free article] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Multimedia Appendix 1

Supplemental appendix.

jmir_v22i8e16504_app1.docx (82.1KB, docx)

Articles from Journal of Medical Internet Research are provided here courtesy of JMIR Publications Inc.

RESOURCES