Table 3.
Characteristics of included articles ordered by comparator and study design
| Authors and year | Aim(s) of the study | Research Design | Participants—year group, total N and intervention and control group N | Domain of clinical reasoning measured | Outcome measure | Main results | Quality (score out of 14) |
|---|---|---|---|---|---|---|---|
| Comparator: teaching as usual | |||||||
| Aghili et al. 2012 | To evaluate whether virtual patient simulations improve clinical reasoning skills of medical students | RCT | 6th years. N = 52 (29 IG, 23 CG) | Data gathering, ideas about patient management | Diagnostic test (using patient cases) | ⇧ Intervention produced significantly greater improvement in data gathering and ideas about patient management compared to teaching as usual (d = 1.55) | Moderate (6) |
| Botezatu et al. 2010 | To explore possible superior retention results with Virtual Patients versus regular learning activities, by measuring the differences between early and delayed assessment results | RCT | 4th & 6th years. N = 49 (25 IG, 24 CG) | Data gathering, ideas about diagnoses, ideas about patient management | Virtual patient cases | ⇧ Intervention produced significantly greater improvement in data gathering, ideas about diagnoses and patient management compared to teaching as usual (average effect size across 5 dimensions, d = 1.57) | Moderate (6) |
| Kahl et al. 2010 | To explore whether the addition of systematic training in iterative hypothesis testing may add to the quality of the psychiatry course taught to fifth year medical students | RCT | 5th years. N = 72 (36 IG, 36 CG) | Ideas about diagnoses | Standardised patient (actor) | ⇧ Intervention produced significantly greater improvements in ideas about diagnoses compared to teaching as usual (d = 1.17) | Moderate (7) |
| Kalet et al. 2007 | To assess the impact of individual WISE-MD modules on clinical reasoning skills | RCT | Clinical years. N = 96 (52 IG, 44 CG) | Data gathering, ideas about patient management | Script concordance test | ⇧ Intervention produced significantly greater improvement in data gathering and ideas about patient management compared to teaching as usual (d = 0.25) | Moderate (9) |
| Lehmann et al. 2015 | Investigated the effect of Virtual Patients combined with standard simulation-based training on the acquisition of clinical decision-making skills and procedural knowledge, objective skill performance, and self-assessment | RCT | 3rd & 4th years. N = 57 (30 IG, 27 CG) | Ideas about diagnoses, ideas about patient management, application of knowledge | Key feature problems | ⇧ Intervention produced significantly greater improvement in ideas about diagnoses and patient management, and application of knowledge compared to teaching as usual (d = 1.91) | High (13) |
| Qin et al. 2022 | To develop a competency-based model of practice-based learning for undergraduate radiology education | RCT | 3rd years. N = 114 (57 IG, 57 CG) | Application of knowledge | Multiple-choice question examination | ⇧ Intervention produced significantly greater improvement in the application of knowledge compared to teaching as usual (d = 0.63) | Moderate (5) |
| Plackett et al. 2020 | To assess the feasibility, acceptability and potential effects of eCREST — the electronic Clinical Reasoning Educational Simulation Tool | Feasibility RCT | 5th & 6th years. N = 264 (137 IG, 127 CG) | Data gathering, flexibility in thinking about diagnoses (reported separately)a | Virtual patient case & Diagnostic Thinking Inventory (DTI) | ⇧ Ability to gather essential information (data gathering; d = 0.19) significantly improved after intervention compared to teaching as usual | High (11) |
| ⬄ There was no significant difference between groups in relevance of history taking (data gathering; d = -0.13) and flexibility in diagnoses (d = 0.20) | |||||||
| Kim et al. 2018 | To explore how students use and benefit from virtual patient cases | Non-randomised trial | 3rd years. N = 255 (129 IG, 126 CG) | Ideas about diagnoses | Clinical rating at end of clerkship by faculty | ⬄ Ideas about diagnoses did not significantly improve compared to teaching as usual (voluntary access to cases) (d = 0.09) | Moderate (8) |
| Raupach et al. 2021 | To investigate the effectiveness of a digital simulation of an emergency ward regarding appropriate clinical decision-making | Non-randomised trial | 4th years. N = 100 (58 IG, 42 CG) | Data gathering, ideas about diagnoses, ideas about patient management (reported separately) | Virtual patient cases | ⇧ Intervention produced significantly greater improvement in diagnostic accuracy (ideas about diagnoses for 2/3 cases; d = 0.81) and patient management (d = 0.81), compared to teaching as usual | Moderate (5) |
| ⬄ Intervention did not significantly improve data gathering, compared to teaching as usual (d = 0.03) | |||||||
| Comparator: tutorial covering the same case | |||||||
| Devitt & Palmer 1998 | To evaluate the intervention by assessing whether it expanded students’ knowledge base, improving data-handling abilities and clinical problem-solving skills | RCT | 5th years. N = 71 (46 IG, 25 CG) | Problem-solving skills | Multi-step clinical problem (patient case) | ⬄ Intervention produced non-significantly greater improvement in problem-solving skills compared to tutorial (d = 0.50) | Moderate (6) |
| Raupach et al. 2009 | To explore whether students completing a web based collaborative teaching module show higher performance in a test aimed at clinical reasoning skills than students discussing the same clinical case in a traditional teaching session | RCT | 4th years. N = 143 (72 IG, 71 CG) | Data gathering, ideas about diagnoses, ideas about patient management | Key feature problems | ⬄ Intervention did not significantly improve data gathering, ideas about diagnoses and patient management compared to tutorial (d = 0.03) | High (10) |
| Sobocan et al. 2017 | To determine the educational effects of substituting p-PBL sessions with VP on undergraduate medical students in their internal medicine course | RCT | 3rd years. N = 34 (17 IG, 17 CG) | Application of knowledge and flexibility in thinking | DTI | ⬄ Intervention did not significantly improve application of knowledge and flexibility in thinking compared to tutorial (d = 0.25) | Moderate (7) |
| Middeke et al. 2018 | To compare a Serious Game, the virtual A&E department ‘EMERGE’ to small-group problem-based learning (PBL) regarding student learning outcome on clinical reasoning in the short term | Non-randomised trial | 5th years, N = 112 (78 IG, 34 CG) | Data gathering, ideas about diagnoses, ideas about patient management (reported separately) | Key feature problems & virtual patient cases | ⇧ Intervention produced significantly better clinical reasoning skills compared to tutorial (d = 0.47) when measured on key features test and for some domains measured by the virtual patient cases – final diagnosis (ideas about diagnoses), | Moderate (6) |
| therapeutic interventions (ideas about patient management), physical examination, instrumental examination (data gathering) | |||||||
| ⬄ There was no significant difference between groups in history taking (data gathering), laboratory orders and patient transfer (ideas about patient management) | |||||||
| Comparator: N/A | |||||||
| Chon et al. 2019 | To test the effect of a serious game simulating an emergency department (“EMERGE”) on students’ declarative and procedural knowledge | Single group pre & post comparison | Clinical years. N = 140 | Data gathering, ideas about diagnoses, ideas about patient management, (reported separately) | Patient case | ⇧ Diagnostic questions (data gathering; d = 0.77), choosing the correct order of diagnostic procedures (ideas about diagnoses; d = 0.65) and treatment suggestions (ideas about patient management; d = 0.82) improved after using intervention | Moderate (5) |
| ⬄ There was no significant difference between groups in diagnostic accuracy (ideas about diagnoses; d = 0.08) | |||||||
| Dekhtyar et al. 2021 | To test the hypothesis that the Symptom to Diagnosis diagnostic reasoning approach videos paired with practice virtual patient encounter simulations could improve the diagnostic accuracy in medical students as evidenced by their ability to diagnose new simulated cases with diagnoses not previously encountered | Single group pre & post comparison | 2nd & 3rd years. N = 285 | Data gathering, ideas about diagnoses (reported separately) | Virtual patient cases | ⇧ History taking efficiency (data gathering; d = 0.47), history taking completeness (data gathering d = 0.32); efficiency of differential diagnosis (ideas about diagnoses; d = 1.16) and completeness of differential diagnosis (ideas about diagnosis; d = 0.93) improved after using intervention | Low (3) |
| Isaza-Restrepo et al. 2018 | To present evidence regarding the effectiveness of a low-fidelity simulator: Virtual Patient | Single group pre & post comparison | 1st-5th years. N = 20 | Data gathering, ideas about diagnoses, ideas about patient management | Standardised patient (actor) | ⇧ Data gathering, ideas about diagnoses and patient management, and presentation of a case significantly improved after using intervention (average effect size across 5 dimensions from 3 evaluators, d = 1.41) | Moderate (6) |
| Kleinert et al. 2015 | To examine whether the use of ALICE has positive impact on clinical reasoning and is a suitable tool for supporting the clinical teacher | Single group pre & post comparison | 3rd years. N = 62 | Ideas about diagnoses, ideas about patient management | Patient cases | ⇧ Ideas about diagnoses and patient management significantly improved after using intervention (d = 0.92) | Low (3) |
| Watari et al. 2020 | To clarify the effectiveness of VPSs for improving clinical reasoning skills among medical students, and to compare improvements in knowledge or clinical reasoning skills relevant to specific clinical scenarios | Single group pre & post comparison | 4th years. N = 169 | Data gathering, ideas about diagnoses, ideas about patient management | Multiple-choice question quiz (using patient cases) | ⇧ Data gathering, ideas about diagnoses and patient management significantly improved after using intervention (d = 1.39) | Low (3) |
| Wu et al. 2014 | To examine the effectiveness of a computer-based cognitive representation approach in supporting the learning of clinical reasoning | Single group pre & post comparison | 3rd-5th years. N = 50 | Problem-solving | Concept maps | ⇧ Problem-solving significantly improved after using intervention (d = 1.17) | Moderate (5) |
a5 articles reported the impact of the virtual patient tools on each domain of clinical reasoning separately while all others reported an aggregate impact score across several domains of reasoning