Table 2.
Characteristics of included studies.
| Author(s), year, and reference | Country | Title or objective | Categorya |
Level of evidenceb | Study objective | Comments and knowledge gap |
| Winkler-Schwartz et al, 2019 [14] | Canada | Artificial Intelligence in Medical Education: Best Practices Using Machine Learning to Assess Surgical Expertise in Virtual Reality Simulation | 1 | IV | The authors developed a checklist to assess surgical expertise in virtual reality simulation. | The study provided a general framework only. The authors emphasized the need to add further elements. |
| Chan and Zary, 2019 [15] | Singapore | Applications and Challenges of Implementing Artificial Intelligence in Medical Education: Integrative Review | 1 | IV | This review evaluated current applications of AIc in medical education and highlighted the main challenges. | The authors acknowledged that a low number of studies were reviewed and stated that conclusions might be inconsequential. |
| Lillehaug and Lajoie, 1998 [16] | Sweden | AI in medical education—another grand challenge for medical informatics | 1 | IV | This comprehensive review discussed the potential use of AI to enhance medical informatics education. | This article was published before the discovery of high-performance computing processors and recent advancements in data recording technology. |
| Frize and Frasson, 2000 [17] | Canada | Decision-support and intelligent tutoring systems in medical education | 1 | V | This study evaluated the use of intelligent tutoring systems in medical education. | This article discusses the potential use of decision support tools but emphasizes the need for further research to validate their usefulness. |
| Zhao et al, 2018 [18] | China | Research on Application of Artificial Intelligence in Medical Education | 1 | V | This article analyzed the application of AI in medical education. | This study evaluated the effect of AI technology on traditional medical education with a focus on personalized learning. |
| Chary et al, 2018 [19] | United States | A Review of Natural Language Processing in Medical Education | 1 | IV | This study reviewed the application of NLPd to medical education and identified concepts from NLP used in those applications. | The authors investigated the integration of NLP to medical education resources using published manuscripts and stated the potentially biased representation of the scope. |
| Caudell et al, 2003 [20] | United States | Virtual patient simulator for distributed collaborative medical education | 1 | IV | The study investigated the feasibility of using a real-time AI simulation engine in medical school curricula. | The study described an ongoing project and did not provide any data about the difference between problem-based learning using virtual patient simulators and standard paper case tutorials. |
| Guimarães et al, 2017 [21] | Portugal | Rethinking Anatomy: How to Overcome Challenges of Medical Education's Evolution | 1 | IV | This literature review evaluated the integration of complementary technology-based methodologies to medical instruction. | The authors discussed the potential of AI in learning analytics-oriented systems to predict behavior but did not make any recommendations about new research studies. |
| Bowyer et al, 2008 [22] | United States | Immersive Virtual Environments for Medical Training | 1 | IV | This study highlighted the role of advanced virtual environments and surgical simulators as a training platform for medical training. | The paper described various virtual reality environments where students can interact with AI-based simulators. |
| Sitterding et al, 2019 [23] | United States | Using Artificial Intelligence and Gaming to Improve New Nurse Transition | 1 | IV | This research discussed the preliminary pilot study data from a virtual reality simulation education intervention that compared virtual reality, augmented reality, serious gaming, and gamification. | The sample size and pending postintervention findings were stated as the limitations of the preliminary findings. |
| Boulet and Durning, 2019 [24] | United States | What we measure … and what we should measure in medical education | 1 | V | This paper focused on the validity of assessment scores and discusses the application of AI to automate the assessment process. | The authors recommended developing new competency assessment practices and highlighted the importance of the application of AI. They did not provide supporting evidence about AI's potential to eliminate the need for human ratings. |
| Conde et al, 2009 [25] | United States | Telehealth Innovations in Health Education and Training | 1 | V | This discussion paper indicated the potential of telehealth technologies for health education and training. | The authors recommended the development of AI applications for patient simulation and the integration of telehealth applications in health education, but the paper did not provide any evidence. |
| Kabassi et al, 2008 [26] | Greece | Specifying the personalization reasoning mechanism for an intelligent medical e-learning system on Atheromatosis: An empirical study | 1 | IV | The objective of this empirical study was to incorporate intelligent techniques in web-based medical education. | The authors described the specification of an intelligent medical learning system for atheromatosis that can interact with students. The design was based on the results of empirical data and the authors did not compare the e-learning system with traditional methods. |
| Klar and Bayer, 1990 [27] | Germany | Computer-assisted teaching and learning in medicine | 1 | IV | This article provided a comprehensive discussion of computer-assisted instruction systems and discussed expert systems' contribution to software for medical learning. | This paper was published before AI impacted multiple fields but the authors successfully envisioned how AI would transform decision making, simulation, and medical education. |
| Yang et al, 2019 [28] | Taiwan | An expert-led and artificial intelligence (AI) system-assisted tutoring course increase confidence of Chinese medical interns on suturing and ligature skills: prospective pilot study | 1 | IV | This paper examined the impact of an AI system tutoring course on clinical training. | This study compared regular, expert-led, and expert-led+AI groups and found an increased improvement in the expert-led+AI tutoring group. Authors recommended AI-assisted tutoring for novice medical interns. |
| Alonso-Silverio et al, 2018 [29] | Mexico | Development of a Laparoscopic Box Trainer Based on Open Source Hardware and Artificial Intelligence for Objective Assessment of Surgical Psychomotor Skills | 1 | IV | This study evaluated the effect of a laparoscopic trainer system that uses an AI algorithm. | The authors described the development of a low-cost intelligent simulator to improve laparoscopic skills and proposed the training as a validated training tool for surgical education programs. |
| Kolachalama and Garg, 2018 [30] | United States | Machine learning and medical education | 2 | V | This perspective article discussed the lack of student access to machine learning content and makes some suggestions to instructors. | This perspective paper only provided an outline and did not provide any evidence. |
| Park et al, 2019 [31] | Korea | What should medical students know about artificial intelligence in medicine? | 2 | IV | This short review emphasized the lack of direct access to machine learning education for clinicians and recommended the inclusion of focused content. | The review emphasized the need to identify correct information about AI. |
| Wartman and Combs, 2018 [32] | United States | Medical Education Must Move From the Information Age to the Age of Artificial Intelligence | 2 | V | This article discussed the need to develop new curricular components to teach the use of AI tools. | This commentary article summarized the authors' perspective and did not provide supporting evidence. |
| Wartman and Combs, 2019 [33] | United States | Reimagining Medical Education in the Age of AI | 2 | V | This paper indicated the need for a more sophisticated mathematical understanding of analytics. | The authors proposed a new curriculum that will include the skill sets required to use AI effectively. |
| Beregi, 2018 [34] | France | Artificial intelligence and medical imaging 2018: French Radiology Community white paper | 2 | IV | This review discussed current applications of AI in medical imaging and recommended AI education for radiology residents. | This position paper summarized AI principles, provided an update on research in the area of AI, and described radiologists' role in providing education about AI. |
| Tang et al, 2018 [35] | Canada | Canadian Association of Radiologists White Paper on Artificial Intelligence in Radiology | 2 | IV | This paper assessed the educational needs of radiologists and medical students, and provided recommendations. | The AI working group recommended the integration of health informatics and computer science courses to analyze the opportunities and challenges associated with new AI tools. |
| Masters, 2019 [36] | Oman | Artificial intelligence in medical education | 2 | V | This review highlighted the demand to learn how to work with AI systems and emphasized the need for AI training. | The authors identified new AI applications in medicine and recommended changes to medical curricula. |
| Chin-Yee and Upshur, 2017 [37] | Canada | Clinical judgement in the era of big data and predictive analytics | 2 | IV | This article explored different approaches to clinical judgment. | Authors indicated that data-driven and AI-based applications move medicine away from virtue-based approaches to clinical reasoning and recommended an integrative approach. |
| Santos et al, 2019 [38] | Germany | Medical students' attitude toward artificial intelligence: a multicenter survey | 2 | IV | This study investigated undergraduate medical students' attitudes toward AI. | The authors designed a survey to explore students' familiarity with AI concepts in radiology and concluded that they did not have an understanding of the basic technical principles underlying AI. |
| Paranjape et al, 2019 [39] |
Netherlands | Introducing Artificial Intelligence Training in Medical Education | 2 | IV | This paper summarized the state of medical education and recommended a framework to include AI education. | This viewpoint paper suggested different AI-related content for different stages of medical education. |
aCategory 1: the use of AI applications in medical and health informatics education; Category 2: AI education.
bEvidence levels were as described by the Oxford Centre for Evidence-Based Medicine Levels of Evidence [40].
cAI: artificial intelligence.
dNLP: natural language processing.