Impact of artificial intelligence on academic performance in medical education: A systematic review

Abstract

Artificial intelligence (AI) plays a significant role in improving the quality of medical education. This study aimed to investigate the application of AI in medical education. A systematic review was conducted of all educational intervention studies in medical courses from January 1986 to 2023. Of the 16755 studies initially identified by our search, 7387 remained after removing duplicates. After that, 6205 studies were excluded for the title and abstract screening. Then, 15 full-text articles were included in our final review. The following keywords were used: artificial intelligence, machine intelligence, medical education, medical teaching, precision medical teaching, and precision medical education. A total of 16745 articles were identified from ISI, PubMed, Scopus, and Educational Resources and Information Center (ERIC) databases. Fourteen studies met the eligibility criteria. The quality of the included articles was appraised by the best evidence medical education (BEME) review of the education portfolio. PICO consists of P = medical student, I: use of AI, C = do not use AI, and I = enhance health professions students’ knowledge, attitudes, and skills. The included studies were synthesized and categorized according to the Kirkpatrick model. Educational intervention outcomes were categorized into three parts^[1]: improvement in learners’ knowledge (N = 6 studies)^[2]; enhancement of students’ attitudes (N = 3 studies); and^[3] acquisition of learners’ skills (N = 10 studies). The reviewed studies examined the impact of AI and virtual reality on enhancing health profession students’ knowledge, attitudes, and skills. However, more research is still needed on integrating AI into diverse curriculum models and sustaining AI’s role in real-world education systems.

Introduction

Artificial intelligence (AI) has developed through the years and is now being noticed with the appearance of deep learning and artificial neural networks. AI paradigms indicate several problem scopes (understanding, decision-making, knowledge, and connection).[1]

AI, including machine learning and deep learning, has many applications in many industries, such as telecommunications, construction, transportation, healthcare, manufacturing, advertising, and medical education.[2,3] Studies show that interest in research and development of AI in the health area has increased in the last two decades.[4] AI plays a considerable role in medical education to improve the quality of education.[5] Medical education includes a lifelong learning continuum extending from undergraduate to postgraduate and continuing medical education.[4] It is also suitable for different health care professionals, expanding from doctors to nurses and other allied health care providers.

The use of virtual reality (VR), augmented reality (AR), AI, machine learning, and new technologies related to big data management help and ultimately promote medical education.[1] Some of the uses of AI include helping medical professionals diagnose diseases early and changing the traditional medical curriculum.[2] In addition, it allows students to take a personal approach to learning issues based on their experiences and preferences. This type of technology can adjust to the level of knowledge, learning level, and goals of students to get the most out of their education. Also, it has the potential to analyze students’ prior learning records to identify deficiencies and provide suitable courses to build on the personal learning experience.[5] Also, the use of AI can decrease the time required for routine administrative work and allow higher education teachers to focus more on teaching and learning.[6] Also, few researchers have addressed the advantages and disadvantages faced when using AI in medical education.[7]

The appearance of coronavirus disease 2019 (COVID-19) extended the use of technology in medical universities.[8] Medical institutions were forced to use e-learning strategies for education. Thus, educational institutions are challenging this modern paradigm shift and its outcomes for the post-COVID-19 era. AI can create new chances for facilitating e-learning in the future.[9] With the accelerating speed of health digitization, AI usage has enhanced due to the increasing demand among medical teachers, and medical universities. Although there is little evidence about the effect of AI on the medical education. Considering the application of AI in medical sciences, which has improved diagnosis, prediction, treatment, and medical research. More accurate diagnosis of diseases, prediction of health problems, personalized treatments, reduction of medical errors, and advancement of medical research can be used in medical education, so this review suggests new horizons for medical teachers and learners of medical sciences in the use of AI and the advancement of learning. This study aimed to investigate the application of AI in medical education. To the best knowledge of researchers, this is the first systematic review in this area of research. The present review study focused on the impact of AI on medical education.

Materials and Methods

This systematic review was conducted on all the educational interventional studies about the role of AI in medical courses from January 1986 to 2023.

Search strategy

The following databases were selected: the Medline database (PubMed), Web of Science (Institute for Scientific Information (ISI)), Scopus, and Educational Resources and Information Center (ERIC). For comprehensiveness of the search, the following keywords were used in the abstract, title, and keyword sections: (“medical education” OR “medical teaching” OR “higher medical education” OR “higher medical teaching” OR “precision medical teaching” OR “precision medical education”) AND (“artificial Intelligence” OR “AI” OR “Machine Intelligence” OR “Machin learning”). A search strategy was developed for each database (Appendix 1).

Inclusion and exclusion criteria

The inclusion criteria for the articles were as follows: being an educational interventional study and assessing the academic performance (including learning and master of performance) of the medical students without any language or time limitation from January 1986 to December 2023. Besides, the exclusion criteria for the search were secondary research, descriptive studies, or observational study design and not being a medical student.

Selecting the studies

Two independent authors (NKh and MH) screened the titles and abstracts according to the inclusion criteria. The full-text articles were reviewed. In the event of any discrepancy regarding eligibility, a third independent author (MK) was involved. The Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) guideline has been followed [Figure 1].[10]

Figure 1.

Study flowchart about the literature research

Data extraction

Data extracted from each study included the type of study, study country, participants, type of intervention, pretest assessment, and type of outcome.

Quality assessment

The quality of the included articles was appraised by the best evidence medical education (BEME) review of the education portfolio.[11] These tools consist of 11 items about the appropriateness of the study design, results, analysis, and conclusions, which were used to examine the quality of the studies. The studies that fulfilled a minimum of eight quality criteria or those meeting six or seven indicators were classified as highquality and mediumquality, respectively. Also, those articles that fulfilled five or fewer indicators were considered lowquality. [Table 1]

Table 1.

Quality assessment of articles

First author, year		Research question		Study subjects		Reliability and validity of the method		Completeness of data		Control of confounding		Analysis of the results		Conclusions		Reproducibility		Prospective		Ethical issues		Triangulation		Quality
Ji Sung Shim, 2018		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
João MesquitaI, 2019		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Megan S. Orlando, 2016		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Angel del Blanco, 2017		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Nickle, 2015		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Eimear Ryan, 2019		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Mads Forslund Jacobsen, 2019		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Sang Gil Han1, 2021		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Grantcharov, 2004		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Hsiang-Ying Chan RN, 2020		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Yang Hou1, 2017		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Ramazan Kurul, 2020		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Neal E. Seymour, 2002		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Eva Neumann, 2018		Yes		Yes		Yes		No		Yes		Yes		Yes		Yes		Yes		Yes		Yes		High-quality
Shah, 2023		Yes		Yes		Yes		No		No		Yes		Yes		Yes		Yes		Yes		Yes		High-quality

The authors reported the studies using the Kirkpatrick hierarchy introduced by BEME for educational contexts.[12]

Result

Study characteristics

As shown in Figure 1, of the 16755 studies initially identified by our search, 7387 remained after removing duplicates. After that, 6205 studies were excluded for the title and abstract screening. Then, 15 full-text articles were included in our final review. The included studies were published over 20 years, between 2002 and 2023.[13,14,15,16,17,18,19,20,21,22,23,24,25,26] Moreover, 64% (none articles) of the studies were conducted between 2018 and 2023.[13,14,15,16,17,18,19,20,21] Out of the 15 studies, five were from Asia, five were from Europe, four were from the US, and one study was from New Zealand. All article designs were randomized control trials. Articles were most likely to have been published in medical education journals. From the 14 articles identified, 12 (85.7%) studies were conducted on medical students, and two (14.3%) studies were done on nurse students [Table 2].

Table 2.

Summary of the reviewed studies

Authors		Country		Aims of the study		Participants		Design		Type of intervention		Main findings		Type of Outcome
Hsiang-Ying Chan et al. (2021)[13]		Taiwan		VR-based documents (VR docs) on knowledge and attitude toward chemotherapy administration in nursing students.		77 nursing students		RCT		VR teaching		Improvement knowledge and attitude		Knowledge score (posttest) and attitude score
Sang Gil Han et al. (2021)[14]		Korea		VRNET versus standardized patient in teaching neurological examinations		95 medical student		RCT		VR-based neurological examination teaching tool (VRNET)		The SP with VRNET group had higher NPE scores than the SP group		Teaching neurological examinations for the medical students
Ramazan Kurul, et al. (2020)[15]		Turkey		Investigate the effect of immersive 3D interactive VR on anatomy training		72 undergraduate physical therapy		RCT				VR group showed a significant increase compared to the control group in determining head and neck region		Anatomy training test score
Mads forslund Jacobsen et al. (2020)[16]		Illinois.		To compare manual and robot-assisted vitreoretinal surgery using a VR surgical simulator		20 Vitreoretinal surgeons		RCT				Improve precision and limit tissue damage compared with that of manual surgery performance of novice surgeons is enhanced with robot-assisted vitreoretinal surgery		Precision and limit tissue damage
João Mesquita et al. (2019)[17]		New Zealand		Impact of online training on the identification of AF driver sites where ablation terminated persistent AF, through a standardized training program		14 fellows-in-training in clinical cardiac electrophysiology		RCT		Video		Digital online training improved interpretation of panoramic AF		Rate of detection
Eimear Ryan, et al. (2019)[18]		Ireland		Impact of virtual learning environment on students’ satisfaction, engagement, Recall, and Retention		40 undergraduates students		RCT		Virtual environment for radiotherapy software		Greater satisfaction/engagement than didactic information using the VLE had higher mean scores for retention than the didactic group		Students’ satisfaction, engagement, recall, and retention
Eva Neumann et al. (2019)[20]		Germany		To determine the benefit of VR cystoscopy (UC) and TURBT training in students		51 medical students		RCT		VR training program for cystoscopy and TURBT		Improve the surgical training) significant differences in procedure time, resectoscope movement, and accidental bladder injury, reduced blood loss		Assessing performance indicators
Ji Sung Shim, et al. (2018)[20]		South Korea.		Comparison of effective teaching methods to achieve skill acquisition using a robotic virtual reality simulator in radical prostatectomy		44 Medical students		RCT		Proctoring or an educational video.		Achieve skill acquisition using a robotic virtual reality simulator		Decrease the mean time for completing the task after overcoming learning curve
Yang Hou et al. (2018)[21]		China		Investigate the effectiveness of VSTS on cervical pedicle screw instrumentation		10 novice residents		RCT		Virtual surgery simulation		Improving performance of novice residents		The average screw penetration distance
Angel del Blanco et al. (2017)[22]		Madrid		Game-like simulation could improve perceptions and performance of novices in surgical block		132 nursing and medical students		RCT		Video stimulation		less fear of making mistakes showed higher perceived knowledge on how they had to behave and what they could and could not do while in the operating theatre and showed a more collaborative attitude with patients and staff		Perception, emotion, and attitude of surgical block
Megan S. Orlando et al. (2017)[24]		New York		To investigate the retention of laparoscopic and robotic skills after simulation training.		40 medical students		RCT		Robotic practice		Robotic skills acquired through simulation appear to be better maintained than laparoscopic simulation skills		Laparoscopic task score
Felix Nickel et al. (2015)[25]		Denmark		VR training with low cost-BL in laparoscopic surgery		84 medical students		RCT		Robotic surgery		The efficiency of the training was judged higher by the VR group than by the BL group		Knowledge test score and operative performance of in the OSAT score
Grantcharov et al. (2004) (25)[26]		Denmark		A randomized clinical trial of VR simulation for laparoscopic skills training		16 Surgical trainees		RCT		VR simulation		Improvement in performance in the OR		Assessment performance (time to complete the procedure, error score and economy of movement score)
Neal E. Seymour et al. (2002)[27]		Virginia.		VR training improves OR performance: results of a randomized, double-blinded study		16 surgical residents		RCT		VR surgical simulation		Made fewer errors, were less likely to injure the gallbladder and burn non target tissue, and were more likely to make steady progress throughout the procedure by review video		Assessing laparoscopic cholecystectomy performance
Shah (2023)		USA		Use of an AI-powered, Bayesian inference-basedCDS software to trainees during the interpretation of clinical and simulation brain MRI examinations		Neuroradiology fellows and diagnostic radiology residents		Experimental		AI-powered, Bayesian inference-basedCDS		Improve the educational value of interpreting imaging studies		knowledge

MRI=magnetic resonance imaging, CDS=clinical support software, TURBT=transurethral bladder tumor resection, NPE=neurologic physical exam, VRNET=VR-based neurological examination teaching tool, VLE=virtual learning environment

AI in medical education

Table 2 demonstrates the quality assessment of the included studies. All the selected studies had high quality. The first publications about the use of VR training to improve operating room (OR) performance for surgical residents were published in 2002, and the results of this study show that the patients were less likely to injure the gallbladder and burn nontarget tissue and were more likely to make steady progress throughout the procedure.[26]

According to the Kirkpatrick outcome levels, the number of articles that assessed the impact of the educational intervention at each Kirkpatrick level is illustrated in Table 3.

Table 3.

List of included studies based on Kirkpatrick levels

Levels		Kirkpatrick outcome level		Studies (n)
1		Reaction – learners’ reactions		1
2A		Learning – change in view or attitude		2
2B		Learning – modification of knowledge or skill		15
3		Behavior – change in behavior (transfer of learning to the workplace)		0
4A		Results – change in the system/organizational practice		0
4B		Results – in patient care outcome		0

Knowledge/skills (levels 2B) were changed in all articles, and changes in view or attitude were in two articles. In addition, one article reported learners’ reactions. There was no article mentioning levels 3 and 4. Moreover, five articles reported two or more outcome levels.

Synthesis of findings

The outcomes of educational intervention are categorized into three parts, namely, (I) improvement in learners’ knowledge (N = 6); (II) enhancement of the student’s attitude (N = 3); and (III) acquisition of learners’ skills (N = 10).

Improvement in learners’ knowledge

Six articles showed the positive effects of educational intervention on students’ knowledge. The summary of interventions can be introduced as follows: the effect of (I) VR-based documents (VR docs) on knowledge[13]; (II) the effect of immersive three-dimensional (3D) interactive VR on anatomy knowledge[14]; (III) the impact of digital online and cloud-based training for detecting AF[17]; (IV) the impact of virtual learning environment on students knowledge in radiotherapy[18]; (V) impact of VR training with low-cost blended learning (BL) in laparoscopic surgery.[22];) VI) impact of AI-powered, Bayesian inference-based clinical decision support on the interpretation of neuroradiology fellows and diagnostic radiology residents.[23]

Enhancement of the student’s attitude

Three articles showed the impact of educational interventions on perception, satisfaction, and attitude. Chan et al.[13] illustrated an improved attitude toward chemotherapy administration in nursing students. Ryan, et al.[18] showed a significant increase in greater satisfaction and engagement in using virtual learning environments (radiotherapy). Blanco et al.[22] also indicated that the intervention group had a more collaborative attitude with patients and staff in the surgical block.

Acquisition of learners’ skills

Seventy-one percent of articles (N = 10) reported the significant effect of educational interventions on skill as follows: impact of VR-based neurological examination on neurologic physical exam scores.[14] Robot-assisted vitreoretinal surgery improves precision and limits tissue damage.[16] VR cystoscopy improves procedure time, resect scope movement, accidental bladder injury, and reduced blood loss.[20] Achieve skill acquisition using a robotic VR simulator in radical prostatectomy.[19] The virtual surgical training system (VSTS) aims to improve the performance of novice residents.[21] Game-like simulation improves the performance of novices in surgical blocks.[22] Robotic skills improve laparoscopic simulation skills.[24] VR enhances operative performance in the objective structured assessment of technical skill (OSAT) score in laparoscopic surgery.[25] VR simulation improves laparoscopic skills in the OR (time to complete the procedure, error score, and economy of movement score).[26] VR training improves OR performance in laparoscopic cholecystectomy.[27]

Discussion

This systematic review was conducted to search for and synthesize educational interventions in experimental studies on medical sciences education from 1986 to 2023. To the best of our knowledge, this research was the first work on this concept.

Considering the role of AI as a cutting-edge technology in medical education, only fifteen articles in this search were found. The rigor of these studies has been high quality because of the Randomized Controlled Trial (RCT) design. In the reported articles, AI has a positive effect on the three levels of knowledge, attitude, and skill. Most studies have shown the effect of AI on improving skills; perhaps AI can provide a rich learning environment and can repeat skills frequently; moreover, it can dominate time and place barriers.

None of the included studies reported any changes in the improvement of patients’ health outcomes or system practice as a direct impact of the educational intervention (Kirkpatrick’s 4 levels) and a change in behavior (Kirkpatrick’s 3 levels). Because AI is an emerging technology and long-term follow-up is needed for evaluation at levels 3 and 4 of Kirkpatrick’s pyramid. In addition, it is not practical to take into account identifying the complex factors that are involved in patient care.

The results of this systematic review illustrated that educational intervention with six different types such as VR-based documents (VR docs), consisting of a variety of handouts and teaching methods to educate nursing students about high-risk nursing techniques can produce better learning results,[13,15,17,18,22,24] anatomy training with a 3D immersive VR system was found to be beneficial, combining online clinical data,[15] smartphone apps and other digital resources provides a powerful approach for training in novel techniques in electrophysiology.[17] The virtual environment of radiotherapy has shown improvement in students’ satisfaction, engagement, and recall.[19] Self-developed game-like simulation to enhance the experience of visiting the surgical block[24] impact of AI-powered, Bayesian inference-based clinical decision support on the interpretation of neuroradiology fellows and diagnostic radiology residents.[23] VR training in laparoscopic cholecystectomy can be used to improve academic performance in light of both domains of Bloom’s taxonomy such as knowledge and attitude for students of the medical sciences in medical programs.[26]

Studies which are consistent with the finding of the present study about the multi-dimensional educational experience is a study by Hanson et al.[28] that showed a 3D experience helped improve understanding, satisfy learning, and cause minimal anxiety for medical sciences students.[29] Novel technologies cause changes in learning preferences.[30] Medical student knowledge and attitudes about using novel technologies were assessed in two medical universities to elucidate. This survey illustrated that students had significantly favorable views about the use of new related technology in medical education.[31] Considering that today’s students are from the generation of digital natives, the teaching methods also need to be changed, using multiple teaching methods will improve their learning and a better attitude.

The variety of surgical skills taught in the articles include vitreoretinal,[16] cystoscopy,[20] prostatectomy,[19] cervical pedicle,[21] and laparoscopic[23,24,25] that can improve the skills of medical students, nursing students, and residents. Most surgical procedures are reported as laparoscopy.[23,24,25]

Advantages of a robotic VR simulator or AI for training surgical skills. It is available for users and would not require educators to physically attend the training. Another advantage is that it could be repeatable at any time.[22] Also, these technologies could help the surgeon attain a proficiency level for a specific skill. So, the surgeon practices technical skills in a safe environment, protecting patient safety and increasing progression along the early stage of the learning curve.[31]

Currently, suitable training methods consist of “observing experts” and “practicing on animals or cadaver models before operating on patients.”[32] These low numbers of training tools may encounter most patients the potential risks due to the initial phase of surgeons’ learning curves.[20]

One of the potential risks is tissue damage, which simulation-based training could reduce. Jacobsen et al.[16] indicated that robot-assisted vitreoretinal surgery cusses improved precision and limited tissue damage compared with manual ones in novice surgeons.

Generally, educational technologies such as AI and VR are found to be very effective in increasing the learning experience for medical sciences students. Therefore, the application of AI in clinical practice is considered a fortunate area of expansion, especially for diagnostic medicine, teleconsultation, and so on.[33]

Despite these advantages, teaching with new educational technologies also has challenges. It must be highlighted that many more skills are integrated into the technical training of a surgeon (including cognitive skills, decision-making, and surrogate planning), and the simulators are only one part that can help the improvement of performance and assessment of competency.[26]

Medical curricula must provide a balanced perspective, identifying AI weaknesses and strengths. Also, curriculum development should be revised to include “AI” topics, using digital technologies to fulfill training.[34,35]

Many current studies show that the use of VR simulators is related to performance in the real environment, but it cannot be denied that robot-assisted surgery in the simulated environment is completely similar to the real environment, and this is a limitation.[16]

One limitation of the present research was the lack of meta-analysis, although a meta-analysis was not possible because of the variation in study designs. However, the current study had the strength of not having any language limitations. In addition, the sample size in the included studies was small. So, a larger trial is required to confirm the findings of the studies.

Conclusion

AI and VR as cutting-edge educational technologies are utilized in higher education to improve the quality of education and develop virtual learning resources. Therefore, the use of AI in medical education is considered an encouraging area of expansion in clinical practice. All articles on the role of AI in medical education focus on the request for curriculum revision according to the newest technologies in education based on health policy. High-tech decision support systems will make changes in the traditional curriculum and improve the medical student’s knowledge, decision-making, and information-managing skills. These articles reviewed the impact of AI and VR in medical education for enhancing the knowledge, attitude, and skill of medical sciences students. Of course, there is still a need for more studies on how to integrate AI into diverse curriculum models and how to maintain the participation of AI in the real education system. It is possible to increase the awareness of faculties and students about the role of AI in medical education by holding workshops and symposia.

Ethics approval and consent to participate

All methods were performed following the relevant guidelines and regulations of the Helsinki Declaration and because it is a systematic review study, it does not need a code of ethics.

Availability of data and materials

The data supporting this study’s findings are available from the corresponding author upon reasonable request.

Conflicts of interest

There are no conflicts of interest.

Appendix 1.

Search strategies for databases

PubMed		(“medical education”[MeSH Terms] OR “medical teaching” OR “higher medical education” OR “ higher medical teaching” OR “precision medical teaching” OR “ precision medical education”)) AND (“artificial Intelligence “[MeSH Terms] OR “AI” OR “machine Intelligence”)
Web of Sciences		(TS=(medical education) OR TS=(medical teaching) OR TS=(higher medical education) OR TS=(higher medical teaching) OR TS=(precision medical teaching) OR TS=(precision medical education)) AND (TS=(Artificial Intelligence) OR TS=(AI) OR TS=(Machine Intelligence))
Scopus		(TITLE-ABS-KEY (medical AND education)) OR (TITLE-ABS-KEY (academic AND teaching)) OR (TITLE-ABS-KEY (higher AND medical AND education)) OR (TITLE-ABS-KEY (higher AND medical AND teaching)) OR (TITLE-ABS-KEY (precision AND medical AND teaching)) OR (TITLE-ABS-KEY (precision AND medical AND education)) AND (TITLE-ABS-KEY (Artificial AND Intelligence)) OR (TITLE-ABS-KEY (AI)) OR (TITLE-ABS-KEY (Machine AND Intelligence))
ERIC		((medical education) OR (medical teaching) OR (higher medical education) OR (higher medical teaching) OR (precision medical teaching) OR (precision medical education)) AND ((artificial Intelligence) OR (AI) OR (machine Intelligence))

Funding Statement

Nil.