Clinical informatics training in medical school education curricula: a scoping review

Abstract

Objectives

This scoping review evaluates the existing literature on clinical informatics (CI) training in medical schools. It aims to determine the essential components of a CI curriculum in medical schools, identify methods to evaluate the effectiveness of a CI-focused education, and understand its delivery modes.

Materials and Methods

This review was informed by the methodological guidance of the Joanna Briggs Institute. Three electronic databases including PubMed, Scopus, and Web of Science were searched for articles discussing CI between January 2010 and December 2021.

Results

Fifty-nine out of 3055 articles were included in our final analysis. Components of CI education include its utilization in clinical practice, ethical implications, key CI-related concepts, and digital health. Evaluation of educational effectiveness entails external evaluation by organizations external to the teaching institute, and internal evaluation from within the teaching institute. Finally, modes of delivery include various pedagogical strategies and teaching CI using a multidisciplinary approach.

Discussion

Given the broad discussion on the required competencies, we propose 4 recommendations in CI delivery. These include situating CI curriculum within specific contexts, developing evidence-based guidelines for a robust CI education, developing validated assessment techniques to evaluate curriculum effectiveness, and equipping educators with relevant CI training.

Conclusion

The literature reveals that CI training in the core curricula will complement if not enhance clinical skills, reiterating the need to equip students with relevant CI competencies. Furthermore, future research needs to comprehensively address current gaps in CI training in different contexts, evaluation methodologies, and delivery modes to facilitate structured training.

INTRODUCTION

Many developed countries are facing an ageing population with multiple morbidities. Holistic approaches to managing patients with multi-morbidities have been proposed.^1,2 Complementing this approach is the increasing digitalization in healthcare, characterized by the integration of digital technologies such as Artificial Intelligence (AI), Internet of Things, and machine learning (ML) and the creation of a comprehensive health data repositories.³ Increasingly, clinicians are required to know how to utilize digital technologies in an efficient and patient-centered manner.^4,5 However, training in the use of these technologies is limited and does not commonly form part of the core medical school curriculum; it may be offered as an elective program in some schools.^6–9 Consequently, medical graduates are trained in these technologies only when they enter residency or specialty training programs.^4,5 Studies have shown that equipping students with relevant digital competencies in a structured and longitudinal manner will enhance their clinical skills.^6–9 Yet, few articles have specifically addressed how digital technologies may enhance or complement core clinical skills, which remain fundamental to the combination of the art and science of doctoring even in the digital age.

The definition of “clinical skills” is multifarious. However, consensus of opinions include clinical examination, clinical reasoning, communication skills, and procedural skills as core domains of clinical skills that medical graduates should be proficient in.^10–12 In light of the rapid progress of digital transformation in healthcare, clinical skills must evolve to embrace technologies such as AI and portable ultrasonography.^13,14 In this information age where available medical knowledge exceeds the organizing capacity of the human mind, medical education must move from “knowledge acquisition” to knowledge management, application, and communication.^15,16 Additionally, as information about medical knowledge and public reviews of physicians become more readily available on the internet and other web-based social spaces, the traditionally paternalistic doctor-patient relationship must be reconsidered for a patient-centered partnership.¹²

Against this backdrop of digital health transformation, clinical informatics (CI) has emerged as a discipline that physicians should be familiar with. Unlike the broader field of “Biomedical and Health Informatics” that involves the combination of healthcare, information technology (IT), and communications in general, CI is a subspecialty field that places an emphasis on the analysis, design, and evaluation of information and communication systems to “improve patient care, enhance access to care, advance individual and population health outcomes, and strengthen the clinician-patient relationship”.^17,18 CI education exists through specialty training (eg, through ACGME recognized specialty training in the United States), or through Masters level courses in numerous universities.

However, the literature on CI education in medical schools, and prior to specialist training, has not been comprehensively reviewed even though there is evidence that CI education enhances students’ clinical skills. The potential benefits of CI education in medical school include empowering junior doctors with the ability to adapt bioinformatics databases to novel clinical situations, and increasing their confidence in broad clinical genetics skills.⁷ Other potential benefits of CI education include improving students’ skills in handling medical data, enhancing digital infrastructure of the health system, enabling precision medicine, and increasing familiarity with medico-legal and ethical issues with health digitalization.⁸ In this study, we performed a scoping review with the following research questions in mind: What are the essential components of a CI curriculum in medical school education prior to specialist training (henceforth referred to as medical school education)? What are the methods to evaluate the effectiveness of a CI-focused education? What are the modes of delivery of CI education?

 

A preliminary search of MEDLINE, the Cochrane Database of Systematic Reviews, and JBI Evidence Synthesis was conducted and no current or ongoing systematic reviews or scoping reviews on the topic were identified. In addressing the 3 research questions, we aimed to (1) explore the essential components of a CI curriculum in medical school education, (2) assess the methods of evaluating the effectiveness of CI education, and (3) examine the modes of delivery of a CI-focused curriculum. In so doing, we anticipated that this review would provide a systematic summary of essential knowledge needed to establish an effective CI curriculum for medical school education. It will also identify any gaps in the literature that could be used to improve the CI curriculum. Findings from this review will be useful for other researchers, curriculum designers, and educational policy makers.

MATERIALS AND METHODS

Search strategy

We conducted a scoping literature review following the methodological guidance of the Joanna Briggs Institute.¹⁹ The results were reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines.²⁰ During the development of our search strategy, we adopted the ACGME definition of CI (defined above).¹⁷ The key domains of CI were identified, which include data science (AI and ML), big data, health information management, and data analytics.²¹ We restricted the scope of this review to medical school education (prior to specialist training) because the evidence base for CI in medical school education is not well established, and we anticipated that CI training at the specialist level may be specialty-specific and focused on concepts or applications that are not applicable to all medical practitioners. By focusing on medical school education, we sought to identify concepts that could be foundational to all medical practitioners. Our search strategy, created with the help of a medical librarian, consisted of medical subject headings, keywords, and text words related to CI and its domains. Keywords included CI, medical informatics, data science, big data, and data mining. Since the realm of data science in healthcare is progressively expanding toward AI and ML, we included these 2 subjects as keywords. The full version of the search strategy can be found in Supplementary Table S1. The initial search was conducted on December 7, 2021 in 3 electronic databases, including PubMed, Scopus, and Web of Science. The search was updated on December 28, 2021.

Eligibility criteria

We included all articles published in English between January 1, 2010 and December 31, 2021. We limited the search by publication date and language due to the rapidly evolving nature of CI and limited access to translation services, respectively. We also included all articles that discuss digital health, CI, AI, and ML in medical schools, regardless of setting. We reviewed experimental (eg, randomized controlled trials), quasiexperimental (eg, pre-post studies), observational and descriptive studies (eg, case studies). Additionally, we included systematic reviews, scoping reviews, editorials, commentaries, and letters to editor since the area of CI in medical school education is relatively novel. The inclusion criteria were developed in alignment with the aims of our review.

Articles were excluded if they focused exclusively on CI in postgraduate or continuing medical education (CME), on teaching CI to allied health professionals, on the domains of CI as tools for medical education as opposed to a topic within medical education curricula, were not in English, or where full-text manuscript was not available.

Source of evidence selection

All identified citations were collated and uploaded onto Mendeley Desktop version 1.19.8 and duplicates removed. Study selection was performed in 2 steps using our predefined inclusion and exclusion criteria. First, HZ and JKT independently screened titles and abstracts and then discussed the discrepancies. Afterwards, HZ and JKT independently screened the full texts and reviewed discrepancies together. Any discrepancies or disagreements between the reviewers were resolved through discussion and consensus, and when required, a third reviewer, XX, was involved as an arbiter.

Data extraction

Data were extracted from selected papers by HZ and JKT using a structured form (see Supplementary Table S2). We met to ensure consistency between forms, resolve disagreements, and refine the form based on increased familiarity with the literature. As the content was broad, we explored themes and subthemes that emerged from the papers using iterative thematic analysis and presented them accordingly. HZ and JKT coded themes and subthemes independently before conducting further discussions to refine the themes. Any disagreements that arose between the researchers was resolved through discussion with the team.

RESULTS

Study characteristics

Our search identified 3055 unique titles, of which 59 full-text articles were included in the final analysis. The selection process of these articles is detailed in Figure 1. Most of the articles were perspective articles (n = 33, 55.9%). These included commentaries, opinion editorials, and letters to editors. Other articles included original research papers, reviews, and a case report (Table 1). There was a diverse representation of study settings, as determined by study location or the author’s affiliated country. Most study settings were from North America and Western Europe (Table 1). Few Asian countries were represented. More than half of the articles were published between 2019 and 2021. Our findings are presented in Table 2 and described below.

Figure 1.

Adapted Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) flow diagram.

Table 1.

Article characteristics

Study characteristics	No. (%) of articles (n = 59)
Study type
Perspective/commentary/opinion/editorial	33 (55.9)
Original research	17 (28.8)
Review	8 (13.6)
Case report	1 (1.7)
Study location or authors’ affiliated country
United States	26 (44.1)
Canada	8 (13.6)
Germany	6 (10.2)
Multiple countries  (Australia, New Zealand,  United States, and European countries)	3 (5.1)
Mexico	2 (3.4)
Taiwan	2 (3.4)
United Kingdom	2 (3.4)
Australia	1 (1.7)
Bosnia and Herzegovina	1 (1.7)
Denmark	1 (1.7)
Netherlands	1 (1.7)
New Zealand	1 (1.7)
Oman	1 (1.7)
Pakistan	1 (1.7)
Serbia	1 (1.7)
South Korea	1 (1.7)
Turkey	1 (1.7)
Year of publication
2021	15 (25.4)
2020	13 (22.0)
2019	7 (11.9)
2018	7 (11.9)
2017	6 (10.2)
2015	2 (3.4)
2014	2 (3.4)
2013	2 (3.4)
2012	2 (3.4)
2011	1 (1.7)
2010	2 (3.4)

Table 2.

Summary of main findings from original articles and opinion pieces

Research question	Themes	Subthemes	Citation (original research articles)	Citation (opinion/perspective articles)
What are the essential components of a CI curriculum in medical school education?
	Utilization of CI in clinical practice
		Safe and effective application and integration of CI tools in clinical practice	22 , 23	9 , 22–30
	Ethical implications
		Knowing the limitations and possible risks of CI tools. This include avoiding the pitfall where certain patient groups may be disadvantaged by AI biasness or inequity of technological access		8 , 24 , 31–39
		Understanding the medicolegal aspects of CI	8 , 33 , 37	24 , 34 , 36 , 38
		Knowing how to embody the humanistic aspects of medicine while using technology (eg, empathy and compassion toward patients)		16 , 26 , 29 , 40–43
		Knowing how to communicate data effectively to patients amid the use of technology and other communication skills		15 , 26 , 30 , 34 , 36 , 38 , 40 , 44 , 45
	Concepts related to CI, AI, and ML
		Data management including data protection and security	8 , 33 , 46 , 47	4 , 15 , 29 , 30 , 34 , 40
		Data analytics, data science, computer science, and mathematical sophistication	48	15 , 24 , 26 , 34–36 , 41 , 42 , 49–52
		Data utilization to improve health systems (advanced training)	6 , 8	53 , 54
	Digital health
		Telemedicine	37 , 46
		EHRs	37 , 55	36 , 39 , 56 , 57
		Health IT	47	38 , 44 , 53 , 58–60
		Wearable devices		28 , 34
		Personalized medicine		61
What are the methods to evaluate the effectiveness of a CI-focused education?
	External evaluation (conducted by organization external to teaching institute)
		Respondents were academic staff
		Qualitative responses (via email/online)	62 , 63
		Quantitative survey	63
		Respondents were students
		Formative examination and survey	64
	Internal evaluation (conducted from within teaching institute)
		Program outcomes
		Academic and operational outcomes	65
		Self-perceived readiness
		Medical Artificial Intelligence Readiness Scale for Medical Students (MAIRS-MS)	66
		Self-perceived knowledge and opinions
		Qualitative and quantitative questions	33
		Satisfaction/experience
		Student satisfaction survey	67	7
		Student learning experience survey		52
		Taught content
		Pre- and post-test for learning performance	67
	Evaluation of methods used to evaluate CI training programmes
		High quality evaluation studies are lacking.		56 , 58
		Limited focus on higher order skills (eg, secondary aggregation, extraction, and appraisal of the data)		56
		Limited practical components (eg, using adjunctive tools, creating patient resources, or conducting audits of recorded information).		56
What are the modes of delivery of CI education?
	Learning styles
		Adaptive learning (eg, Master Adaptive Learning Model)	48
		Precision education	48
		Horizontal and vertical integration of content		25
		Curriculum mapping to enhance interdisciplinary teaching cooperation	47
	Delivery styles
		Blended learning module	67	58 , 68
		Interactive sessions	6
		Distance learning/virtual classroom		45 , 69
	Multidisciplinary approach
		Collaboration with professional organizations		24
		Collaboration with other academic departments		24 , 36 , 70 , 71
		Conferences with experts		24
	Application of content in different contexts
		Crossinstitutional compatibility (of EHR module)	55

Essential components of a CI curriculum

Overall, the articles highlighted that the fundamentals of CI in medicine should be incorporated throughout the years of medical school, with some arguing for its utilization in clinical practice more so than others.^9,29 There was also consensus across the articles that other skills, such as driving innovations, and acquiring full proficiency in CI, are better suited to be taught as extracurricular components or in the advanced years of medical education instead.³⁰

Several articles discussed how AI systems can be critically appraised to ensure their safe and effective utilization in clinical practice.^9,28,29 These include Soong and Ho’s (2021) study that highlighted how AI can be utilized in history-taking, physical examination, procedural and clinical decision-making skills without the algorithms replacing the clinical reasoning process.⁹ Others, such as those by McCoy et al (2020), Wartman and Combs (2019), and Sánchez-Mendiola et al (2013) discussed the importance of imparting to students a better understanding of the inputs required to receive meaningful results, and of applying probabilities meaningfully to support clinical decision-making processes.^31,32,35

Additionally, a number of articles underscored the value of integrating key principles such as the ethical implications of using ML and AI tools. These included knowing the limitations and possible dangers of utilizing such tools, avoiding the pitfall where certain patient groups may be disadvantaged as a result of AI bias or inequity in technological access, and understanding the medicolegal aspects of AI.^7,28,39,41 Moreover, the humanistic aspect of medicine is an important consideration in the age of digital technology; at least 10 articles highlighted the importance of displaying empathy and compassion toward patients as well as communicating data meaningfully to them amid the use of technology in healthcare.^{4,5,21,31,34,39,41,43,46,51}

Others discussed some of the AI concepts that should be taught to students, which include data analytics, data science, and computer science.^{21,41,46,51–53} Developing their understanding of data management, be it in terms of data protection and security, the interpretation of clinical research results, the handling of big data in the context of decision-making, took center stage in at least 8 of the articles.^{9,10,14,21,31,34,38,39} Advanced training entails equipping them with skills in utilizing data to improve health systems.^6,53

Our review also underscored digital health as a field that would be useful for medical students. According to the US Food and Drug Administration, digital health broadly includes technologies such as mobile health, health IT, wearable devices, telemedicine, and personalized medicine.⁷² Indeed, 11 articles discussed how students can be taught to interact with patients effectively while using tools such as telemedicine or electronic health records (EHRs).^{4,6,28,29,37,38,46,73–76}

Methods to evaluate the effectiveness of CI education

Ten articles discussed methods to evaluate the effectiveness of CI education. The evaluation techniques discussed can be broadly classified into external and internal evaluations. External evaluations were conducted by organizations external to the teaching institute, which assessed the effectiveness of CI by the teaching institutions. Hurley et al (2011) and Walpole et al (2017) used online surveys sent to academic staff to evaluate CI education in Canada and the United Kingdom, respectively.^62,63 Walpole et al (2017) also administered a quantitative survey to collate descriptive data about CI education.⁶³ Blacketer et al (2021) used a formative examination and survey to evaluate medical students’ understanding of ML concepts and critical analysis of ML research articles.⁶⁴

Internal evaluations were conducted from within the teaching institutes, which assessed the effectiveness of CI education conducted by the institutions. These included self-perceived readiness,⁶⁶ self-perceived knowledge and opinions regarding CI,³³ satisfaction or experience with the CI course,^7,52,67 and the assessment of taught content.⁶⁷

Two articles reviewed the methods used to evaluate CI training programs. Car et al (2021) conducted a scoping review of digital health training programs for medical students and found that high-quality evaluation studies are lacking, since most articles evaluated courses by using an uncontrolled before-and-after design.⁵⁸ Rajaram et al (2020) conducted a systematic review of educational interventions training medical students and residents in the use of EHRs and found possible training gaps due to limited focus on higher order skills (eg, secondary aggregation, extraction, and appraisal of the data) and limited practical components (eg, using adjunctive tools, creating patient resources, or conducting audits of recorded information) in the educational programs.⁵⁶

Modes of delivery of CI education

Fourteen articles discussed the modes of delivery of CI education. The broad themes discussed were delivery styles, learning styles, utilizing a multidisciplinary approach, and ensuring content was applicable in different contexts. A variety of learning styles for teaching CI were discussed, including blended learning modules,^58,67,68 interactive sessions, clinical skills demonstrations, and enrichment sessions on specialized topics.⁶ Distance learning and virtual classrooms were also discussed, as it enables CI education to reach learners across a larger geographic region.^45,69

Three articles elucidated how the modes of delivery should cater to different learning styles. Cutrer et al (2021) explained the importance of adaptive learning using the Master Adaptive Learning Model, and the role for precision education in CI.⁴⁸ James et al (2021) discussed horizontal integration (incorporate ML into core doctoring and clinical skills courses) and vertical integration (utilizing ML concepts in clinical practice environment) of CI education.²⁵ Behrends et al (2017) highlighted the curriculum mapping experience used to identify common ground between CI and other medical subjects, and in doing so aim to identify meaningful interdisciplinary teaching cooperation.⁴⁷

A multidisciplinary approach to CI education was also discussed, with the need to collaborate with professional scientific organizations and academic departments beyond medicine (eg, bioinformatics, bioengineering, computer science, and statistics).^24,36,70,71 Valikodath et al (2021) highlighted that conferences with domain experts could be a good avenue for delivering practical CI education.²⁴ Finally, Pereira et al (2018) noted that practical aspects of CI education, particularly the use of EHRs, should have cross-institutional compatibility to reduce training burden on learners.⁵⁵

Mapping competencies with ACGME milestones

We analyzed 17 original research articles and explored the extent to which they covered the ACGME CI and milestones framework by identifying the core competencies of ACGME fulfilled by each article, if any.⁷⁷ These are namely, patient care, medical knowledge, practice-based learning and improvement, interpersonal and communication skills, professionalism, and systems-based practice (Table 3). Specific milestones that should be accomplished within each competency are also highlighted under the respective columns (Table 3).

Table 3.

Summary of ACGME competencies and milestones covered in selected original research articles

Article (Brief description)	ACGME competencies and milestones
	Patient care Consumer informatics applications, portals, and telehealth Existing and emerging data sources	Medical knowledge Project management Health information technology (HIT) knowledge of current and new testing, implementation, and monitoring	Practice-based learning and improvement Optimization, downtime, functional requirements Clinical decision support (CDS) Analytics Human-Computer interaction (HCI) and user interfaces (UI) Reflective practice and commitment to personal growth	Interpersonal and communication skills Communicate effectively with multiple constituencies Building consensus Interprofessional and team communication Communication within health care systems	Professionalism Governance Mentorship Professional Behavior and ethical principles Accountability/conscientiousness Self-awareness and help-seeking	System-based practice HIT knowledge of current and new testing, implementation, monitoring Standards and interoperability Data integrity/security
Behrends et al (2017) (Describes where learning objectives related to Medical Informatics (MI) in Hannover coincide with other subjects and where they are taught exclusively in MI)	✓	✓	✓	✓	✓	✓
Cutrer et al (2021) (Exploration of medical school’s educational programming approach to the rapid expansion of information)	✓	✓	✓	✓	✓	✓
Foadi et al (2021) (Implementation of learning outcomes with respect to digital competencies in the compulsory curriculum at Hannover Medical School)	✓	✓	✓	✓	✓	✓
Hersh et al (2017) (Development of a medical informatics curriculum at Oregon Health & Science University)	✓	✓	✓	✓	✓	✓
Walpole et al (2017) (Survey of senior academic staff and educators about practice of health informatics in United Kingdom medical schools)	✓	✓	✓	✓	✓	✓
Aulenkamp et al (2021) (Scan of medical education courses with digital competencies in German universities)	✓				✓	✓
Machleid et al (2020) (Mixed methods survey to assess European medical students’ perceived knowledge and opinions toward digital health, the status of digital health implementation in medical education, and the students’ most pressing needs)	✓	✓				✓
Pontefract and Wilson (2019) (Development of competency domains and learning outcomes to integrate electronic patient records into undergraduate education for healthcare students)	✓			✓		✓
Sánchez-Mendiola et al (2015) (Assesses knowledge change and satisfaction in medical students after a biomedical informatics course curriculum)	✓		✓	✓
Sendak et al (2021) (Presentation of academic, operation, and domain understanding outcomes)	✓	✓				✓
Blacketer et al (2021) (Formative examination and survey to evaluate medical students [in Australia, New Zealand, and United States] understanding of machine learning)		✓			✓
Karaca et al (2021) (Development and psychometric assessment of a tool to assess the perceived readiness of medical students on AI technologies and its applications in medicine)		✓			✓
Edirippulige et al (2018) (A national interview study and interpretative phenomenological analysis with participants from all 19 medical schools in Australia)	✓
Huang et al (2020) (Development and implementation of a blended learning model course regarding application of data science in medical fields for medical students)	✓
Hurley et al (2011) (Environmental scan of health informatics teaching courses in Canadian Medical Schools)	✓
Pereira et al (2018) (Development of a single competency-based HER [Epic] onboarding process portable across multiple institutions)	✓
Wood et al (2021) (Assesses medical student and faculty attitudes toward AI, in using a semistructured survey)

The extant literature reveals a lack of discussion on the ACGME milestones and competencies. Of the few that explained all competencies were those that attempted to implement specific learning outcomes and objectives related to digital competencies in the medical school curriculum. Articles that examined student or faculty attitudes or understanding of CI did not discuss the competencies in depth.

DISCUSSION

This scoping review summarizes the existing literature regarding CI training in medical school education, mapping key themes that can inform medical education and future research and identifying gaps in the literature. We found an increasing number of articles each year from 2017 to 2021, with more than half of the articles being published between 2019 and 2020. This reflects a growing interest in CI as information and communication systems become more pervasive in the healthcare landscape. The breadth of topics addressed in selected articles (Table 2) confirms the wide-ranging nature of the discipline.

Nonetheless, there are a number of gaps that need to be addressed by future research. Based on our scoping review, the existing literature are largely focused on the importance of communication skills while using CI and few discuss how CI can be effectively utilized in treatment and diagnosis. Moreover, existing studies that utilized evaluation methodologies to evaluate the efficacy of the teaching interventions are not robust enough as they were mostly based on self-assessment of study participants. These included post-intervention self-reported confidence, reviews of learning experiences, or post-intervention tests. Few articles incorporated a pre-post study design or performed validation studies on their assessment tools. Hence, there is a need for discussion and consensus on the core competencies of medical school CI curricula, robust modes of assessment to determine the efficacy of CI education programs, discussions on how to enhance education capabilities alongside the humanistic aspects of medicine, and importantly, recommendations on implementing CI training beyond the Western context. To this end, we propose 4 recommendations that medical educators should consider in their efforts to develop and deliver CI curricular in medical schools.

Understand the context for CI education

The majority of the literature in this scoping review was from North America and European countries. This corresponds with the advances in digital health transformation in these countries, particularly in the adoption of EHRs and digitalization of medical systems and processes. In contrast, the smaller number of articles from Asian countries could be because EHR adoption and healthcare digitalization are still in the early stages.

Medical education should be guided by the context in which medicine is practiced. Medical educators should consider the state and trajectory of healthcare digitalization in their country. Where digital health transformation is still in its infancy, the literature has highlighted that health educators should focus on EHR literacy and fundamental skills in CI, including ways to collect, analyze, and use health data.^6,55,56,61 In contexts where the health system has undergone significant digital transformation, educators should focus on more advanced CI skills including AI, ML,^{30,35,36,50,51} advanced database management,⁷ and metacognition.⁴⁸ We recognize that the ACGME competencies and milestones were designed for specialist training post-medical school. Nevertheless, we recommend that medical educators consider the ACGME competencies and milestones when designing their CI curriculum. In this study, we have identified original research articles that discuss the 6 ACGME CI competencies and milestones (Table 3). As suggested by our findings, introductory materials that address these competencies and milestones will prepare junior doctors for the digitally enabled future.

In addition, we recommend that bioethics of CI should be a key component of medical education and tailored to the medical practice context.^26,44 Given the pace at which information technologies is advancing, medical students should be aware of ethical pitfalls in emerging technologies. Students need to understand about the risks around data security and privacy, and the potential for AI technologies to perpetuate biases.³⁰

Develop and implement evidence- and theory-informed CI education

As demonstrated by our scoping review, CI education in medical school education is heterogeneous.^46,63 Medical educators developing new CI curricula or updating existing curricula should consider the experience and evidence from available literature (Table 2). Curriculum maps, such as those described by Hersh et al (2017) at Oregon Health & Science University, are useful for curriculum development because the maps describe CI competencies, their associated learning objectives, and timeline to introduce each competency during the educational journey.⁶ Medical educators could work at the national, regional, and international level to develop evidence- and theory-informed guidelines for CI teaching and assessment.^50,63

We noted the barriers to implementing CI education, which include the already demanding curriculum with competing priorities²³ and the limitations of CI capabilities (addressed in section “Enhance CI education capabilities alongside humanistic aspects of medicine”). The traditional medical education structure needs to be reviewed in light of the rapid expansion of medical knowledge contributing to the overwhelming medical curriculum. Medical students should be trained in adaptive skills; critical thinking and data analytics should be promoted over rote memorization, and assessment methodologies in schools should change to reflect real world circumstances.⁴⁸ CI education underpins these adaptive skills; Behrends et al (2017) undertook a mapping exercise of the medical curriculum at Hannover Medical School and found that CI extensively overlapped with all medical subjects.⁴⁷ CI education should not be viewed as an additional subject within an already dense medical curriculum. Rather, medical educators should integrate CI education within the core curricula and reduce or eliminate content that are less relevant to future clinical practice.

Develop robust evaluations to determine efficacy of CI education

In view of the limitations of evaluation strategies discussed in the extant literature, research should explore pedagogies and assessment strategies for CI education,⁶³ utilize validated assessment techniques,⁷⁸ and examine long-term retention of learning outcomes to fully understand the impact of different curricular and instructional design approaches.⁷⁹ A valid assessment tool should measure the knowledge and skills that are the goals of the curriculum. Psychometrically valid assessments are available for evaluating CI competencies at the specialist training,⁸⁰ but not at the medical school level. This could be a future area of research and development.

Enhance CI education capabilities alongside humanistic aspects of medicine

In our review, many authors opined that the lack of CI education capability is a barrier to implementing CI curriculum.^16,24,50,62 Many authors proposed interprofessional and cross-faculty collaboration between healthcare experts, engineering, and computer science faculties as a means to overcome barriers to effective implementation of CI curriculum.^{8,16,50,62,71} We identified 2 articles that detailed experience with an interdisciplinary approach.^27,71 Interestingly, 1 article highlighted the need to support educators with training opportunities.⁵⁰ Furthermore, with the advancement of CI capabilities, there should be a renewed focus on the humanistic aspects of medicine.^41–43,50 Critical attributes such as professionalism, communication, empathy, compassion, and respect should be emphasized even more in medical schools amid increasing healthcare digitalization and taught by educators who are trained in these areas.

 

Our review is underpinned by several limitations. First, we only included articles written in English and did not explore the gray literature, that is, articles published in non-scientific journals such as government reports. There might be articles in non-English languages that could have provided insights into CI education in countries where English is not commonly used. Nonetheless, given the variety of recurring themes in the works we analyzed, the comprehensiveness of topics is achieved in this review. Second, our review only focused on medical school education. We acknowledge that an evaluation of articles on postgraduate or CME may provide further insights on valuable tools for CI training. This would be at the potential caveat of identifying insights that might only be applicable to a certain field of postgraduate training. Finally, it should be noted that more than a third of the articles are based in the United States. Identified themes and insights would need to be adapted for medical training programs in other contexts.

CONCLUSION

We have summarized published literature about the state of CI education and recommend ways to improve the delivery and assessment of CI. Despite the large volume of literature on CI training in medicine, few discussed the integration of CI in the core curriculum of medical school education, particularly within the context of clinical training. Based on our review, although some articles addressed the usefulness of teaching CI competencies for clinical practice, some skills, particularly communication skills, are emphasized more than others, such as procedural and clinical reasoning skills. Furthermore, robust evaluation strategies were seldom employed in the studies, which calls for more robust assessment tools such as psychometrically valid assessments for future research. There is also little consideration for a CI curriculum that is contextualized to the needs of different healthcare contexts, particularly in Asian countries, which can be addressed by future studies. Overall, in view of the benefits that CI education brings to patient care, a structured training within the core medical school curricula will equip students with relevant digital competencies for the future healthcare landscape.

FUNDING

This study was supported by SingHealth Duke-NUS Medicine Academic Clinical Programme under Seah Cheng Siang Distinguished Professorship in Medicine.

AUTHOR CONTRIBUTIONS

The corresponding author, HZ, and the second author, JKT, designed the study, analyzed and interpreted the data, as well as drafted and revised the manuscript. The third, fourth, and fifth authors, XX, JT, and FKY, reviewed and edited the manuscript. All authors read and approved the submitted copy.

SUPPLEMENTARY MATERIAL

Supplementary material is available at Journal of the American Medical Informatics Association online.

DATA AVAILABILITY

The data underlying this article will be shared on reasonable request to the corresponding author.