A Student-Led Clinical Informatics Enrichment Course for Medical Students

Introduction

Clinical informatics leverages data to integrate analytics and clinical decision-making with aims to improve the quality, safety, and cost effectiveness of care delivery. 1 Numerous examples of clinical decision support systems (CDSS) have been shown to aid clinicians in diagnosis, outcome prediction, and disease management. Examples include but are not limited to infectious disease diagnosis, 2 thrombosis prophylaxis, 3 perioperative pediatric management, 4 and neurosurgical outcome prediction. 5 Such systems have been shown to benefit patient safety by reducing the incidence of adverse events such as medication errors, 6 7 8 care quality by ensuring adherence to established treatment guidelines, and cost reduction 9 by automating processes and decreasing resource waste. 10 11

Clinical informatics is a formally recognized medical subspecialty 12 13 with a multitude of applications and career opportunities as information technology becomes more intertwined with modern practices of medicine. 14 15 Significant strides have been taken to integrate core clinical informatics competencies into medical education, 16 17 18 as studies have shown medical students desire both broad training in informatics and opportunities for career exploration. 19 20 Examples include lecture-based online electives, 21 credit-based introductory medical informatics courses, 22 one-on-one faculty mentorship programs, 23 and isolated problem-based learning exercises. 24 While these examples demonstrate a growing emphasis on informatics education, they are limited with respect to number of participants and cohesive, clinically applicable skill-building that go beyond basic acquaintance of core concepts. To address this need, more integrated and contemporary informatics training seminars have recently emerged at institutions around the United States (i.e., New York University Langone Health, 25 University of California at San Francisco, 26 Oregon Health & Science University, 27 and Duke 28 ). In this spirit, the authors—four second and third year medical students—designed and delivered a 12-week informatics enrichment elective for first and second year medical students at our institution. Our goal was to introduce clinically relevant informatics concepts and foster development of technical skills, so that participants could apply informatics into research projects and during their time on the wards. We welcome medical students with similar goals to reach out for additional information.

The Challenges

We identified three challenges of introducing a clinical informatics curriculum in medical school as follows: (1) defining clinically applicable curriculum content for varied student needs, (2) supporting inclusivity for diverse student experiences and abilities, and (3) maintaining student engagement and minimizing attrition. After 2 years of piloting our course, the following reflection relays our approach to and learnings from tailoring content to first and second year medical students, who may not yet have deep medical knowledge or technical experience. We offer a few recommendations for others wishing to form similar educational initiatives.

We faced the challenge of tailoring a curriculum for broad student goals and needs. Clinical informatics encompasses a breadth of topics, each with varying relevance for medical students. 16 For example, every physician will interact with electronic health record (EHR) systems and require skills to efficiently navigate, correctly update, and effectively synthesize information. As reflected by accreditation standards to increase resident involvement in quality improvement, medical trainees are expected to engage with and improve patient care quality and safety. 29 As such, the role that informatics can play in monitoring and delivering quality metrics can be appreciated by all trainees. In terms of depth of knowledge, clinicians need not understand the mechanics of advanced statistical tests or be proficient in programming, but it is imperative that they understand how to apply decision support in care and how to interpret common statistical results from clinical studies and recognize limitations before applying results to patient care. 30 Similarly, while advanced gene sequencing technologies and machine learning (ML) algorithms are beyond the scope of medical practice, clinicians should be equipped for the advent of genomics and personalized medicine by understanding how decision support tools can utilize these technologies to transform individual patient care. 31

Needs Assessment

Prior to the elective's start, we surveyed our peers to understand their goals and topic interests. Participants were eager to learn skills for data capture (by chart review or database query), dataset organization and cleaning, ML, and use of basic statistical tools (Stata/R/Matlab). Additionally, students voiced interest in learning about the EHR structure and workflow optimization for more streamlined patient care encounters. We chose to offer lectures on introductory concepts, informal sessions to introduce career opportunities, and workshops intended to develop technical skills and showcase applications for clinical practice and research endeavors ( Table 1 ). Other notable parts of our curriculum included hands-on walk-through tutorials on implementing an ML workflow to predict office visit no-shows, navigating Epic during clerkships, customizing Epic SmartPhrases, cleaning clinical datasets using R, analyzing bioinformatics sequencing data using R, informatician faculty panels, industry speakers from leading digital health companies, and submitting institutional EHR data requests. After the elective's first iteration, certain didactics were modified in collaboration with leadership from our clinical informatics department to reduce redundancies and selected the most clinically relevant topics. Prior attempts at curriculum modifications through interdepartmental collaboration have shown to foster the development of new innovation. 32 After each weekly session, students were also asked to complete meeting-specific feedback forms which allowed us to iteratively adapt the elective's curriculum throughout its progression. Through collective decision-making with students and informatics experts, and keen identification of clinically relevant topics, we were able to design a curriculum at the forefront of students' interests and objectives.

Table 1. Clinical Informatics in Medicine curriculum with session offerings in 2020 and 2021 ( n  = 42 students; 2020: 24 students; 2021: 18 students) .

Session	Hours	Type	Attendance (%)	Description
Introduction to clinical informatics	1	Didactic discussion	100	Overview of data architecture, ontologies, and career opportunities with institution-specific examples
Radiology informatics	1	Didactic discussion	78	Application-focused discussion of informatics in the field of radiology
Supervised learning	3	Experiential	76	Guided walk-through of ML workflow (from raw data to model validation) to design classifier to predict appointment no-shows using Python
Data architecture and ontologies	1	Didactic discussion	81	Advanced lecture on institution data warehouse and ontologies
Pathology informatics	1	Didactic discussion	67	Application-focused discussion of informatics in the field of pathology
Deep learning for image processing	3	Experiential	69	Guided walk-through of ML workflow to design deep learning model to classify images using Python
Industry discussion	1	Didactic discussion	70	Guest speaker from leading digital health company
Assessing model performance	1	Didactic discussion	59	Lecture on common statistical metrics
Leveraging data and ML for leadership decision making	1	Didactic discussion	63	Application-focused discussion of informatics applied to hospital administration-level decisions
Evidence based medicine and clinical decision support	1	Didactic discussion	89	Summary of the clinical decision support developmental life cycle based on evidence based management guidelines
Health Information Technology	1	Didactic discussion	89	Brief overview of the current state of health information including architectures, data networks, and data flow
Precision medicine	1	Didactic discussion	72	Clinical considerations and general themes of applying unique patient specific genes and environmental considerations to improve personalized care
Clinical data analysis	2	Experiential	78	Guided walkthrough of data extraction, cleaning, and confirming data quality using R and deidentified patient dataset
Introduction to single cell sequencing	2	Experiential	72	Guided walk-through of basic analysis for sample single cell sequencing dataset using R
Introduction to Epic EHR	1	Experiential	72	Tutorial on Epic workflow and guided walk-through on building SmartPhrases

Abbreviations: EHR, electronic health record; ML, machine learning.

Focus on Useful Skills

To reach a broad medical student audience, the cognitive load theory was applied as a framework to teach applicable skills during medical school or residency. 33 Hands-on learning, for example, should not rely solely on skeletonized ML models with hyperparameter fine-tuning or other similar exercises. While growing fields, such as ML, are exciting to trainees, students at our institution reported that this highly specialized exercise was not valuable, unless ML was their domain of research interest. In contrast, students reported that the workshop on data extraction, cleaning, and quality checking using R was applicable to student research and quality improvement projects which often involve retrospective clinical data analysis and chart review. 34 35 Similarly, students appreciated the walk-through of Epic EHR, particularly on customizing templates and Smartphrases which taught skills they can put to use during clerkships.

Focus on Peer Education

As is innately present in any medical school class, our students had exceptionally diverse prior experiences. Students varied from past engineering majors to those with no prior background in programming or data analytics. This divide required consideration early on, as many interested students expressed hesitation during enrollment due to limited prior experience with informatics. The choice of student instructors was central to our mission of making our curriculum accessible to all medical students. Peer-teaching has been shown to result in student performance at least equivalent to that of faculty-led teaching with the additional benefit of cognitive congruence. 36 37 Cognitive congruence is the shared understanding of prior knowledge and recent challenges from undergoing the same curriculum. In addition, social congruence between peer-teachers and students fosters an open and supportive learning environment for questions and discussion. 38 39 As credentialed EHR trainers are expensive and often lack the clinical experience of a physician, institutions have also utilized medical students for physician training during new EHR implementations with comparable results, high student satisfaction, and significant cost savings. 40 Thus, technical workshop sessions were entirely led by students to ensure that concepts were simplified and taught in a language understandable by medical students. As upper classmen are uniquely positioned to share their approaches on facing common challenges experienced by medical students, we hoped peer-teaching would promote dissemination of curated, high-yield information for those just beginning their clinical experiences and forays into medical data.

Focus on Individualized Learning Styles

The rigorous medical school preclerkship curriculum necessitates students to be selective in their participation of extracurricular activities, requiring us to address challenges of attendance and engagement early on. Student interest was highest when content delivery was tailored to experiential learning through peer-to-peer hands-on workshops, live demonstrations, and question and answer (Q&A) sessions with industry and academic leaders ( Table 1 ). Practically, sessions were recorded online, scheduled around major examinations, and designed to be all inclusive with respect to work and teaching material, with no time commitment requirement outside of weekly meeting times. The elective was graded “Pass”/”Fail” based on 75% attendance which allowed for self-directed learning. 33 Students with little technical background could focus on real-world applications, while those more technical students could explore more advanced topics. Emphasis on student learning styles and understanding of competing priorities were foundational to the development of our informatics curriculum.

Conclusion

Our elective is the first of its kind at our institution, one which brings the growing field of clinical informatics to the early medical education through student-led experiential learning. The lessons presented here were the collective thoughts and trials from 42 medical students who took the elective during a 2-year pilot study. Our emphasis on curriculum content, audience, and delivery methods led to a positive student experience. Also, 100% of respondents in a final feedback survey reported that they “would recommend the elective to future students.” Undoubtedly, the elective can be improved. As future classes and new student instructors take the reins in leading this elective, its content and delivery will continue to evolve and be refined.

Funding Statement

Funding None.