Artificial Intelligence and Radiology Education

Abstract

Implementation of artificial intelligence (AI) applications into clinical practice requires AI-savvy radiologists to ensure the safe, ethical, and effective use of these systems for patient care. Increasing demand for AI education reflects recognition of the translation of AI applications from research to clinical practice, with positive trainee attitudes regarding the influence of AI on radiology. However, barriers to AI education, such as limited access to resources, predispose to insufficient preparation for the effective use of AI in practice. In response, national organizations have sponsored formal and self-directed learning courses to provide introductory content on imaging informatics and AI. Foundational courses, such as the National Imaging Informatics Course – Radiology and the Radiological Society of North America Imaging AI Certificate, lay a framework for trainees to explore the creation, deployment, and critical evaluation of AI applications. This report includes additional resources for formal programming courses, video series from leading organizations, and blogs from AI and informatics communities. Furthermore, the scope of “AI and radiology education” includes AI-augmented radiology education, with emphasis on the potential for “precision education” that creates personalized experiences for trainees by accounting for varying learning styles and inconsistent, possibly deficient, clinical case volume.

© RSNA, 2022

Keywords: Use of AI in Education, Impact of AI on Education, Artificial Intelligence, Medical Education, Imaging Informatics, Natural Language Processing, Precision Education

Summary

The increasing presence of artificial intelligence (AI) applications in radiology necessitates education to prepare trainees and radiologists as proficient users and stewards of AI technology. The authors propose content for an AI curriculum, suggest resources for formal and self-directed learning, and discuss the potential of AI-augmented radiology education.

Key Points

• ■ Continued growth of artificial intelligence (AI) necessitates education to prepare radiologists for effective, ethical, and judicious use of AI tools in their clinical practice.

• ■ This report identifies resources from national organizations for formal introductory AI education and additional supplemental resources for self-directed learning.

• ■ AI facilitates “precision education,” enabling personalized trainee experiences that account for differences in learning styles and specific limitations in clinical exposure over the course of training.

Introduction

Persistent growth of artificial intelligence (AI) applications in radiology has piqued the interest of radiologists and radiology trainees. Surveys show that trainees are keenly interested in learning about AI (1,2). The breadth of AI-augmented tasks throughout the radiology value chain emphasizes the notion that radiologists will interact with AI beyond diagnostic processes. However, education about AI faces several limitations owing to variable availability of expertise and resources. Increasing access to AI education requires a collaborative effort between AI experts, radiologists, and trainees to create a framework that addresses trainees’ needs within a context of shifting paradigms and resources. Accordingly, several groups have issued calls to action for preliminary AI curricula and offered frameworks for integrating AI education with current radiology training (3–5).

Fortunately, calls for increased AI education have prompted the creation of formal curricula in radiology training programs (6). National organizations have consolidated resources for independent learning spanning from foundational concepts to advanced topics in the development and clinical deployment of AI. Moreover, courses offered through sponsoring organizations, including the Radiological Society of North America (RSNA) and the Society for Imaging Informatics in Medicine (SIIM), offer both self-directed learning and formal didactics to allow flexibility for hectic clinical and training schedules. As national organizations and individual programs work to provide education in AI, continued interdisciplinary discussion will ensure that the latest developments are at the forefront of trainee education. Critical assessment of the current state of AI education is warranted to understand persisting knowledge gaps and new challenges that arise from the clinical deployment of AI applications.

Furthermore, the topic of “AI and radiology education” lends to the discussion of AI’s potential to augment radiology training, underscoring the ability of AI to enable “precision medical education” (7). AI-augmented education enables training programs to create personalized experiences accounting for trainee learning styles and inconsistent, possibly deficient, exposure to clinical cases.

Demand for AI Education

Sustained growth of AI in radiology requires radiologists to operate AI tools adeptly and guide the creation of clinically meaningful applications. Although on-the-job training may provide a level of proficiency, it may not suffice to provide the critical evaluation necessary to ensure the safe, ethical, and effective deployment of AI in practice.

The introduction of AI to radiology draws parallels with the rise of MRI, which initially experienced substantial research attention and exponential growth in publications in the 1980s and 1990s (8). Advances in MRI technology, coupled with an improved understanding of its strengths and limitations, prompted its clinical deployment as an essential diagnostic tool. Accordingly, radiology residencies adapted by providing robust MRI experience during training, allowing radiologists to serve as experts in the underlying physics, clinical application, and critical assessment of MRI in practice. In this context, AI research demonstrates similar, if not greater, exponential growth as did MRI research. Searching the terms machine learning or deep learning on PubMed reveals a near quadrupling of AI-related publications from 2016 to 2019 (9). These publications include traditional research articles as well as perspectives on the ethics of AI, medicolegal considerations, health policy, and guidelines from leading organizations in medicine. Following this trend, AI will likely continue to dominate conversations from academic platforms to casual conversations. Ultimately, patients will learn about the role of AI in their care and may become susceptible to misinformation from lay media in the absence of knowledgeable sources. Consequently, radiologists must prepare to assume the role of a trusted authority on AI; they should leverage an understanding of basic principles to evaluate new applications and capitalize on opportunities to use AI to optimize patient care. Early exposure to AI education during training will allow trainees to serve as AI experts to guide the clinical deployment of AI.

Surveys continue to show a demand for AI education, which reflects recognition of AI’s mounting research and clinical presence (1,2,10–12). Recent studies demonstrate global interest in the integration of AI education in residency programs; many expect that AI holds potential for a substantial shift in radiology workflow within the next 10 years (2,10). Moreover, the surveys suggest that exposure to AI can affect the outlook of radiologists and radiology residents toward AI; specifically, education may reduce fear of this unfamiliar technology and inspire further exploration into understanding AI tools used in clinical practice (1).

In fact, exposure to AI as early as medical school may help combat misinformation that can influence medical students’ perceptions of radiology. Surveys have demonstrated that most radiologists, radiology trainees, and medical students believe that AI will have a substantial impact on their careers (13). However, “substantial impact” may represent differing sentiments for radiologists and trainees compared with medical students. Specifically, attending physicians and trainees convey a positive attitude toward the impact of AI on their practice within the next 5 years. On the other hand, surveys suggest that anxiety regarding AI-driven “displacement” of radiologists may discourage medical students from pursuing a career in radiology (14). Review of survey data specifically from medical students reveals that a majority of respondents desire AI education before residency (10–12). Survey data also suggest that although current perceptions regarding AI’s impact on radiology negatively influence the decision to apply to radiology, exposure to AI may have a protective effect by addressing potential misperceptions (14,15). Thus, AI education offers an opportunity to dispel myths regarding radiology career security and may help assure successful National Resident Matching Program results for radiology residencies.

Content

Defining appropriate content for an AI curriculum can be a daunting task given the scope and rapidly changing nature of AI in medicine. Accordingly, the creation of an AI curriculum should involve recognizing the target audience. For the purposes of this discussion, the target audience for an introductory AI curriculum consists of trainees or radiologists with limited prior experience in imaging AI. In this context, an AI curriculum should feature content areas that introduce learners to the fundamentals of machine learning (ML) and to the clinical implementation of AI. Content should be crafted from the perspective of the radiologist as a user and critical evaluator of AI tools. Specific frameworks for AI education will require institution-specific evaluation of trainee needs and available resources, building on suggested AI education curricula from prior literature (3,16). The following discussion offers key content elements for an AI curriculum:

1. Basics of ML: Understanding ML’s principal concepts and common terminology establishes a framework for advanced content, including the critical evaluation of algorithms. Example topics for discussion include ML techniques (ie, supervised, unsupervised, transfer, and federated learning), ML architecture, metrics for algorithm evaluation, the importance of standards and interoperability, and potential sources of bias in ML. This stage of the curriculum should provide an overview of the breadth of use cases with a discussion of subfields within AI, such as computer vision and natural language processing (NLP). Discussions must include data collection and curation and concerns about privacy, safety, and bias in AI models. Further technical knowledge pertaining to the creation of deep learning models may be offered to trainees with the specific intent to develop AI algorithms (ie, “tool creators”).

2. Imaging informatics: Foundational knowledge of imaging informatics provides a basis for understanding the translation of AI into clinical practice. Trainees and radiologists should understand the informatics infrastructure necessary to facilitate seamless integration and complex interaction of AI tools.

3. Governance and regulation: Radiologists need to understand the legal framework and regulatory environment to serve as stewards of AI. Important considerations include local, national, and international regulatory approval, radiology’s role in enterprise-wide AI initiatives, and AI’s risks and benefits to patients and providers, as well as continued monitoring of deployed AI tools. Changing paradigms in regulatory processes, such as U.S. Food and Drug Administration (FDA) clearance, will require routine updates with critical commentary on each iteration. Discussion should include relevant limitations, including the need for external quality and performance assessment beyond FDA clearance, which may only account for patient safety and the extent of radiologist input, and potential stepwise FDA assessment for product updates (17,18).

4. Ethics: Radiologists who evaluate and deploy AI tools in practice must understand potential biases and disparities that can harm patients. An introduction to AI ethics should aim for mindfulness of the steps in algorithm development prone to bias, ultimately allowing them to critically assess for these pitfalls before clinical implementation.

5. Economics and the “business” of AI: With a growing selection of AI tools, radiologists will need to identify those factors that justify continued investment in AI initiatives. Demonstrating return on investment will require the identification of key performance indicators (KPIs) that quantify the impact of AI tools on patient care and financial returns. Discussion should also cover suggested AI business models (such as subscription versus “per use,” bundling with other software, and the AI marketplace) and Medicare and private insurance reimbursement for AI-based services.

6. Clinical AI implementation: Radiologists will need to vet AI tools for clinical deployment. This component of the curriculum should focus on integrating AI tools into existing clinical workflow, incorporating interoperability standards, and monitoring AI algorithms in practice. Clinically relevant KPIs vary based on the AI intervention and could include the following: examination duration, examination volume after optimizing resource use, time to report a study with and without AI, time to emergent treatment, and rate of radiologist agreement with AI results. Further discussion can include the use of business intelligence tools to monitor algorithm performance and select KPIs.

Ideally, didactic content should be supplemented with experiential learning to solidify concepts while learning to troubleshoot situations encountered when working with AI in practice. Experiential learning may involve simulated exercises within virtual environments or real-time projects, depending on available resources.

An emphasis on practical content, including topics enumerated earlier in this section, is crucial, particularly as trainees assume future roles tasked with selecting among competing AI solutions for varying diagnostic tasks. Trainees confident in their ability to distinguish model performance, to assess AI purchasing considerations (ie, evaluating commercial AI solutions in radiology [or, ECLAIR] guidelines), to determine efficient integration of AI solutions (ie, independent vs picture archiving and communication system [or, PACS]–based orchestrator), and to understand potential security and data privacy risks will contribute meaningfully to their practice, arguably adding more value compared with those lacking these skills (19,20).

Should those learning about AI learn how to code? The current trend is for trainees and radiologists to collaborate with a multidisciplinary team including programmers and data scientists who are the primary coders. As with other informatics problems, successful AI implementation requires far more attention to people and processes than to technology. AI curricula that are overly technical risk intimidating learners and falsely portraying requirements for successful AI implementation. The audience of trainees and radiologists who will be primary coders is much smaller and will benefit from a formal advanced track emphasizing data science concepts, supplemented with opportunities for trial and error through hands-on coding workshops via platforms like Google Colab or Teaching Machine (6).

Continuous education beyond initial formal AI education is necessary and may take the form of hybrid fellowships in imaging informatics and imaging AI or graduate degree programs, such as a master’s degree in biomedical imaging and imaging informatics. These advanced pathways will need initial AI education, featuring the content listed in this article, as a foundation to reinforce key concepts and build more complex topics.

Barriers

As suggested by the gap between the demand for AI education and the provision of formal AI training, there are several barriers that must be addressed prior to the adoption of an AI curriculum (21):

1. Access to mentors: The availability of AI expertise varies across institutions, which can limit consistent local lectures or research involvement. Solutions include guest lectures, training programs for senior faculty to become AI educators, and participation in national AI and informatics initiatives, including the National Imaging Informatics Course – Radiology (NIIC-RAD) offered by the RSNA and SIIM. Involvement in national organizations can provide opportunities to develop mentoring relationships.

2. Limited time: Trainees’ and radiologists’ schedules are already strained with clinical duties, research endeavors, and family responsibilities. Adding AI education requires creativity to respect ongoing commitments. For trainees, solutions include elective “mini-fellowships,” dedicated supplementary residency tracks, or an AI fellowship that provides protected time for AI education. For radiologists, continuing medical education or certificate courses offer an opportunity for pursuing AI education while justifying time away from clinical duties.

3. Limited trainee interest: Trainees may not all desire education in AI, as some may be put off by unfamiliar concepts and highly technical information. Although including AI content on standardized examinations could prompt trainee interest in AI education, the addition of content to the wide range of topics already covered on resident examinations may draw ire. Alternative means of stimulating trainee interest include sessions demonstrating the practical application of AI in medical imaging, providing tangible examples that trainees can relate to in their daily workflow. Practical examples that inspire trainees can encourage the dedication of time to the pursuit of AI education.

4. Varying trainee experience: Trainees’ varying familiarity with computer science, data science, and related topics will limit the value of a one-size-fits-all approach to AI education. Accordingly, AI curricula will need to account for trainees who may benefit from more introductory lessons and those who may require advanced topics.

5. Rapid developments in AI technology: AI curricula will need to keep pace with the rapid developments and shifting paradigms in imaging AI. Educators should maintain awareness of these changes and update content to ensure that their audience can build on the latest developments impacting the practice of radiology.

Requirements

Physician leaders at local and national levels can help justify investment in an AI curriculum among competing initiatives. Faculty champions can advocate for dedicated AI lectures and protected time for supplementary tracks and research endeavors. Faculty leaders can spark trainees’ interest in AI by demonstrating its future role in radiology practice and by emphasizing how AI-savvy trainees can help shape the practice of radiology and standards of care.

Justifying the renewal of AI education will require an assessment of curriculum success. Inclusion of AI topics on standardized examinations, or offering specific AI “qualifying” examinations, could provide an objective measure of AI curriculum effectiveness. However, formal testing of AI topics would require a standardized AI curriculum to ensure that all test takers meet baseline eligibility for assessment. Alternatively, assessing the clinical impact of trainee AI endeavors may offer an indirect means of assessing trainee AI competency.

Resources

Fortunately, collaborations between national organizations and course offerings from educational platforms actively work to close the gap between the demand for AI education and the provision of learning tools.

Several radiology societies offer structured, self-guided courses for standardized education in informatics and AI (Table 1). These courses offer core content intended as an introduction to establish baseline proficiency for those without extensive prior experience. Specifically, the NIIC-RAD course provides a primer on the fundamentals of imaging informatics that support AI tools in radiology workflow. The course’s hybrid format allows for self-guided learning with a “flipped classroom” approach to didactics, allowing for conversations with experts in the field of imaging informatics. The course offers financial aid and options for bundled pricing to increase access to the course. The RSNA Imaging AI Certificate provides structured education to develop competency in working with AI tools in practice. The course offers on-demand lectures and hands-on exercises for the evaluation, deployment, and monitoring of AI tools.

Table 1:

Key Introductory Educational Resources on Imaging Informatics and AI from Major Radiology Societies

Table 2 describes additional resources for AI education, including formal programming courses, video series from leading organizations, and blogs from AI and informatics communities. The Magician’s Corner, published in Radiology: Artificial Intelligence, is a notable highlight for self-directed learning (22). This series distills complex ML topics into concise articles designed for those starting out in AI education and algorithm development.

Table 2:

Selected Supplemental AI and Informatics Education Resources

Trainees at academic institutions may have easier access to local informatics and AI expertise, with potentially abundant opportunities for on-site learning or research involvement. In contrast, trainees at community centers lacking in local informatics and AI expertise may benefit from web-based materials, such as those listed in Tables 1 and 2. Web-based educational resources, such as the NIIC-RAD course, provide opportunities for flexible learning and networking with mentors. Trainees relying on web-based materials should seek support from their training programs for dedicated time and financial support to participate in courses that eliminate geographic barriers in order to bring trainees and experts together. Establishing relationships with other trainees and instructors through these programs facilitates future collaboration for personal growth and guidance for nascent local informatics initiatives.

AI-augmented Education

We envision a future where AI enables precision education, allowing for personalized experiences tailored to trainee learning styles and accounting for specific limitations in clinical exposure. The breadth and volume of available cases vary within a given training program, predisposing to trainee deficiency in content exposure. Leveraging advances in NLP models can augment endeavors to address these deficiencies by directly assigning cases and curating comprehensive teaching files on the basis of feedback from trainee-specific case logs (7). For example, NLP tools can screen new patient examinations and assign them to worklists on the basis of trainee-specific case history to ensure that trainees are equally exposed to varying imaging modalities and radiologic subspecialty examinations. AI-augmented education can also leverage NLP tools to query institutional archives for cases with similar diagnoses or clinical history to further supplement deficient content areas. Furthermore, automated curation of cases can support trainee simulations for experiences such as independent call, allowing trainees to build confidence in a controlled setting while identifying areas for further improvement (7). Computer vision algorithms can also aid radiology education by automating tedious tasks, such as lesion measurement, and by flagging abnormal findings for junior residents still learning characteristic imaging features. From the perspective of education administration, AI tools and business intelligence solutions can monitor trainee milestones, allowing for active supervision and early intervention for potential gaps in proficiency or case/procedure volume.

The advent of generative adversarial networks (GANs) in imaging offers another opportunity to leverage AI for radiology education. Image synthesis enabled by GANs provides a means of simulating pathologic conditions in resource-constrained regions (23). Training programs with a dearth of subspecialty cases may use GANs to synthesize cases derived from the few examples available in clinical practice. A similar approach can support global health initiatives endeavoring to train radiologists without access to diverse, local cases. Furthermore, this approach leveraging GANs may be combined with NLP models to analyze specific patterns of reporting errors or subspecialty underexposure to augment resident case exposure with targeted, synthetic cases.

Though AI-augmented education holds great potential for improving radiology training, caution is warranted to ensure trainees evade pitfalls that limit their interpretive abilities. Junior trainees without substantial prior radiology experience may become dependent on computer vision tools, limiting their ability to evaluate studies without initial AI interpretations. Trainees using AI early in their experience may also be prone to accepting false-positive and/or false-negative AI results, while those with more experience may correctly identify evidence arguing against false AI results. Residency programs may choose to introduce AI tools later in radiology training to first allow for the development of robust interpretive skills, followed by AI-augmented interpretation to aid workflow efficiency during more senior experiences, including call shifts. Additionally, training programs using AI applications must include instruction on biases associated with using AI for diagnostic imaging, such as automation bias, to ensure that trainees learn to be critical of AI tools and avoid interpretive pitfalls that can lead to patient harm. While AI tools for radiology education enable “precision education,” there must be a balance between AI-augmented learning and critical evaluation of AI tools in practice.

Conclusion

AI’s burgeoning influence on the practice of radiology presents a need for education to ensure radiologists maintain proficiency and stewardship in the clinical use of this technology. Establishing frameworks for AI education requires a collaborative effort from trainees, radiology leadership, data scientists, informatics professionals, and national organizations to offer holistic curricula and deploy solutions for AI-augmented education. However, a one-size-fits-all approach is unlikely, and AI education initiatives will have to account for barriers specific to their target audience. Fortunately, both formal and self-directed learning resources offer solutions to close the gap between the demand for AI education and the provision of vetted content. Endeavors for AI education will establish precedents for radiology training, allowing future generations to continue building on flexible frameworks to reflect changing paradigms in imaging AI.