Overcoming the Challenges in the Development and Implementation of Artificial Intelligence in Radiology: A Comprehensive Review of Solutions Beyond Supervised Learning

Abstract

Artificial intelligence (AI) in radiology is a rapidly developing field with several prospective clinical studies demonstrating its benefits in clinical practice. In 2022, the Korean Society of Radiology held a forum to discuss the challenges and drawbacks in AI development and implementation. Various barriers hinder the successful application and widespread adoption of AI in radiology, such as limited annotated data, data privacy and security, data heterogeneity, imbalanced data, model interpretability, overfitting, and integration with clinical workflows. In this review, some of the various possible solutions to these challenges are presented and discussed; these include training with longitudinal and multimodal datasets, dense training with multitask learning and multimodal learning, self-supervised contrastive learning, various image modifications and syntheses using generative models, explainable AI, causal learning, federated learning with large data models, and digital twins.

INTRODUCTION

Deep learning (DL), a type of artificial intelligence (AI), is increasingly used in radiology and clinical practice [1]. The effectiveness of DL methods in classifying images and detecting pathological lesions in a supervised manner has been successfully demonstrated in various fields of medicine [2,3,4,5,6,7,8,9,10,11]. Numerous studies have focused on developing and validating DL models in radiology, which includes neurological, thoracic, abdominal, and breast imaging. With an accuracy comparable to that of human experts, convolutional neural networks (CNNs) [12,13] and transformer networks [14] typically classify images based on the presence or absence of disease. In addition, from previously diagnosed lesions, CNNs can extract information regarding morphological characteristics of the lesion, such as size, shape, or outline, as in the case of breast lesions or chest nodules [15,16,17,18]. Therefore, many computer-aided diagnosis (CADx) and detection (CADe) software that utilize DL methods have received approval from the Food and Drug Administration (FDA) and have been successfully commercialized [19]; moreover, the software can offer other advantages, including speed, efficiency, low cost, increased accessibility, and the upkeep of faithful behavior in clinical practice. In prospective studies, early-career radiologists and physicians have noted performance improvements using DL models [20,21,22,23]. Despite the promising results of current AI technology, numerous challenges exist in the development and deployment of supervised learning-based AI systems. First, a major obstacle is the requirement of a huge volume of high-quality medical imaging data to achieve good AI performance [24]. This demanding requirement can be attributed to the high-dimensional nature of the data and the presence of noise and irrelevant information [25], which pose challenges in extracting high-level features. Even expert physicians may encounter difficulties in labeling data with high quality owing to the high cost involved, occasional ambiguity in the ground truth information, tedious nature of the labeling task, and risks to patient privacy. Secondly, although numerous radiologic tasks are performed in actual clinical settings, most radiologic applications are designed for specific clinical situations, and consequently, broad-range radiologic tasks are overlooked. Third, DL models often overfit training data, leading to poor performance when applied to new data [26,27]. Specifically, variations in imaging protocols, manufacturers, and patient populations can lead to significant variability in the appearance and semantics of images [24], making it difficult to develop robust models in response to these variations. In general, most radiology datasets are long-tailed, which can lead to underrepresented classes and biases in the trained model [28]. Fourth, DL models are often considered as “black boxes” [29] regarding model interpretability, which may negatively affect the use of AI in clinical decision-making. Radiology is a rapidly evolving field, and DL models require frequent retraining to keep pace with the latest advancements in imaging techniques and new types of data [24]. This review presents and discusses various potential solutions for addressing the challenges of supervised AI.

A Quick Primer before We Start

Representation Learning

The concepts of representation learning, autoencoder, encoder, decoder, and latent space are presented in Figure 1 using an example. Representation learning allows a model to automatically map the representation space (z) needed for detection or classification of raw data (x). z is a vector in the latent space with smaller dimensions than the input (x). In this example, the model was trained to differentiate between the X-ray images of different body parts, including those of the head, chest, abdomen, spine, pelvis, and upper and lower extremities. Therefore, after training, the model can be used as an encoder to map the input modality (X-ray images, denoted as x) into different latent spaces, where the locations represent the X-ray images of different body parts (Fig. 1A). An early example of representation learning is auto-encoder. An autoencoder is composed of two components—an encoder and a decoder—and can be used to generate synthetic data. For instance, the encoder maps the input modality into the latent vector (z), and then, based on the vector z, the decoder generates a synthetic sample X-ray image of the input body part (Fig. 1B) [30].

Fig. 1. A conceptual diagram of representation learning and auto-encoder. A: Representation learning implies training an encoder to automatically map into the representations space, z, needed for detection or classification from input modality (e.g., X-ray images), x, where z is a vector in the latent space with smaller dimensions than those of input modality, x. This model was trained to differentiate X-ray images of different body parts, including those of the head, chest, abdomen, spine, pelvis, and upper and lower extremities. B: Auto-encoder is mainly composed of two components: an encoder and a decoder. The encoder maps the input modality into a latent vector, z, and then, based on the vector z, the decoder generates a novel sample of the target modality.

Generative Models

Various types of generative models exist, such as generative adversarial networks (GANs), variational autoencoders (VAEs), flow-based models, and diffusion models, which are a class of machine learning algorithms that aim to learn the underlying data distribution of a given dataset, enabling them to generate new data samples that resemble the original data (Fig. 2) [31,32,33,34,35,36]. Each model is based on different theories—such as adversarial training [31,32,36], maximization of the variational lower bound [35], invertible transformation of distribution [34], and mimicking the Markov chain of diffusion steps to slowly add random noise to the data—and then the models learn to reverse the diffusion process to construct the desired data samples from the noise [33]. Among them, the GAN and diffusion models have been successfully applied in radiology research, such as image denoising [37,38,39], image reconstruction [40,41,42], intermodality image synthesis [43,44,45], improved image segmentation [46,47,48,49], image registration [50,51], classification [52,53], anomaly detection [54,55,56,57,58,59,60,61], and disease progression modeling [62].

Fig. 2. Typical examples of generative models include generative adversarial networks, variational auto-encoder, flow-based models, and diffusion models with different theories—such as adversarial training, maximization of the variational lower bound, invertible transformation of distribution, and mimicking the Markov chain of the diffusion steps to slowly add random noise to data—and then the models learn to reverse the diffusion process to construct desired data samples from the noise.

Overcoming the Challenges

Longitudinal and Multi-Modal Dataset

In real-world radiology settings, images are acquired via multiple modalities and time points that can be combined and analyzed. For example, a multiscale and multimodal deep neural network classifier built with a combination of fluorodeoxyglucose-positron emission tomography (FDG-PET) and structural magnetic resonance imaging (MRI) showed performance improvement [63]. Moreover, a disease progression model using longitudinal data with varying time intervals can improve robustness against missing data and performance in Alzheimer’s disease [64]. Multiple follow-ups with repeated measurements are the norm in clinical practice, thereby affecting an increase in the statistical power [65,66]. In the event of incomplete data and/or limited knowledge of the disease and pathogenesis, multimodal data, such as various types of imaging modalities, demographics, laboratory tests, and electronic medical records (EMRs), may prove crucial to achieve an improved diagnosis [67,68]. Therefore, for better applicability in clinical settings, longitudinal and multimodal datasets should be considered when training AI-based models. However, it is important to acknowledge that although these datasets can enhance model performance, they do not eliminate the need to address the inherent challenges of DL models, such as handling long-tailed distributions and improving model explainability. In addition, the utilization of these datasets can present significant risks to patient privacy. Therefore, it is crucial to utilize these datasets in conjunction with the following strategies aimed at addressing the fundamental challenges of DL models.

Dense Training with Multi-Task Learning and Multi-Modal Learning

Dense training with multitask learning (MTL) and multimodal learning offers several notable advantages, particularly in the field of radiology. Multimodal learning enables the integration of diverse data sources such as clinical information, imaging modalities, and patient demographics, potentially enhancing the overall predictive power of AI models [69,70,71,72,73,74,75]. By collectively incorporating multiple factors, the accuracy and reliability of the diagnoses can be improved. Furthermore, joint training of models on various tasks using MTL can facilitate the learning of shared representations and leverage correlations among different modalities, which may contribute to improved performance across multiple domains [76]. Additionally, dense training with MTL can offer regularization effects, which helps to prevent overfitting and potentially enhances the generalization capability of the model [77], thereby encouraging the network to learn relevant and robust features simultaneously for multiple tasks, which may translate into better performance on individual tasks. Moreover, this approach may improve the interpretability and explainability. When models are trained to perform multiple tasks, they are forced to capture meaningful and discriminative features, potentially enhancing the transparency of the decision-making processes. Such transparency can be valuable in building trust among radiologists and other healthcare professionals, as it enables them to understand and interpret the underlying factors contributing to the model’s predictions. Recently, different MTL architectures have emerged, and based on where information or features are exchanged within a network, encoder-focused [78,79,80,81] and decoder-focused [82,83,84] designs, among others, are utilized. Many studies have focused on AI in radiology because of the complexities involved in training owing to high-dimensional data and the limitations of data acquisition. These radiology studies demonstrated the superiority of the MTL over other models; for instance, MTL has shown better performance than alternative approaches in various tasks, such as segmenting thoracic organs from computed tomography (CT) slices [85]; object detection, segmentation, and classification in breast cancer diagnosis using full-field digital mammogram datasets [86]; identification and segmentation of coronavirus disease 2019 (COVID-19) lesions from chest CT images [87]; and identification of hemorrhage and segmentation in head and neck CT images [88] (Fig. 3). However, the extent to which multimodality enhances the robustness to distribution shifts, promotes patient privacy, and addresses other limitations remains a topic of ongoing research.

Fig. 3. A typical case of hemorrhage detection with multi-task learning in head and neck computed tomography images, including classification, segmentation, and reconstruction with consistency loss. Modified from Kyung et al. Med Image Anal 2022;81:102489 [88], under the permission from Elsevier. SEG = segmentation, REG = registration, ICH = intracerebral hemorrhage.

Vision-language (VL) multimodal learning—a prominent area of multimodal research that utilizes radiographic images and their associated free-text descriptions, such as chest radiographs (CXRs) and their reports—is becoming increasingly significant [12,89] (Fig. 4). Images and paired radiology reports provide mutually beneficial semantic information to enhance the training quality of various tasks such as diagnosis classification and report generation in radiology. However, significant technical challenges are encountered in processing both images and clinical reports to learn joint representations. These challenges arise from differences in the high dimensionality, heterogeneity, and systemic biases of multimodal datasets. Recent progress in VL multimodal learning was achieved by extending bidirectional encoder representations from transformer-based architectures (BERT) [90]. Previous studies can be broadly categorized into vision-language understanding (VLU) tasks, which include visual question answering, text-conditioned image retrieval, diagnosis classification, and vision-language generation (VLG) tasks such as image captioning and report generation [91,92]. Numerous studies have explored the applications of VL multimodal learning in radiology. Hsu et al. [93] focused on a VLU task, specifically, image report retrieval, using both supervised and unsupervised methods. Liu et al. [94] focused solely on the VLG task of radiology reports by implementing a CNN-recurrent neural network (RNN) architecture and a hierarchical generation strategy. Liu et al. [95] proposed a transformer encoder-decoder-based approach that uses prior and posterior knowledge of distillation techniques using different modalities. Wang et al. [96] introduced a self-boosting framework with various modules using images and reports based on the collaboration of a primary generation task and an auxiliary image-report matching task. Yang et al. [97] developed MedWriter, which integrates a hierarchical retrieval mechanism to automatically extract reports and sentence-level templates. Recently, large language models (LLMs), such as BERT [90] and generative pre-trained transformer (GPT) [98], have been used to embed high-level semantics or generate expert-level reports in the near future.

Fig. 4. Multi-modal learning using chest X-ray images and clinical reports for visual language understanding or visual language generation tasks. Modified from Moon et al. IEEE J Biomed Health Inform 2022;26:6070-6080 [91], under the permission from IEEE.

Self-supervised Learning with Contrastive Learning as a Foundation Model

A foundation model is any model trained on a broad dataset, typically using large-scale self-supervision. It can consolidate information from various modalities and can subsequently be adapted, for example, through fine-tuning, for a wide range of downstream tasks. Examples of foundation models include the BERT, GPT, and contrastive language–image pre-training (CLIP) [99].

In general, contrastive learning (CL) is an approach used to learn representations in an unsupervised or self-supervised manner with the aim of bringing similar samples closer together while pushing dissimilar ones farther apart in the latent space (Fig. 5A). Automatic labeling without human effort is one of the most powerful approaches to unsupervised or self-supervised CL [100,101]. In medicine, CL can help train an encoder with large-scale medical images or other data, particularly in a self-supervised learning (SSL) manner, because SSL can automatically generate pseudo-labels based on different augmentations of the same or other images without human-annotated labels (Fig. 5B). The encoder can help differentiate between healthy and unhealthy individuals or identify specific diseases or conditions using transfer learning with a relatively small amount of annotated data. This approach is particularly useful for applications, such as CAD and image segmentation, which can assist radiologists in disease diagnosis by analyzing medical images and providing a list of potential diagnoses [102,103].

Fig. 5. A typical example of contrastive learning of supervised and self-supervised learning. A: Supervised contrastive learning with positive pairs of follow-up images of the same patient (red dots) and negative pairs of different patients (blue dots). B: Two different augmentations of medical images are randomly performed. Their embedding vectors are obtained through an image encoder to encourage a model to learn similar representations in the same class and dissimilar representations in different classes with similarity loss in a self-supervised manner.

Using CL, a neural network can identify abnormal brain structures on MRI scans, which can aid in the diagnosis of Alzheimer’s disease, differentiate between different types of images even in the case of image overlap, and improve the accuracy and efficiency of diagnosis by reducing the amount of time and effort required for manual interpretation of images [104]. This technique can aid in the early detection of diseases and provide timely treatment to patients. For medical image segmentation, to separate and identify different structures or tissues within an image, CL can be used to accurately detect segment-specific structures, such as the heart, head, and neck, in CT scans, thereby improving the accuracy of measurements and identifying abnormalities [105,106]. Additionally, contrastive unpaired image translation [107] can be used to synthesize pediatric CXRs, which can help improve the diagnosis of diseases within specific structures based on CT images [108]. Overall, CL has shown great promise as a tool for improving the accuracy and reliability of image analysis and diagnosis in a variety of medical settings.

Denoising, Fast Image Reconstruction, Inter-Modality Synthesis, and Synthetic Data Generation Using Generative Models

In radiology, the use of generative models have shown promising results, improving various aspects such as the image quality, reduction of radiation exposure, and shortened acquisition times. GANs have been particularly effective in CT applications such as denoising [37,38] and metallic artifact removal [39]. In addition, GANs have been explored as a means for artifact correction in accurate radiation therapy planning and as an image guide in cone-beam CT images [109,110]. In MRI, DL techniques have been employed to accelerate image acquisition and improve the image quality. Notably, GANs have demonstrated their superiority over traditional methods, such as compressed sensing MRI, in terms of faster reconstruction and enhanced image quality [40,41,42]. GANs have also been utilized to convert low-magnetic-field MRI into high-magnetic-field MRIs [111], enabling the generation of high-quality images, even with lower magnetic field strengths. This conversion has the potential to reduce the cost and complexity of the MRI systems. Generative models that facilitate the intermodality synthesis offer several advantages. This approach streamlines imaging workups, reduces radiation exposure, and lowers costs by synthesizing images from safer and more cost-effective alternatives such as ultrasound or MRI [44,112,113]. Moreover, intermodality synthesis enhances model performance by enabling training on multiple modalities [114] and fills gaps by imputing missing sequence data [115]. Synthetic CT images generated from MRI data can be used for attenuation correction of PET images [45]. In addition, the use of synthetic data to balance medical datasets is widely recognized as a means of enhancing the performance of models in detecting, segmenting, and predicting medical conditions [116,117,118,119]. However, it is crucial to assess whether synthetic samples accurately capture the complexities and variations in real-world medical data. Further evidence is required to establish the substitutability of synthetic data for real data.

Anomaly Detection Using Generative Models

Anomaly detection—a technical term implying the identification of data points in data that do not fit normal patterns—is gaining attention in radiology, because it can identify abnormal lesions on images by learning the data distribution of normal images in semi-supervised learning [120,121] (Fig. 6). It is noteworthy that while anomaly detection can potentially encompass non-pathological findings, such as congenital anomalies, the term is generally used in the medical field to specifically refer to the detection of pathological or clinically significant abnormalities. Therefore, anomaly detection in radiology focuses primarily on identifying abnormal findings that are clinically relevant and may require further investigation or intervention. These algorithms offer several advantages that make them an attractive option, because anomaly detection can be trained without the need for high-quality annotated data, which can be time-consuming and expensive to obtain. This makes anomaly detection a more efficient alternative to supervised learning methods applied to rare or less frequently occurring diseases, which have not been collected enough to train a model in a supervised manner. This approach makes them more clinically applicable, as it helps to detect several conditions rather than just focusing on one or a few specific diseases. Anomaly detection in medical imaging can be approached using various training methods, including supervised, semi-, and un-supervised learning, with several popular network architectures, such as VAEs [122,123,124,125,126,127,128,129,130,131], GANs [55,56,57,58,59,60,61], and diffusion models [54]. Because GANs have the potential to generate high-resolution medical images, GAN-based anomaly detection can reconstruct the closest normal image to a pathologic input image by leveraging normal data from healthy individuals. The reconstruction error between the input (real images) and output (synthetic images) helps to evaluate the anomaly scores for the input data and enables the identification of pathologic anomalies. In the field of radiology, anomaly detection models have been used in various modalities, including CXRs [55,56,122,132,133], mammography [123,124,134], breast ultrasound [57], cardiac CT [135], brain PET-CT [136], brain CT [58,125,126], and brain MRI [59,60,61,127,128,129,130,131,137]. Notably, these studies have proposed the potential utility of anomaly detection for various medical images.

Fig. 6. Anomaly detection with an encoder to map images into a latent space (z), with the data (blue dots) classified as normal or abnormal boundary (red dashed line).

Better Explainability and Validation of Artificial Intelligence

DL networks have remarkably improved their performance with multiple hidden layers of non-linear activation functions; however, this occurs at the cost of interpretability and explainability, thereby aptly earning the nickname of “Black box” [138]. For adequate patient communication and rapport, providing the reasons for decision making in medicine using AI is important. The European Union’s General Data Protection Regulation recommends that automated individual decision-making should be based on the explicit consent of the patient, and meaningful information about the algorithms should be provided to them [139]. However, one atudy [140] pointed out that there can be a “level of opacity” in explainability (i.e., whether the explanations cannot be shared according to the nature of the algorithm or the explanations will not be shared by the developer). Contacting all the authors of individual DL studies and asking them to propose a clear mechanism of action is infeasible and impractical. As the mechanisms underlying some medical discoveries are unclear, further studies are needed to confirm their safety and utility (e.g., the mechanism of action of metformin has not yet been clearly revealed [141], but its safety and utility have been confirmed [142]). Nevertheless, there is still an increasing demand for the explainability of the “black box” in medicine [143].

Medical AI researchers attempt to explain the performance of their models in many ways. Traditional machine learning methods, such as linear regression, support vector machines [144], and tree-based models [145,146], are explainable models. Some algorithms provide textual explanations directly, such as medical visual question answering [147]. However, most researchers in medical DL favor visual explanations such as class activation maps (CAMs) [148] and gradient-weighted CAM (Grad-CAM) [149]. In addition, a local interpretable model-agnostic explanation (LIME) [150] method can be applied to explain the results visually. A review article [151] has shown an increasing trend in the use of explainable AI (XAI) in medical image analyses.

The performance of medical DL should be validated and reported using a more standardized study design, regardless of whether the model is explainable. In particular, handling confounding variables to isolate causality and associations is important in medical research [152,153,154]. Confounding variables should be considered when developing and validating medical DL models. However, exploiting these important confounding variables while developing the DL model can induce shortcut problems [155], because such confounding variables may already be reflected in the results as radiomic features. The identification and exploitation of these shortcuts should be considered when developing robust medical DL models [156]. Nevertheless, exploiting confounding variables to develop DL models and handling them in statistical models may induce overadjustment issues [157]. In addition, factors such as anatomical side markers and image quality can act as confounding variables, which may negatively affect the generalizability and credibility of the research [158]. Furthermore, similar to the comparison of intention-to-treat and per-protocol analyses in clinical research [159], a comparative study on exploitation vs. no exploitation of confounding variables can provide better perspectives for research on DL in medical image analysis. Reporting guidelines for studies on AI in medicine have been developed and registered in the EQUATOR (Enhancing the QUAlity and Transparency Of health Research) Network Library [160,161]. These guidelines include the Consolidated Standards of Reporting Trials–Artificial Intelligence (CONSORT–AI), Standard Protocol Items: Recommendations for Interventional Trials–Artificial Intelligence (SPIRIT–AI), Standards for Reporting of Diagnostic Accuracy Studies–Artificial Intelligence (STARD–AI), Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis–Artificial Intelligence (TRIPOD–AI), Developmental and Exploratory Clinical Investigation of DEcision-support systems driven by Artificial Intelligence (DECIDE-AI), and Clinical Artificial Intelligence Model (CLAIM). These guidelines aimed to improve the transparency, quality, and reproducibility of research in the field of AI in medicine. A recent article pointed out that most research on the use of AI in medical imaging lacks standardized study designs appropriate for real-world clinical settings [162], whereas another study put forth relevant methods for evaluating the clinical performance of medical DL [163].

Causal Learning

Causal learning is an emerging field of machine learning that focuses on the causal relationships between variables in a system [164,165]. The goal of causal learning is to understand the underlying mechanisms governing a system and predict how changes in one variable affect another. Recently, causally-enabled methods for medical analysis that use imaging data have garnered increasing interest [166]. These methods aim to address the challenges in medical imaging using causal inference to better understand and analyze data. Wang et al. [166] proposed a normalizing flow-based causal model, similar to the study by Pawlowski et al. [167] to harmonize heterogeneous medical data. This method was applied to T1 brain MRI to classify Alzheimer's disease to infer counterfactuals, which were then used to harmonize medical data. Pölsterl et al. [168] circumvented the identifiability condition, which states that all confounders should be known; they leveraged the dependencies between causes to determine substitute confounders. This method has been applied in brain neuroimaging to detect Alzheimer’s disease. Zhuang et al. [169] also proposed an alternative to expectation maximization for the dynamic causal modeling of functional magnetic resonance imaging brain scans. They developed an approach based on a multiple-shooting method to estimate the parameters of ordinary differential equations under the noisy observations required for brain causal modeling. They suggested augmentation of the multiple-shooting adjoint method to calculate the loss and gradients of their model. Clivio et al. [170] proposed a neural score-matching method for causal inference in high-dimensional medical images to avoid preprocessing them into a lower-dimensional latent space. da Silva et al. [171] used a generative model to synthesize MR images of brain atrophy to examine and investigate various hypotheses regarding the causes of brain growth and atrophy. Overall, these studies demonstrated the potential of causally enabled methods for medical analysis that utilize imaging data. Causal learning can provide novel insights into the obstacles in machine learning for medical imaging, including the dearth of high-quality annotated data and the mismatch between the training and target datasets. Further research in this field could lead to novel and improved methods for analyzing medical images and provide a better understanding of the underlying causes of diseases.

Federated Learning for Overcoming Privacy Concerns

Federated learning presents a promising solution for training models with large and diverse datasets, while preserving patient privacy [172,173]. Enabling multisite collaboration improves generalizability and fosters research on rare diseases that require collective effort for data curation. In federated learning, the weights and parameters of the global model are sent to the different participating institutions. Each institution trains the local AI models using local data and updates the model weights accordingly. After completion of local training, these updated weights are sent back to a central server to coordinate a global AI model. This procedure is repeated until the global model achieves a specific target (Fig. 7). Despite its advantages, federated learning in clinical settings faces several challenges in effective implementation [174]. The primary obstacle is data heterogeneity in each institution. Variations in data types, formats, acquisition protocols, annotation formats, and terminologies across institutions can hinder effective collaboration. Standardization agreements among the institutions involved can help overcome this challenge. Another challenge is technical limitations, such as different computational infrastructures across institutions. Federated learning is a fundamental technique in the future of digital health. Further research and development in federated learning and a concentrated effort in addressing these challenges will contribute to unlocking its full potential in radiology and medical imaging.

Fig. 7. A conceptual diagram of federated learning with training global and local artificial intelligence (AI) models. The weights and parameters of the global model were sent to the different participating institutions. Each institution trains local AI models using local data and updates the model weights accordingly. After completion of local training, these updated weights are sent back to a central server to coordinate a global AI model. This procedure is repeated until the global model achieves a specific target.

Additional Paradigms

Predictive Imaging Biomarkers

Several studies have focused on the extraction of predictive biomarkers from medical images using DL. The development of predictive imaging biomarkers from medical imaging can be approached using various training methods, including supervised, semi-, self-, and un-supervised learning, with several popular network architectures such as CNNs, VAEs, and GANs. Some studies [175,176,177] showed that DL can extract prognostic information from medical images. As these images contain age and sex information, DL-based models can learn demographic information [178,179,180,181]. Studies have shown that neuroimaging-derived age prediction corresponds to the influences of other disorders, such as cognitive impairment and Alzheimer’s disease, and age and sex per se [182,183,184], which have been explored in several brain illnesses. The disparities between predicted and chronological ages may be attributed to the accumulation of age-related alterations in pathological circumstances [185,186,187] or protective factors in brain aging [188,189]. Furthermore, CXR-derived age can be used as an imaging biomarker to indicate the state of the thorax or metabolism [175,176,177] and successfully predict lifespan, mortality, cardiovascular risk, and heart failure prognosis [175,177,190], providing a solid foundation for the imaging biomarker concept. Another approach for extracting relevant prognostic imaging biomarkers involves training a deep survival model using staged binary classifiers of death or incident cancer [175,191,192].

Digital Twin

As simulations have become more prevalent in medicine, the future of precision medicine includes providing tailored diagnoses and treatments for each patient through the development of digital twin (DT) technology. Recently, several studies on DTs in medicine have been published [193,194,195,196,197,198,199,200,201,202], including a wide range of applications such as healthcare management [193], fitness [194], simulating viral infections [195], smart city well-being [196], remote surgery [197], and cardiovascular diseases [200,201,202].

DTs based on image data have significant potential for revolutionizing radiology. These virtual replicas provide radiologists with a comprehensive and dynamic representation of a patient’s anatomy, physiology, and pathology, thereby enabling improved treatment planning and decision-making. By leveraging a patient's radiological imaging data, such as personalized anatomy and imaging biomarkers, DTs offer several benefits in various medical fields. For instance, in orthopedic surgery, DTs created using CT or MRI scans allow surgeons to simulate surgery in advance [203,204]. This enables them to tailor the procedure to the patient's unique anatomy, potentially improving surgical outcomes. By visualizing the joint in a virtual environment, surgeons can plan surgery more accurately, assess different approaches, and anticipate potential complications. In cancer treatment planning, DTs created from a patient's radiological imaging data, such as MRI, PET, or CT scans, can provide a detailed representation of the anatomical and biological characteristics of a tumor [205,206]. This allows medical professionals to test different treatment modalities, such as radiation therapy or chemotherapy, for DTs. By simulating the effects of various treatments, DTs can be used to predict potential effectiveness and side effects. This information aids in the selection of the most suitable and least harmful treatment plan for patients, thereby optimizing the chances of successful outcomes. Similarly, the DTs of a patient's cardiovascular system can be created based on imaging data obtained from angiography, echocardiography, or MRIs, among others [207,208,209]. These DTs can predict the progression of conditions such as atherosclerosis or the risk of events such as myocardial infarctions. They can also be utilized to simulate the effects of interventions, such as stent placement, enabling doctors to plan surgeries more effectively and make informed decisions regarding patient care.

Additionally, DTs offer potential solutions to existing challenges in the AI field. They enable the generation of synthetic yet realistic data through simulations, the augmentation of real-world data, and the enhancement of the robustness of AI training. By incorporating DTs into AI models, the predictive capabilities can be enhanced, because these twins have the potential to dynamically simulate future outcomes based on changing conditions. Furthermore, DTs are based on established physics and human anatomy and provide a more interpretable framework for AI models. This interpretability fosters trust and understanding among healthcare professionals, facilitating AI-driven decision-making.

SUMMARY AND CONCLUSION

Many CADx and CADe software packages with DL methods have received clearance from regulatory agencies and have been successfully commercialized; they offer advantages, including speed, efficiency, low cost, increased accessibility, and the upkeep of faithful behavior in clinical practice. Despite these achievements, several challenges must be addressed in the deployment and development of AI in healthcare; these include ensuring patient privacy, obtaining access to large volumes of high-quality data, achieving generalizability of AI models, and establishing the explainability of the decision-making process. To overcome these issues, some of the various possible solutions are as follows: inclusion of training with longitudinal and multimodal datasets, dense training with multitask and multimodal learning, new generative models including anomaly detection, XAI, self-supervised contrastive and federated learning with large-scale data, causal learning, and DTs. For instance, MTL allows a single AI model to perform multiple tasks simultaneously, thereby increasing its efficiency and adaptability. DL can be used with multi-modality (different types of data) and longitudinal data (data collected over time) to enhance AI's capacity to handle complex medical data. This helps to reduce false positives, leading to more accurate diagnoses and treatments.

Recently spotlighted techniques, such as generative models and federated learning, can provide excellent strategies for improving healthcare outcomes while maintaining patient privacy and data security. Synthetic medical data generated using generative models, such as GANs, can create realistic data samples without compromising patient privacy. Federated learning also allows multiple organizations to collaboratively train a shared AI model without directly sharing sensitive patient data and ensures privacy while still benefiting from a larger, more diverse dataset. In addition, generative models can significantly contribute to radiology by improving various aspects of image processing. These models can aid in fast image acquisition, robust denoising (reducing noise or artifacts), and image reconstruction, resulting in higher-quality images for analysis. Furthermore, generative models can be applied to other research areas in radiology, thereby expanding the possibilities of AI-driven advancements. Semi-supervised and unsupervised learning approaches can save time in AI development and enable broader coverage of various diseases. Specifically, anomaly detection using semisupervised learning can help identify unusual patterns in data and facilitate early diagnosis and intervention for patients. Finally, XAI focuses on developing AI models that provide understandable explanations of predictions and decisions. This approach is crucial for building trust of medical professionals in implementation of AI in clinical practice, to improve their understanding, and to allow the validation of AI-driven recommendations. By enhancing trust and transparency, XAI can accelerate DL research in the medical field, thereby paving the way for more effective and widely adopted solutions. DTs and virtual replicas of real-world objects, systems, or processes can be used to simulate patient conditions, predict treatment outcomes, and optimize clinical decision-making.

More robust technological advancements suggest that significant changes can occur in radiology. Nonetheless, with the integration of better AI into medical imaging, we envision these technological breakthroughs as better cooperative mechanisms designed to alleviate workload and minimize distractions. Currently, DL is used in radiology during the early stages of adolescence. LLMs, such as GPT v4, have recently created a strong impact. However, a crucial factor for AI development and clinical integration into radiology is fostering a comprehensive understanding of the technology, radiology practice, and clinical workflow. Furthermore, the active participation of radiologists, scientists, engineers, and commercials is vital for creating a wide range of radiological applications.