Interventional Radiology ex-machina: impact of Artificial Intelligence on practice

Abstract

Artificial intelligence (AI) is a branch of Informatics that uses algorithms to tirelessly process data, understand its meaning and provide the desired outcome, continuously redefining its logic. AI was mainly introduced via artificial neural networks, developed in the early 1950s, and with its evolution into "computational learning models." Machine Learning analyzes and extracts features in larger data after exposure to examples; Deep Learning uses neural networks in order to extract meaningful patterns from imaging data, even deciphering that which would otherwise be beyond human perception. Thus, AI has the potential to revolutionize the healthcare systems and clinical practice of doctors all over the world. This is especially true for radiologists, who are integral to diagnostic medicine, helping to customize treatments and triage resources with maximum effectiveness. Related in spirit to Artificial intelligence are Augmented Reality, mixed reality, or Virtual Reality, which are able to enhance accuracy of minimally invasive treatments in image guided therapies by Interventional Radiologists. The potential applications of AI in IR go beyond computer vision and diagnosis, to include screening and modeling of patient selection, predictive tools for treatment planning and navigation, and training tools. Although no new technology is widely embraced, AI may provide opportunities to enhance radiology service and improve patient care, if studied, validated, and applied appropriately.

Introduction

Artificial intelligence (AI) is a methodology of computer engineering that uses algorithms to tirelessly process data, understand its meaning and provide the desired outcome, continuously redefining its logic and selecting the optimal pathway to address the applied question.

Although its official birth is referred to as in 1955, when John McCarthy first defined it as “the science of making intelligent machines” [1], AI may have its roots with the invention of robots, whose first mention can be traced back to the third century in China. Over a millennium later, it was Leonardo Da Vinci, during the Renaissance, who made a detailed study of human anatomy and designed his humanoid robot. His sketches, rediscovered only in the 1950s, were in fact, a source of inspiration, even leading to the invention of the surgical system carrying the Da Vinci name, that facilitates complex surgery with a minimally invasive approach and can be controlled by a surgeon from a remote console.

Over the years, AI development has relied upon the introduction of artificial neural networks (RNA) in the early 1950s, and their subsequent further evolution into the "computational learning models," including Machine Learning (ML) and Deep Learning (DL).

ML is based on algorithms that can improve task performance and decision making or predict outcomes. Specifically, with a previous training period in which ML algorithms focus on specific or characteristic features through exposure to examples, they learn to extract desired meaningful patterns from data, even seeing features that are beyond human perception [2].

DL is made of multilayered, “or deep,” artificial neural networks: the numerous neural layers between input and output contribute to the plasticity of DL and offer the potential to define new patterns of intelligent classifications, mimicking the human brain mechanisms. Different from ML, DL can automatically discern the relevant features from data (often with an unknown mechanism or overall rationale) even if unlabeled, allowing to learn new patterns and to determinate more complex relationships, which is valuable if compared with a human reader who can only partially detect and use a fraction of the total big information content of digital images [3].

Combining ML/DL image processing with clinical and pathology/histology data, to find direct correlation between intrinsic Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) patterns and specific pathology and histology subtypes, is the purpose of a new amazing sector of research called “Radiomics” [4–6].

Thus, AI is capable of globally revolutionizing healthcare and clinical practice, and will inform, influence, and transform radiologists more than most physician disciplines. AI will augment and support instead of replacing radiologists, via enhanced performance, both in terms of speed and precision, facilitating and improving the efficiency and accuracy of the diagnosis process. This will also help to customize treatments and use resources with maximum effectiveness, which are important priorities for interventional radiologists, who often diagnose and treat in real time and at the same time. Moreover, the utilization of novel navigational techniques that superimpose virtual pre-procedural 3D anatomic data onto real-world 2D visual images in real time, such as Augmented Reality (AR), multi-modality fusion, or Virtual Reality (VR), enables improved accuracy for minimally invasive treatments and reduced risks, complications and radiation, which are central goals for Interventional Radiology (IR) [7].

The purpose of this article is to emphasize how AI is relevant not only for Diagnostic Radiology, but also for IR. The potential applications of AI in IR go beyond computer vision, to include screening and modeling of patient selection, predictive and supportive tools for treatment planning and execution, training and intra-procedural registration, segmentation, navigation and treatment planning. Their real-time utilization in the angiography suite, with the help of augmented/virtual reality, could improve patient selection, operator’s expertise, drug discovery, and multidisciplinary pursuit of delivery of tailor-made precision medicine.

Fields of application

Workflow solutions: patient scheduling, screening and counseling

AI may directly optimize the interventional radiologist’s daily practice with intelligent patient scheduling, beforehand identifying patients at high risk and taking steps to avoid the risk, as well as reducing the likelihood of missing care, via intelligent scheduling and patient selection [8].

In addition, AI algorithms can generate a relevant summary from the available patient records, including patient problem lists, clinical notes, laboratory data, pathology reports, vital signs, prior treatments, and prior imaging reports to give the radiologist the most pertinent contextual information during the evaluation of a patient for diagnosis or therapy [9, 10].

In addition, intelligent algorithms are proposed for patient safety screening or reports, which have the potential for applications in radiology practice (for example, MRI safety screening or administration of contrast material) [11].

Moreover, the AR/VR can improve patient’s compliance through the possibility to show a simulation of the experience in angiography suite, allowing the patient to arrive calmer and more aware of what awaits at the time of the procedure [12, 13].

Clinical and Imaging decision support tools for treatment planning

One of the hardest challenges for IR is the identification of an accurate way to predict the success rate of a specific treatment in a specific patient, and to estimate/forecast the outcomes and benefits of a treatment in advance. Similarly, big data science has attempted to investigate how a patient’s demographic and pathologic characteristics before the treatment can influence its efficacy [14].

ML- or DL-based predictive models can try to classify patients as responders and non-responders to specific treatment algorithms. Such AI models based on DL or ML may help interventional radiologists to overcome this patient selection challenge, especially in the field of interventional oncology. Such tools hold the potential to reduce unnecessary and useless procedures and interventions, healthcare costs, and decreasing the risk for the patient [15].

From this perspective, both clinical and imaging decision support tools have been created. Among clinical ones, ML has been showcased in the evaluation of the relative prognostic significance of the Child–Turcotte–Pugh score and albumin–bilirubin grade, along with other risk factors, within a cohort of patients undergoing trans-arterial chemoembolization [16]. In addition, a ML algorithm based on serum creatinine in electronic health records was capable of predicting Acute Kidney Insufficiency (AKI) up to 72 h before onset, in specific patients, potentially allowing clinicians to intervene before kidney damage manifests, in advance of diagnostic and interventional radiology procedures and treatments [17].

Among imaging decision support tools, image texture analysis has made it possible to extrapolate “radiological biomarkers” that allow a prognostic pre-treatment evaluation.

For instance, the performance of pre-ablation CT texture features may predict post-treatment local progression and survival in patients who undergo tumor ablation, using ML to identifying specific CT texture patterns, as demonstrated by Daye et al. for adrenal metastases. During pre-procedural assessment, in fact, when CT-derived texture features were included in addition to clinical variables, there was an increase in accuracy to more than 95% [18]. Similarly, using MRI texture features and the presence or absence of cirrhosis, Abajian et al. assessed a ML algorithm able to predict chemoembolization outcomes in patients with hepatocellular carcinoma, obtaining a very good negative predictive value (88.5%) [19].

Additionally, in Neuroradiology, AI algorithms have been used to assist not only the diagnosis, but especially the treatment decisions in acute ischemic stroke, from estimating time of onset (that is one of the most relevant clinical criteria to decide if a patient is eligible for lytic treatment) [20, 21], lesion estimation [22], and salvageable tissue (essential in decision making before endovascular treatment) [23, 24]. This aided in analysis of cerebral edema [25, 26], and predicting complications, such as hemorrhagic transformation after treatment [27–29] and patient outcomes, as well as identifying factors that will contribute to neurological deterioration and increased morbidity or predict motor deficit in stroke patients after treatment.[30–33].

However, AI is not able to provide an explanation of the underlying cause and pathophysiology: more complex interpretation problems like these require deductive reasoning, medical knowledge and mechanisms to integrate the findings within a clinical, personal, and societal context. This is why the holistic human intelligence and the medical judgment of interventional radiologists will be augmented and cannot be completely replaced [34].

Image guidance: pre-procedural assessment and intraprocedural supportive tools

First of all, during pre-procedural assessment in a procedure room setting such as the angiography suite, it is critical to evaluate anatomy and its pathophysiologic changes. In this regard, Augmented Reality (AR) or Virtual Reality (VR) may find its best application. Through advanced 3D rendering and manipulation of imaging in space, AR and VR allow operators to conceptualize difficult anatomy, to increases realism in procedural planning (when compared with standard 2D images), and to improve procedural skills in a previous simulated environment, with no risk to patients [12].

Furthermore, by typical AI tasks may be applied to preoperative imaging to semi-automatically detect or identify specific patterns or organ or lesion segmentation. This is already used in breast, lung, liver and prostate cancer diagnosis, but is also valuable for interventional radiologists, such as with the biologic evaluation of an atherosclerotic plaque, which may allow adaptation of treatment technique [35].

For intra-procedural guidance, AI finds application in a wide range of settings:

• Automation of protocols, based on information gathered from the electronic health records and tailored to optimize radiation exposure in a quicker and automatic way [36].

• Real-time evaluation for proper catheter, sheath, or implant selection, such as stent or prothesis to treat stenotic vascular lesions or aortic aneurisms [37].

• Prediction of ischemia-related lesions, from quantitative CT angiography, for instance in myocardial or diabetic foot treatment [38, 39].

• Real-time estimation of fractional flow reserve, based on fluoroscopic imaging, for identifying ischemia-related stenosis in coronary artery or venous disease [40].

• Automatic vessels analysis, by use of intravascular ultrasound (IVUS): in complex cases of peripheral vascular diseases, it benefits by aiding with analysis of vessel size, lesion characteristic and potential post-treatment effects.

• Fusion of pre-procedure 3D images onto intraoperative 2D images: synchronizing MRI, CT or Cone Beam CT cross-sectional preoperative imaging with intraoperative real-time fluoroscopic or ultrasound images, allows for more precise guidance for biopsies [41–45] and local image guided therapies [46, 47] (Figs. 1, 2, 3) and improves problem solving during procedures [48–50]. Moreover, it is possible to generate, from CT images alone or in combination with MRI data, a virtual angiography [51] or “angioscopy” that could be a valuable procedure planning tool for interventional radiologists such as for aortic endovascular repair (Figs. 4, 5).

• Augmented Reality and AI-based navigation systems: by superimposing virtual 3D anatomic data, acquired using CT, CBCT or MR onto real-world 2D visual images in real-time, planning software generates a virtual device trajectory that is overlaid onto visual surface anatomy, facilitating accurate visual navigation, theoretically without need for fluoroscopy [7, 52–54]. Via integrated matching AI software, automatic landmark recognition and motion compensation was enabled using fiducial markers linked by a computer algorithm. This system is already applied in a range of minimally invasive surgery procedures including lesion targeting/localization, spinal/para-spinal injections, arthrograms, tumor ablation, bone biopsies and more recently in vertebroplasty, with reduction in patient radiation exposure compared with fluoroscopy alone [7, 47].

• Automatic tools for multi-modality registration and segmentation, as an enhanced and more ergonomic approach for fusion guided procedures, such as biopsy and ablation. This would enhance standardization and normalization, potentially enabling lesser experienced IRs to perform more advanced procedures, reproducing the skill or experience or techniques of experts.

• Touchless interaction devices directly correlate pre-procedural, patient-specific and literature-derived information, which reduces distraction, errors, and procedure time. These include eye-tracking systems, inertial sensors, cameras or webcams, and voice-driven smart assistants. In particular, the possibility of interrogating a previously instructed intelligent assistant to obtain suggestions on which device is most appropriate during a specific intervention procedure, before removing it from the sterile container, or what is the availability of the same in hospital stock or make a cost analysis with respect to another device, is even being studied by a group of researchers from the University of California. Its application could allow the operator to choose between two devices not only according to the appropriateness for the treatment but also in relation to the outcome data or patient-specific anatomy, optimizing results in terms of time and resource savings [55].

Fig. 1.

"Endobronchial navigation for lung tumor biopsy". a Pre-procedural CT planning; b 3D roadmap with tumor segmentation; c Real-time 3D fluoroscopic image

Fig. 2.

“Automatic 3D detection of arterial bleeding of the leg in a traumatic patient”—a CTA reveals arterial bleeding of the leg in a traumatic patient. b Initial 2D angiography does not demonstrate bleeding. c Target segmentation was performed on the CTA dataset after CBCT-CT fusion; then 3D CBCT datasets were synchronized with the C-arm and overlaid on live fluoroscopy, during the intervention, to facilitate catheter navigation to the damaged vessel. d Once the target arterial culprit was engaged, a selective angiogram is performed to confirm correct targeting of the bleeding site

Fig. 3.

“Automatic 3D detection of prostatic arteries using Cone-Beam CT during Prostatic Arterial Embolization”—a CBCT identification of prostatic arteries; b Realization of 3D roadmap; c Overlap on fluoroscopic images”

Fig. 4.

“Endovascular navigation: virtual aortoscopy”—a Preoperative CT image; b, c 3D volume rendering of Aorta, d Creation of virtual angioscopy of the Aorta

Fig. 5.

“Endovascular navigation: virtual aortic dissection angioscopy “—a, b Preoperative assessment on CT images; c Creation of virtual angioscopy to evaluate the approach for dissection treatment

Other hypothetical examples could be ML models that analyze the relationships between catheter or needle position, therapeutic effect and patient outcomes; particularly in the field of ablative therapies, this would allow for the estimation of ablation margins, optimal probe placement and selection of energy settings, potentially even minimizing risk of damage to nearby structures [1, 56].

Training and education

Machine learning combined with augmented reality systems could give trainees a new platform for honing their procedural skills, leading to novel ways of assessing performance and improving their competency [37]. In particular, AR and VR applications have been shown to improve upon current training methods in motivation, interactivity and learning of material; advanced 3D rendering and manipulation of imaging in space allows trainees to conceptualize anatomy and to improve procedural skills in a simulated environment, with no risk to patients. Moreover, VR increases realism for trainees in procedural planning when compared with standard 2D images and improve multidisciplinary communication with remote colleagues [12].

Limitations

Despite continuous progress over the past decades, AI has not and may never reach the holistic intelligence of humans; AI algorithms are intensely data-hungry, which can be difficult to obtain, and training can be time-consuming and expensive and require the clinically relevant question to be posed. Deep learning annotations for training may require bulk of human ground truth, and data curation may be a hurdle.

Whereas human intelligence can be acquired through even one simple observation and without feedback, AI software must see the same object potentially hundreds of thousands of times to recognize it, needing constant iterative feedback, to indicate whether it has guessed correctly or not.

One of the main weaknesses of AI systems is the lack of experience in real life, which is necessary to make it reliable for realistic questions, and capable of abstraction and generalization. Moreover, clinical practice is studded with atypical presentations of diseases that current AI systems might misdiagnose, because no finished training set can fully represent the variety of obscure or rare cases that can be seen in real clinical practice. Clearly, rare findings or features present possible weaknesses due to lack of large volume of particular features.

In addition, more complex problems of interpretation of radiology typically require deductive reasoning, using knowledge of disease processes and selective integration of information from previous examinations or patient records, and currently there is no learning system capable of achieving the most complex integration of thinking. [3]. In fact, some clinical problems, which may seem solvable in theory, are unsolvable in practice and are intractable for computers. Beyond the pre-treatment decision-making process, interventional radiologists perform sophisticated procedures that require technical expertise and instant intraprocedural decisions and in these situations, and even the fastest computer can get stuck in an infinite cycle or take an unimaginably long time to produce an accurate decision or result [34].

A further limitation of AI, in particular of DL algorithms, is that they are relatively opaque: it remains difficult to clarify mechanisms or what the different parts of a large network do. This makes it difficult to delineate the limits of the network or debugging errors in image interpretation.

Ethical issues, privacy, and data sharing on the use of patient data for training the AI models are timely, relevant, and variable by geography. Although AI might improve reproducibility or standardization of specific medical practices, physicians must take responsibility for the medical diagnosis and treatment provided to patients. The clinical advice provided by AI software should always be reviewed by an experienced healthcare professional, who may or may not approve the recommendation provided.

In fact, AI applications could produce significant ethical, legal and governance problems in the healthcare sector if they cause abrupt disruptions to its contextual integrity and relational dynamics.

Actually, the current regulation of AI in healthcare is subject to a miscellaneous guidelines and subjectivity of the authorities; the EU policy observes the following three Directives on medical devices: 93/42/EEC Council Directive concerning Medical Devices [57], 90/385/EEC Council Directive on the approximation of the laws of the Member States relating to active implantable medical devices [58], Directive 98/79/EC of the European Parliament and of the Council on in vitro diagnostic medical devices [59]. Many diverse and different data sharing, and privacy policies govern how different geographies interact. Federated learning might overcome some of these challenges, whereby the AI model weights are exchanged instead of the actual private or identifiable patient imaging data.

With many of these limitations, today's AI systems are produced as an auxiliary tool for radiologists rather than a replacement; decision making within the IR is likely to remain a matter of natural intelligence for radiologists, or a multidisciplinary team (supported by the AI), but with the physicians taking full responsibility, considering the input from the AI but not necessarily following it in a prescriptive manner [60].

Technology remains neutral and is only as useful or strong as the human guidance used to annotate or ask the appropriate questions of the correct data. Since the data used to train ML and DL algorithms are generated by humans, the utility all depends on how it is designed and implemented.

Conclusions

Although concerns have been expressed about the impact of AI on the radiology professional community, AI is not actually a threat, but a great opportunity. Used correctly, it could augment conventional practice of Diagnostic and also Interventional Radiology. Although futuristic and speculative, AI combined with Augmented Reality/Virtual Reality could help the interventional radiologist make faster, correct and more accurate and efficient, cost-effective decisions in terms of diagnosis, planning and treatment management, by allowing earlier disease identification, earlier treatments at an earlier stage and with less invasiveness, thus improving patient care and satisfaction.

Human and medical judgment, and even more empathy and compassion in communicating critical findings (upon which millions of patients rely in their medical care), are difficult to quantify and even more difficult, or even impossible, to simulate.

Therefore, modern radiologists need to be aware of the basic principles of AI; thanks to their ability to adapt and innovate, they will remain essential for medical practice and take a more decisive role in the process of digitization and personalization of medicine.

Essentials

• AI can improve patient’s compliance through the possibility to show a simulation of the experience in angiography suite through the use of AR/VR, allowing patient to aware of what awaits him/her at the time of the procedure.

• Through the use of ML DL algorithms, AI models predict the success rate of a specific treatment in a specific patient, to estimate/forecast the outcomes and/or the cost efficiency and benefits of a treatment before performing it, perhaps via investigation of demographic and pathologic characteristics in advance.

• During pre-procedural assessment in the IR/angiography suite, advanced 3D rendering and manipulation of imaging in space allow operators to conceptualize difficult anatomy, to increases realism in procedural planning when compared with standard 2D images, and to improve procedural skills in a previous simulated environment, with no risk to patient.

• One of the main weaknesses of AI systems is the lack of clear mechanistic rationale and real-life experience, and the lack of the capability to abstract and generalize. Atypical presentations of diseases could be misdiagnosed by AI systems, because no finished training set can fully represent the variety of cases that can be seen in clinical practice.

• Radiologists and IR should embrace AI tools that augment the practicality, quality, and efficiency of their practice and reject any AI model that does not.

Authors' contributions

Each author has contributed to conception and design, analysis and interpretation of the data, drafting of the article, critical revision and final approval.

Funding

This study was not supported by any funding.

Declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standards

This article does not contain any studies with human participants or animals performed by any of the authors.