Use of Artificial Intelligence in Non-Oncologic Interventional Radiology: Current State and Future Directions

Abstract

The future of radiology is disproportionately linked to the applications of artificial intelligence (AI). Recent exponential advancements in AI are already beginning to augment the clinical practice of radiology. Driven by a paucity of review articles in the area, this article aims to discuss applications of AI in non-oncologic IR across procedural planning, execution, and follow-up along with a discussion on the future directions of the field. Applications in vascular imaging, radiomics, touchless software interactions, robotics, natural language processing, post-procedural outcome prediction, device navigation, and image acquisition are included. Familiarity with AI study analysis will help open the current ‘black box’ of AI research and help bridge the gap between the research laboratory and clinical practice.

Introduction

Discussions on the future of radiology are often primarily centered around the revolutionary, future impact of artificial intelligence (AI). AI advancements, including the applications of machine learning, and more recently, deep learning, constitute a large section of research in radiology. It is important to distinguish between AI and machine learning (ML), often incorrectly used interchangeably. AI is broadly defined as the science of making intelligent machines, using varying techniques including statistical analysis and ML.¹ While ML, a sub-section of AI, is defined as techniques used to design and train algorithms to learn from available data, and improve its performance.¹ The implementation of ML was previously limited by the scarcity of high-quality healthcare data. However, the recent explosion of digital healthcare data (in terms of clinical data from electronic health records, real-time clinical monitoring signals, genetic sequencing, and imaging data) has led to large data sets, well-suited for the training of ML algorithms.^2–4 These large data sets are available to researchers and can be customized to study different topics via a set of inclusion and exclusion criteria. Although there are some caveats involved, namely the data is usually single-institutional, and the data is not easily accessible to the non-clinician researcher. Also, data available for ML algorithms has increased with the advent of genomic sequencing, and increased imaging utilization by the healthcare system.⁵ With research advancing at such a staggering pace, review articles have been necessary to inform the practicing radiologist about the development of clinically relevant research that allows for predictive clinical recommendations and personalized medicine. Increasing utilization of complex AI algorithms has also been enabled by the increased availability of inexpensive high-performance computing. AI has the potential to transform every step of the clinical process including diagnosis, therapeutic decision choice, intra-procedural guidance, and follow-up imaging. Work is also advancing in non-interpretive uses of AI in radiology including for scanner utilization, workflow, and billing.^6,7

Much of the literature in the field of interventional radiology (IR) has centered on applications of AI in interventional oncology. These studies (including review articles) primarily focus on optimizing patient identification and selecting prognostic predictors of IR procedure efficacy for treating cancers.^8–13 Research in AI has transformed the prevailing paradigm by using large data sets to uncover hidden associations and build predictive models that would be very difficult to do via the traditional technique of conducting prospective clinical trials. While clinicians are often familiar with the intricacies of logistic regression and hazard ratios, AI research has inherent complexity and specialized nomenclature that makes it difficult for many clinicians to critically analyze AI literature. Without some background, clinicians may be unable to determine the research’s potential utility or clinical impact. Also, it can be difficult to explain the functioning of AI methods – they appear to be ‘black boxes,’¹⁴ – and this can further widen the gap between the AI research laboratory and clinical practice.

The purpose of this article is to firstly provide an overview and brief definitions of some terminology involving ML, including recent advances in the field. Next, applications of AI in non-oncologic IR across procedural planning, execution, and follow-up, are described. Finally, potential future applications of AI in the field are proposed.

Overview of Machine Learning Concepts

Generalizations about what ML is ‘under the hood’ are sometimes made, comparing it to a combination of ‘if-statements’ and fancy statistics. In reality, ML is an umbrella that covers many different techniques, and learning is a property that emerges from a combination of several disparate branches of mathematics that can vary based on the approach used. The usefulness of ML comes from its ability to serve as a tool to solve problems using data by having a computer extrapolate patterns from this data, rather than explicitly creating instructions for the machine to follow. Machine learning allows machines to learn how to perform tasks that they were not explicitly programmed to execute.¹⁵ Applications of ML have been incorporated into every major industry since it can revolutionize any task involving data analysis/predictions. Some everyday examples include email spam filters, social media advertising, image recognition, and fraud detection.

Much of the recent excitement for machine learning has been due to developments in the realm of ‘deep learning’ brought on by the promising results neural networks have yielded when faced with increasingly abstract tasks. Inspired loosely by our own biology, neural networks consist of many ‘neurons’ connected into layers. Each neuron performs a very simple computation: a weighted sum followed by an activation function. There are input neurons, output neurons, and those neurons in between found in layers called ‘hidden layers.’ Neurons in layers adjacent to one another are connected, with each connection having a weight. These weights are learned in a training process using data inputs where the correct output is known. The input data features propagate through the hidden layers and are output by the final layer. The strength of activation of each neuron in this structure is reliant on the degree of activation of the neurons in the layer before it., based on their weighted sum. When an input passes through a particular layer, it leads to a web of activations in these neurons that layer-by-layer propagate through the entire network. At the output layer, a final neuron’s activation represents the network’s response to a given input. While each neuron is simple, complex behavior arises due to the use of many neurons in many layers. It is difficult to explain a network’s exact behavior in response to an input in terms of its connection weights. Instead, its performance can be probed in several ways characterizing the network’s strengths and weaknesses and the importance of the input features to reduce the perception that the network is a black box.¹⁶

There are many variants of neural networks, but two of the most currently relevant variants are Convolutional Neural Networks (CNNs) and Long Short-Term Memory networks (LSTMs). CNNs have revolutionized computer vision,¹⁷ and in doing so are more broadly applicable to medicine, especially via computer-aided detection (CAD) systems in radiology.¹⁸ LSTMs networks, on the other hand, are best applied to inputs that vary in time such as in natural language processing.¹⁹ In the following sections, applications of both types of neural networks in IR will be presented.

It is important to understand, however, that whatever reasoning the network generates reflects the training data provided to it. The accuracy of the model in completing its assigned task is directly related to its training data being complete, unbiased, and representative of the testing set. This means that if the data is sparse, then the model itself will be incomplete and bound to miss cases that were unlike those included in its training data. Furthermore, if the data is biased, then the model will provide answers in line with these biases instead of being a reflection of the truth.²⁰ To combat these pitfalls, the training data needs to be large and varied to create robust predictions for cases that may be considered fringe while keeping itself as generalized as possible. It is also important for researchers to be aware of the limitations of their data, and thus only use their trained models for tasks it was adequately trained to complete. Used responsibly, ML methods such as neural networks will likely revolutionize the healthcare sector in many ways such as allowing for personalized medicine, testing the efficacy of health policies and clinical trials, and integrating clinical care with the behavioral determinants of health.²¹

Overview of AI-led advances of IR interventions:

Radiomics

First described in 2012, radiomics is the high throughput extraction and analysis of data from computed tomography (CT), positron emission tomography (PET), or magnetic resonance (MR) imaging.^22,23 Radiomics data consists of quantitative imaging features in a form that lends itself to building models that relate these imaging features to gene and protein signatures and phenotypes. Thus, by treating medical images as more than visual aids, these methods harness their potential to provide diagnostic and prognostic information.²² The process of radiomics consists of several universal steps, each with unique challenges. These steps are image acquisition, identification of the volume of interest, segmentation, extraction, and quantification of features, populating databases with these features, and mining the data to develop classifier models.²² Image acquisition through CT, PET, and MR modalities allows for a wide variation in the acquisition protocol, making it difficult to standardize images across institutions, machinery, and patient populations. This presents a challenge in radiomics, as it is crucial to extract data that can be directly compared (Figure 1).

Figure 1:

Schematic Showing the different Applications of Artificial Intelligence in Non-Oncologic Interventional Radiology

Based on the literature, different aspects of the applications of artificial intelligence in non-oncologic interventional radiology were identified and illustrated schematically using a Circle-Spoke diagram

In radiomics, ‘volumes of interest’ often refer to tumors and suspected tumors. Radiomics also allows for the identification of sub-volumes within these regions, which gives radiomics its high prognostic potential.²⁴ Segmentation of volumes of interest can be challenging, as these volumes can be indistinct. It is generally agreed upon that computer-aided border detection and manual fine-tuning is the best approach to accurate segmentation. The actual features that are extracted can be either semantic or agnostic. Semantic features include size, shape, vascularity, etc. while agnostic features include mathematical, quantitative descriptors such as histograms, textures, and Laplacian transforms.^23,24 The larger a radiomics dataset, the more power it has because the mining process involves discovering patterns across a large amount of data. This process can involve machine learning approaches, statistical approaches, neural networks, and Bayesian networks.²⁴

Radiomics is most widely applied in the field of oncology, largely owing to support from the National Cancer Institute.²⁵ The use of radiomics in oncology has led to advancements in the ability to diagnose cancer, determine tumor prognosis, and assess treatment options.²⁴ While radiomics has mostly been used for interventional oncological applications at this time, its use in image segmentation and diagnoses bodes well for future applications in non-oncologic IR as well.²⁴

Touchless Software Interaction

In the operating room and the interventional radiology suite, physicians face challenges interacting with computers during procedures within the confines of a sterile environment. Touchless human-computer interaction technologies have the potential to decrease the possibility of error and inefficiency during procedures by offering physicians an alternative, smoother method for interacting with medical software in sterile environments.

Machine learning has emerged as an important approach for streamlining these touchless software-assisted interactions using voice and gesture commands. Machine learning frameworks can be trained to classify voice commands and gestures that physicians may employ during a procedure. These methods can contribute to better recognition rates of these actions by cameras, speakers, and other touchless devices.^26,27

In one study, inertial sensors worn on the head and body produced data that was collected and classified as low-dimensional body gestures.²⁸ These sensors eliminate the problems associated with cameras such as ensuring adequate illumination or the need to perform the gestures in the camera’s line-of-sight.²⁹ These are both common problems in a dimly lit interventional suite with multiple moving parts. These gestures were then learned and parameterized by machine learning software, which produced a 90% recognition rate.²⁸ Another study extended this technique to include the option of voice-based or switch-based activation of gesture recognition, with both options achieving a precision of within 2–4% of the parameter range.²⁹

Such gesture recognition principles have also been applied to IR procedures, such as percutaneous nephrostomies.³⁰ In this application, the physician interacts with ultrasound visualization software to determine a needle entry and exit trajectory into the renal pelvis. Physicians were asked to produce five fixed gestures, and angular information for each gesture was manually determined to train and optimize the algorithm to recognize these gestures with 92–100% accuracy. Furthermore, the mean placement error in selecting the target on the ultrasound snapshot was 0.55 ± 0.30 mm.³⁰

Natural Language Processing

Natural Language Processing (NLP) is broadly defined as the field of ML dedicated to the analysis, manipulation, restructuring, and generation of spoken and written language. NLP has special applications in the field of diagnostic radiology to generate standardized reports from the automated extraction of large amounts of clinically relevant information found in the radiologist’s reports.³¹ Owing to the increased diversity of NLP applications in clinical medicine, it naturally impacts how IR is practiced as well.

With the rise of widely available devices such as the Microsoft Kinect and the Google Home, a new wave of research is aimed at re-deploying these pre-fabricated, relatively cheap devices to the procedural suite.^27,32 In one particular application for device sizing and compatibility in IR, a Google Home speaker was outfitted with an NLP implementation.²⁷ Input queries were processed and sent to the cloud which contained data on 475 IR devices for sizing and compatibility (e.g. what size sheath is required for a particular Amplatzer plug).²⁷ Once the solution was determined, it was played back to the operator using the smart speaker, with technically satisfactory outcomes.²⁷ Expanding on prior work on the use of ML in touchless software interaction, standardized machine learning frameworks have allowed for the use of LSTMs in NLP (as mentioned previously) for multimodal gesture recognition that outperforms other state-of-the-art.³³

NLP can revolutionize IR outside of the procedural suite as well. In one study, scientists developed a virtual text-based radiology assistant/consultant.³⁴ It was developed to serve as the communications point-of-contact between referring providers and IR. The program simulates conversation on a traditional text-based platform. Once queries are entered into the system, relevant information (in the form of infographics and websites) are sent back to the provider.³⁴ This tool allows clinicians to focus on patient care and reduces the time spent on repetitive communications tasks.³⁴ On the referring provider’s end, this tool allows them to obtain information quickly without any wait time for a provider to be available. Further quality control needs to be performed to make sure the system forwards queries to the provider reliably when it receives questions outside of the data it was trained on.

Post-Procedural Outcomes

One of the most basic examples of ML is the decision tree, well known even outside of computer science. Originally used in the field of statistics to help predict outcomes using decisional probabilities, decision trees have influenced an entire branch of ML. This approach’s branching logic is simple enough to be followed and recreated by hand. A robust example of decision trees in ML, known as the Random Forest, is an ‘ensemble’ or collection of many decision trees working together. The notion is that many weak learners (or weak predictors) can be combined to create a stronger one.^35,36 A technique like the random forest can find relationships in high dimensional data that are likely beyond the scope of what an individual researcher with the help of traditional statistical methods could do themselves. The random forest, and other similar methods, can incorporate all the measurable data that a patient possesses. It also incorporates complex nonlinear interactions between these variables and use this information to support decision making and can solve unique problems in patient care, such as advanced prediction of patient outcomes to more personalized plans of care.^35,36

In IR, this approach has been successfully utilized to predict iatrogenic pneumothorax after CT-guided transthoracic biopsy (TTB), in-hospital mortality after transjugular intrahepatic portosystemic shunt (TIPS), and occurrence of length of stay >3 days after uterine artery embolization (UFE).³⁶ These applications are particularly enabled by the availability of large sets of patient-associated clinical data from the adoption of electronic health records.³⁶ One of the major strengths of this study is the fact that all of the variables that are input to the model are pre-admission values.³⁶ This method would allow for patient-care optimization based on risk-stratification performed at the time of admission.

Vascular Imaging

The field of vascular imaging has been significantly improved by the applications of AI. These advances extend both to peripheral arterial disease (PAD),³⁷ as well as to neuro-interventional radiology (NIR).^38–40

In PAD, an ML-based model outperformed a traditional statistical model using stepwise logistic regression both for the identification of PAD (p=0.03) and predicting future mortality (p=0.10).³⁷ Limitations of this study included PAD defined just by clinical metrics (ankle-brachial indices) instead of symptom metric and a short follow-up time. The ability to seemingly ‘throw’ a series of metrics at an ML algorithm and allowing it to construct a model that outperforms statistical techniques is a hallmark of ML-based research. This allows for risk-stratification, not possible with other techniques. In the setting of acute ischemic stroke, advanced age is associated with higher morbidity and mortality. One study utilized an ML algorithm to optimize the selection of elderly patients, who would benefit the most from an endovascular thrombectomy (ET).⁴⁰ This study was particularly important, given the fact that elderly patients have been under-represented in clinical trials on ET. ML techniques have also been used to predict the outcomes for endovascular embolization of brain arteriovenous malformations.³⁸

Device Navigation

In medicine, there has been a general move towards minimally invasive surgery to reduce patient morbidity and mortality. Percutaneous procedures particularly rely on a high degree of accuracy to ensure appropriate placement of devices inside the body, and to guarantee that the surrounding tissue is left undamaged. This level of precision is provided by accurate image-guidance. For this task, 2-D fluoroscopic guidance, as well as 3-D guidance using intra-procedural cone beam CT is utilized.⁴¹ The primary advantages secured by utilizing AI include automated landmark recognition, compensation for motion artifact, and generation/validation of a safe needle trajectory.⁴¹ Fritz et al., have shown the efficacy of an augmented reality system for a variety of IR procedures performed on cadavers, including spinal injection and MR-guided vertebroplasty.^42,43

Auloge et al. performed a 20 patient randomized clinical study to test the efficacy of percutaneous vertebroplasty, for patients with vertebral compression fractures.⁴¹ Patients were randomized to 2 groups: procedures performed with standard fluoroscopy and procedures augmented with AI-guidance.⁴¹ The metrics studied included trocar placement accuracy, complications, trocar deployment time, and fluoroscopy time. All procedures in both groups were successful with no complications observed in either group. No statistically significant differences in accuracy were observed between the groups. Fluoroscopy time was lower in the AI-guided group while deployment time was lower in the standard-fluoroscopy group.⁴¹ Further research needs to be performed to be able to generalize these results to the whole spectrum of percutaneous procedures.

Robotics

The increased utilization of robotic assistance in IR has been reviewed in the literature.^44,45 The greatest impact offered by existing robotic surgery is increased precision and accuracy, along with an increase in degrees of freedom.⁴⁴ In IR, the use of robotics also allows for radiation protection by limiting the operator’s exposure to radiation during procedures.⁴⁴ Applications of computer vision can reduce the tissue trauma prevalent in both traditional open surgery, as well as endoscopic procedures. Using sensors located at the catheter tip, Fagogenis et al. created an autonomous robotic navigation system with haptic (touch-based) vision, for utilization in an aortic paravalvular leak closure.⁴⁶ ML and image processing algorithms used the sensor data to provide information on the type of tissue that the catheter tip is touching (e.g. blood, cardiac tissue, valve) and the level of force applied. Autonomous control was found to be faster than joy-stick controlled robotic navigation.⁴⁶ In an article published in Nature Machine Intelligence, Chen et al. described an autonomous robotic guidance system for obtaining vascular access.⁴⁷ This deep learning system utilizes near-infrared and duplex ultrasound imaging to steer needles or catheters into submillimeter vessels with minimal supervision.⁴⁷ This system can autonomously compensate for the patient’s arm motion, as well as identify the success or failure of the cannulation attempt based on ultrasound and force feedback.⁴⁷

Image Acquisition

The goal of this collection of techniques is to optimize image acquisition without a loss of image quality. Metrics to be reduced include radiation⁴⁸, contrast dose⁴⁹, and image-processing time⁵⁰. Altered image appearance and artifacts have limited the utilization of this technology in the past. However, deep learning algorithms have been shown to perform well in terms of structural fidelity and are faster as compared to the competing state-of-art methods. Routinely obtained as a part of IR procedures, imaging optimization can improve the clinical practice of IR while reducing risk (through contrast-injection and radiation) to the patient.

One study used a deep convolutional neural network to map low-dose CT images to their respective normal-dose images in a patch-by-patch fashion.⁵⁰ This method enables the algorithm to minimize artifacts while preserving the structure of the image. In the field of nuclear imaging, deep learning has been used to reduce radiotracer requirements for amyloid PET/MRI without sacrificing diagnostic quality.⁴⁸ This model used low-dose PET images and simultaneously acquired multimodal MRI sequences as input. This technique allows for ultra-low-dose imaging, which reduces patient radiation exposure to levels close to those found during a transcontinental flight.⁴⁸

Future Directions and Conclusion

AI has been already been applied in a variety of applications in IR, ranging from procedural optimization to workflow optimization. In this section, ways in which AI can further impact the field are proposed.

Operator inefficiency is often caused due to the need for repetitive verbal communication between the proceduralist and the circulating nurse for the manipulation of the hardware or software in the room. One study successfully used a voice recognition interface to adjust various parameters during laparoscopic surgeries.⁵¹ These include hardware related tasks such as the initial set-up of the light sources and the camera, as well as procedural steps such as the activation of the insufflator.⁵¹ In IR, a similarly significant delay is caused by the proceduralist having to wait for the circulator to manipulate the C-arm as well as reposition the patient on the table. A voice-recognition based system would allow for significantly lower delays during procedures. Furthermore, studies have used eye-tracking to optimize training during surgical procedures.⁵² A tangentially related use of this technology could be applied for fluoroscopy beam collimation in IR. An algorithm’s analyses of operator eye fixation and gaze patterns could automate collimation, thereby narrowing the fluoroscopy beam and reduce the total radiation dose that the patient is exposed to. This technique could automate this process instead of being reliant on communication between the operator and circulating nurse for a task that is often left to an afterthought.

With the increasing clinical nature of IR, ML applied to widely available data from electronic health records can be used to manage complications. Contrast-induced acute kidney injury (AKI) is a well-documented cause of nephropathy after interventional procedures that utilize contrast media.⁵³ The extent of renal impairment is often difficult to assess due to limited prior laboratory values. Without these values, it is difficult to obtain a sense of the acuteness of the changes in the patient’s creatinine. ML has been successfully been applied to the intensive care unit (ICU) setting, to accurately predict pre-admission hemoglobin and creatinine in patients.⁵⁴ The model uses clinical data commonly available within the first few hours of ICU stay, with predictions averaging within 0.3–0.4mg/dl of actual creatinine values.⁵⁴ Having accurate predictions of a patient’s baseline creatinine would be indispensable to guide the modality as well as the intensity of treatment.

An important issue in the use of AI to inform the clinical practice of radiology is the potential propagation of bias inherent in the data.⁵⁵ An initially equitable algorithm can be made biased by prejudiced data/human decisions.⁵⁶ Some suggestions to prevent this bias from affecting AI performance include: making it a fundamental requirement to be able to explain and interpret the output of every clinical AI system, to eliminate the ‘black box’.⁵⁵ Additionally, transparency about removing existing biases in raw data used in an algorithm and avoiding adding new biases should be included in the model description.⁵⁵

The use of ML in robotics and augmented reality systems have also been studied for use in trainee education, and have been proposed for future use in IR.¹³ For example, researchers in urology have utilized ML to process procedural data in robot-assisted radical prostatectomy to evaluate performance and predict outcomes.⁵⁷ This study collected automated performance metrics (obtained from the surgical robot system) such as camera idle time, dominant instrument moving time, and camera moving time. These metrics were successfully used to predict an extended length-of-stay for patients’ post-procedure, utilizing a random forest model.⁵⁷ Furthermore, certain metrics of performance such as surgery time and foley catheter duration as predicted were also significantly associated with the actual measured values.⁵⁷ One way ML can be used in robotics in IR can be by the use of biomechanical algorithms to predict an interventionalist’s motion based on posture and weight distribution. The use of lead aprons for radiation protection is known to cause a variety of musculoskeletal issues.⁵⁸ The use of predictive human biomechanics would allow for a weight-bearing exoskeleton to better predict and replicate the movements of an interventionalist in real-time. This would allow for weight offloading and thereby, reduced stresses on the operator’s body.

With further advancement of the field, there is a significant space for AI to further optimize channels in IR, ranging from trainee education to risk stratification, to fundamentally alter the clinical paradigm of the field.