Artificial Intelligence in Interventional Radiology

The concept of artificial intelligence (AI) elicits futuristic images in the minds of many, perhaps reminding them of films about dystopian worlds. References to AI go back to the 1950s when British computer scientist Alan Turing developed the Church-Turing thesis, which states that natural numbers cannot be computed by a human being unless they are computable by a Turing machine (computer). Due to the fact that computers were not powerful at the time, applications were limited. ¹

John McCarthy, esteemed mathematician and computer scientist, coined the term “artificial intelligence” in 1956 during a highly attended summer workshop in Dartmouth. He complained that, “as soon as it works, no one calls it AI anymore.” ² By 1959, machines began playing checkers better than humans. In 1965, McCarthy founded the Stanford Artificial Intelligence Laboratory, where groundbreaking research was conducted in computing and automation. ³

AI took a back seat during the 1970s and 1980s (the so-called AI winters), but major breakthroughs occurred soon afterward. The 1990s were the dawn of faster computers and big data. In 1997, Deep Blue—a chess-playing supercomputer developed by International Business Machines (IBM) Corporation—defeated reigning world champion Gary Kasparov in a pair of six-game chess matches. ¹ IBM later developed Watson in 2011, the supercomputer that defeated two reigning champions at Jeopardy! This was also the year that McCarthy passed away.

AI in radiology, though still in its infancy, has made tremendous strides. AI has been investigated in interventional radiology (IR) as well, though to a lesser extent. This article will summarize the role of AI in radiology and IR thus far and delve into its future applications.

Understanding the Vocabulary

A large part of understanding the current state of AI is understanding the vocabulary. “AI,” “machine learning (ML),” and “deep learning (DL)” are commonly used terms that are, unfortunately, not synonymous. The following is a broad overview of the definitions relevant to radiology and IR.

Artificial Intelligence

Artificial intelligence, also known as machine intelligence, is a broad umbrella term used to denote a machine's cognitive abilities that are akin to those of the human mind. ¹ Lynn Parker, director of the Division of Information and Intelligent System of National Science Foundation, defines AI as “a broad set of methods, algorithms, and technologies that make a software ‘smart’ in a way that may seem human-like to an outside observer.” ²

AI is essentially an umbrella term for a variety of applications and techniques ( Fig. 1 ). In a sense, the term “AI” resembles terms like “science,” “biology,” and “mathematics” since there are multiple completely unrelated branches that fall under these broad categories. For example, AI is used in commercial software in the form of speech processing, navigation systems, video games, and meeting scheduling. AI also involves hardware in the form of robotics. In radiology, the most commonly cited form of AI is ML. ⁴

Fig. 1.

Branches of artificial intelligence. (Derived from Andriole. ⁴ )

The current era of AI has been referred to as “narrow AI” or “weak AI,” in which the AI specializes in one limited area or task. In that regard, one form of AI can be powerful enough to beat a reigning world chess champion but would not know how to beat a 5-year-old at tic-tac-toe. ² We are far from the kind of AI that Hollywood constantly describes: artificial general intelligence, also known as “strong AI” or “human level AI,” where the machine can reason or be “self-aware.”

Machine Learning

Machine learning, a subset of AI, is the process in which a machine learns from its environment and uses the knowledge for future applications. ¹ Though this sounds elusive, ML is best understood as a class of advanced statistical techniques in which the program develops algorithms based on input datasets and then applies those algorithms to unseen datasets. ² A simplified analogy is how processing programs construct a best fit line for multiple provided data points. Machine “learning” simply uses iterative techniques to make predictions from available data and then apply the trends to new data. The ideal ML model should include inputs that are relevant to the outcome and generalizable enough to be applied to the given population.

Within medicine, the ML process is much complex and variable. Most ML applications focus on predicting outcomes, patient distribution, and detection based on epidemiological and trial-driven input. In radiology, ML requires a great deal of computing power, datasets, and materials. ML has gained a lot of press, especially at recent Radiological Society of North America (RSNA) annual meetings. ^{4 5 6} Typical ML tasks in radiology are the identification of specific patterns/conditions of image segmentation, partitioning digital images into meaningful parts (i.e., pixels or segments) for interpretation. Applications of ML include, but are not limited to, fatty liver detection on ultrasound, carotid plaque characterization on computed tomography (CT), and CT coronary angiography lesion detection. ⁷ Due to the growing influence of ML on vital decision making in medicine, many have advocated for methods to evaluate the accuracy of ML algorithms (cross-validation). ⁵

Artificial Neural Network

Machine learning itself has many sub-fields, one of which is neural networks, also known as “artificial neural networks” (ANNs). ² ANN approximates the complex relationship between paired inputs and outputs by using a simple set of building blocks loosely inspired by biological neurons. Each neuron consists of a set of weights and biases. The inputs to a neuron are multiplied by their corresponding weight and summed. The resulting scalar value is then passed through a nonlinear activation function. An ANN is trained using a technique called “back propagation,” which iteratively modifies the weights given a set of input and output training pairs. As with all ML, ANNs are a form of complex math and statistics. Keep in mind that actual biological brain connections are much more complex. ²

Convolutional neural networks (CNNs) are a special type of ANN adapted to regular signals such as images in radiology. A simple CNN contains five layers: input, convolution, pooling, a fully connected layer, and an output layer ⁸ ( Fig. 2 ). A “convolution” involves taking one of more inputs in the form of matrices (which may, for example, correspond to an image dataset), matching them onto an image/pixel, multiplying them by an arbitrary value, and transposing the value into a new matrix. This technique is applied to imaging in edge detection and sharpening features, which accentuates the data found at edges/interfaces. This is also used to bin or compress large data (at the cost of information/resolution) by transforming multiple pixels into one representative pixel. ⁹

Fig. 2.

A pictorial representation of the codependence and interconnectivity of neurons in a convolutional neural network. (Adapted from Phung. ⁸ )

Deep Learning

Deep learning, also a subset of ML, utilizes many ANN layers and allows a more sophisticated performance desirable for imaging. ² In radiology, DL surpasses traditional ML in that the user does not have to hand-craft the features going into the models; the machine extracts the features autonomously and classifies them as it goes. Not only is this easier, but the machine may be better at extracting features that are not humanly intuitive. For example, the human eye sees only approximately 2 ⁷ to 2 ⁸ bits. CT scans register 2 ¹⁶ bits of information (which is why the radiologist needs window to see all of the data). Contrary to humans, the machine does not have this visual deficit. ⁹

A recent advance in DL is generative adversarial networks (GANs), in which a computer is trained to create images that look “real” even though they are completely synthetic ( Fig. 3 ). ¹⁰ One can observe GANs' potential at the Web site https://thispersondoesnotexist.com , which artificially generates faces of what convincingly look like real human beings. GANs utilize a “generator” and a “discriminator.” The generator creates a fake distribution, and the discriminator decides whether the distribution came from the generator or if it is actually real data. This constant “battle” between the generator and discriminator is the reason these are termed “adversarial” neural networks. The system improves with multiple iterations. A special type of GANs (called conditioned GANs) is used to transform magnetic resonance (MR) sequences to artificially rendered CT, PET to attenuation correction CT, and one MR sequence to another—all of which can be useful in improving diagnoses and avoiding extra imaging/radiation. ¹¹

Fig. 3.

Pictorial representations of how generative adversarial networks work. (Adapted from Erickson. ¹⁰ )

Indeed, DL is a trillion-dollar industry that has been strongly driven by the corporate sector: Apple, Google, Microsoft, Facebook. ⁴ The most pressing issue with DL and GANs is the massive amount of data needed to develop trustworthy results. Since GANs have many parameters, they need millions of examples. For instance, there are plenty of photos of human faces in direct profile for GANs to analyze, resulting in the convincing artificial photos seen online. However, this is not the case for cats. Most cat photos come in multiple poses, which limits GANs' ability to construct artificial cats accurately, as seen in many of the cat reconstructions from a similar Web site: https://thiscatdoesnotexist.com .

Applications to Interventional Radiology

AI has been fleshed out to some extent in the diagnostic arena. However, its applications in IR are in their infancy and vastly variable. The IR community is currently seeking answers for how to apply AI to IR, so much so that the Society of Interventional Oncology held its first Artificial Intelligence Hackathon, a 1 hour open session in which teams pitched their AI-based research project proposals to panelists and industry representatives for a grand prize of a $7,000. ¹²

Although there are limitless possibilities in IR, AI has potential to improve pre- and periprocedural imaging, clinical predictive models, robotics, automation of repetitive tasks, workflow, and teaching. ¹³

Imaging

Many of the AI applications in diagnostics are directly translatable to preinterventional planning. Volumetric analysis has been used for many years to determine liver segment size in preparation for locoregional therapy. ¹⁴ AI is already being used in periprocedural angiography to assess vessel stenosis, flow dynamics, and stent size planning. ¹⁵ Vessel tracking software for hepatic vasculature is available for hepatic angiography and has even been extended to outside the liver. ¹⁶ Multimodality imaging fusion has also been studied and has become commercially available: ultrasound/CT for tumor ablation, ¹⁷ ultrasound/MRI, and ultrasound/positron emission tomography/CT. ¹⁸

As previously mentioned, GANs have been used to artificially transform preoperative MR into artificially rendered CT, which can be helpful to the interventional radiologist in studying vascular anatomy. One can imagine the role of GANs applied to preprocedural arterial-phase CT or MR to construct artificial digital subtraction angiography in various projections, which can be made available to the interventionalist even before the procedure takes place. This can save interventionalists a great deal of time and fluoroscopy during complex procedures like prostatic artery embolization, where the target arteries may be difficult or nearly impossible to identify. This can also be extremely helpful in planning studies for hepatic radioembolization, which often requires a mapping visceral angiogram to be performed 2 to 3 weeks prior to treatment. Once GANs technology matures, it may even be possible to do away with mapping angiography altogether. However, it is important to keep in mind that GANs produce images that are “like the real images,” not the real images themselves. This can be dangerous, as our reconstructed images would essentially be hallucinated data from the input imaging.

Clinical Decision Making

IR, as with all clinical specialties, takes great interest in patient prognostic indicators and predictors of treatment response. In the IR arena, there are a multitude of studies investigating such predictors. ^{19 20 21 22 23 24} Some investigators have studied AI for these purposes. Abajian et al used ML in pre-locoregional therapy MR to predict hepatocellular carcinoma response. ²⁵ Another group utilized ANN to compare the prognostic performance of the albumin–bilirubin grade and the Child–Turcotte–Pugh score for hepatocellular carcinoma status post transarterial chemoembolization. ²⁶ The greatest limitation in applying AI to such prognosticators is the paucity of data. More studies with millions of patients must be conducted to provide reliable results.

Procedural Accuracy

As the patient demand grows in IR, so does the need for periprocedural quality. It is unfortunate when a complex, invasive, and potentially life threatening biopsy procedure yields an indeterminant result. Furthermore, the advent of targeted patient therapy has necessitated more tissue than previously needed. ANN and DL have been employed in tissue sample analysis, which not only mitigates the amount of tissue needed but also can be used perioperatively for the interventionalist to confirm that the biopsy is diagnostic. ²⁷ One can imagine AI's role in doing away with direct tissue sampling altogether.

Robotics and Navigation

Robotics and automation have played major roles for surgical subspecialties and have begun to do so for IR as well. A robot is “a reprogrammable, multifunctional manipulator designed to move materials, parts, tools, or other specialized devices through various programmed motions for the performance of a variety of tasks.” ²⁸ As such, robots are a subset of AI, albeit much different from the forms prevalent in diagnostic radiology. Medical robots include ROBODOC (first medical robot invented in 1992 for orthopaedic surgery) and the Da Vinci for urology. In IR, the Light Puncture Robot has been utilized in France for CT/MR-guided thoracic and abdominal punctures. ²⁹ In ultrasound, robot-assisted needle steering for percutaneous access has been investigated. ³⁰ The AcuBot is a robot used in fluoroscopic interventions for perispinal nerve and facet joint injections. ³¹ Robots have also been used for perioperative endovascular steering in cardiology and aortic grafting. ^{32 33 34}

Image-guided navigation tools, a form of AI, have also been studied. ¹⁸ One type is electromagnetic navigation (a similar technology to satellite-guided Global Positioning System), which is used in intraprocedural CT biopsy tracking. ³⁵ Another tool used with CT and MR is optical tracking (e.g., the Activiews CT-Guide needle guidance system; Stryker Corporation, Kalamazoo, MI), which uses a video camera with skin markers for localization. ^{18 36} Laser guidance with conventional and cone-beam CT is also commercially available. ^{37 38}

Recently, automation in the form of touchless interaction software has allowed the operator to be able to perform procedures without placing his/her hands on equipment. ³⁹ Voice recognition software has come into play in this arena. ⁴⁰ Virtual reality is on the horizon, now being used to navigate and to artificially construct blood vessels into three-dimensional renditions made available to the interventionalist in real time. ⁴¹

Resident Training

Hospital demand for fast and accurate imaging turnaround time and reliable procedural technical success has resulted in increased involvement of attending physician in tasks previously completely delegated to resident physicians. This inevitably will take its toll on resident education. AI has thus been investigated to provide simulated learning alternatives for residents. ⁴² The efficacy of robotic guidance in virtual surgical simulators has been studied. ⁴³ Video games (which utilize AI) have been implemented in diagnostic radiology education. ⁴⁴ Others are currently utilizing AI with ML algorithms for resident performance evaluation. ⁴⁵ AI has even been investigated at the medical student level for scoring written tests and automating the assessment of procedures, communication (via facial recognition or linguistic analysis), and history-taking skills (speech to text). ⁴⁶ As with other applications of AI, meticulous cross-validation is required to ensure fairness.

The Pitfalls of AI

AI, in all its facets, is a powerful tool. Interventionalists, diagnosticians, and specialists now have a far greater potential to provide excellent patient care owing to the advances in robotics, ML, and DL. However, no tool is perfect. Major pitfalls of AI come in the form of lack of data and lack of direction. AI can be useful only if it answers a clinically relevant question, has robust reliable data, strict definitions, meticulous cross-validation, and appropriate training. ⁴ In sum, garbage in equals garbage out.

Second, as AI grows and enters the corporate space, patient integrity must remain of utmost importance. Interestingly, the advent of big data (e.g., Google in 1998) took place only a few years after the Health Insurance Portability and Accountability Act (HIPAA) of 1996. As big data matured, patient privacy was clearly not at the forefront of these companies' agenda. There are no technical standards on how to de-identify medical images. Are data really de-identifiable? How would de-identified labels be generated? Because AI today requires volumes of well-partitioned data, vast monetary value has been placed on these data. This is where the corporate sector has taken the lead. Mishandling that data can be ethically detrimental. ⁴⁷ One of our greatest challenges is how to prevent unethical use of data for monetary profit.

Third, it is important to note that all data are inherently biased. For example, say the question arises of who should be the next chairman of a company. Deciding to use AI to make an unbiased selection may seem appealing, but imagine if the machine was trained by data based on prior selections. Imagine if the prior selections all happened to be males. Inevitably, the machine would develop an algorithm that is biased toward males. Similar issues arise in disease diagnosis and treatment algorithms. ⁴⁸ An important example occurred when an AI software was trained with multiple chest radiographs to recognize pulmonary tuberculosis. Little did the investigators know, all of the positive radiographs came from a clinic in which the radiographs were labeled “TB clinic.” Therefore, any patient from the TB clinic was biased with a positive result. ⁴⁹ We must be wary of the data we provide to machines and be vigilant of the limitations of AI.

In addition, it is important for the clinician to be aware of automation bias: the tendency for humans to favor machine-generated decisions over human decisions. This can lead to lack of monitoring of AI and blind agreement with its decisions. In IR, this may be seen when the interventionalist trusts an erroneously favorable pre-radioembolization lung shunt fraction when there was clear evidence of arteriovenous shunting during the procedure. Additionally, vessel tracking software is prone to errors; an operator must not trust a false target vessel acquired over his/her angiographic experience.

Finally, although there is no magic to AI (just a lot of math), it is difficult to discern how exactly the machine bases its patterns, especially with proprietary DL and GANs that autonomously collect and categorize information. This is the so-called black box of AI. ⁵ This no-man's land (what ML experts refer to as “lack of explainability”) lends itself to uncertainty. We have a moral duty both to understand the data put into AI and how that data are being processed. ⁵⁰

Conclusion: What Does the Future Hold?

One important take-home point is that AI, much like the field of IR itself, is a discipline with a multitude of diverse branches. All subsets of AI strive to develop machines with (seemingly) cognitive capabilities. As time and generations pass, and as technology evolves, so will machines. The primary determinant of evolution is the origin, what we put into the system. Machines need to be cautiously equipped with reliable, unbiased data and with clear tasks. In healthcare, we need to respect our patients' privacy when implementing the technology and always place patients' well-being over profit. We must not be overtaken by the “black box” of AI. We must demand transparency in our algorithms, or risk losing privacy, security, and impartiality.

IR was born from innovation. Our specialty thrives on the principles of AI. Whether it be in imaging, robotics, or clinical decision making, the common principle of thinking outside of the box comes into play. It is the duty of the interventional radiologist to understand the available technology and collaborate with experts in the multifaceted branches of AI. It is difficult to predict what is next for AI: more robust ML and algorithms, more powerful software, more capable robotics, further commercialization, the Hollywoodesque Artificial General Intelligence (unlikely). The evolution of AI will depend on unmet needs, changing times, and human pioneers. Interventional radiologists must identify these unmet needs, take actions to address them, and ultimately improve the delivery of patient care.