From research to reality: The role of artificial intelligence applications in HCC care

Abstract

Deep learning (DL) refers to the use of deep artificial neural networks to analyze complex data. DL is the most widely used artificial intelligence (AI) method and is increasingly employed across various sectors of our society, including clinical and translational research.¹ Specifically, DL methods have substantial capabilities in the handling of complex and unstructured data, such as medical images and text data.

Primary liver cancer, with HCC encompassing about 80%–90% of cases, is a substantial global health burden and there is a critical need for improved early detection, proper prognostication, and treatment algorithms.² Recent evidence suggests that DL could support liver cancer management from early diagnosis to patient stratification and treatment selection.^1,3 However, research results cannot be directly used in clinical routine. Integrating AI into liver cancer treatment is complex because it must address both chronic liver disease and cancer. The evolving nature of these diseases often leads to discrepancies between AI training data and real-world applications, posing a challenge to the effective deployment of AI applications.⁴ AI methods are medical devices that require clearance or approval by the Food and Drug Administration (FDA) in the United States of America, and Conformité Européenne (CE)-marking after assessment under the Medical Device Regulation (for radiology-based approaches) or In Vitro Diagnostics Regulation (for histopathology-based approaches) in the European Union. The European Commission has recently agreed on the AI Act, which establishes a systematic regulation of AI applications based on risk assessment. This regulation aims to ensure the safety of AI applications, including those used in health care. Only very few AI methods from academic research have successfully transitioned, or are close to transitioning, into clinical routine. This observation is not necessarily alarming—after all, only very few molecular biology studies are transitioning to approved pharmaceuticals for clinical use. However, it would be clearly desirable to have a higher number of academic AI methods ultimately implemented for the benefit of patients. Diseases such as breast cancer or prostate cancer have seen clinical approval of many AI methods in the last 5 years, while AI for HCC lags behind these other areas. Creating a potent AI application demands meticulous planning, from identifying the core issue, outlining its scope, to data collection, model development, and testing as well as subsequent regulatory approval and clinical evaluation (Figure 1).

FIGURE 1.

Schematic depiction of developing AI medical devices for liver cancer care and implemented examples. The development of AI methods for liver diseases encompasses a comprehensive process that begins with clearly defining the problem. Such problems can range from screening potential liver ailments, identifying specific biomarkers, making precise diagnoses, planning therapeutic interventions, and executing therapy, to predicting prognosis outcomes. To fuel these AI algorithms, data collection becomes pivotal. This involves accumulating the appropriate input data modalities like images from radiology scans, videos from endoscopic procedures, textual patient records, and even multimodal data that combine multiple sources. Following data gathering, model development is undertaken, where algorithms are trained, fine-tuned, and tested against diverse data sets to ensure accuracy and reliability. However, before these AI solutions can be integrated into clinical practice, they must undergo stringent regulatory approval processes to ensure patient safety and efficacy. Such approvals often require comprehensive documentation, evidence of performance, and alignment with medical standards. Once approved, clinical validation is imperative. This step involves testing the AI model in real-world clinical scenarios through controlled clinical trials, closely monitoring for any adverse events, and constantly evaluating the model’s performance. Postimplementation monitoring then ensures that the AI system remains effective and safe in diverse and evolving clinical settings, adapting to new data and feedback for continuous improvements. Abbreviation: AI, artificial intelligence.

To provide a systematic overview of approved AI products, we queried the official FDA database for AI and machine learning (ML)-enabled medical devices.⁵ In the database, 77% of all AI/ML-based devices are listed in radiology, and <1% are listed in the field of gastroenterology. We identified 546 AI/ML devices in gastroenterology, radiology, and pathology. We excluded devices that were out of the scope of HCC care. From the 3 fields included, we extracted all devices that contained the keywords “liver*,” “hepa*,” or “HCC” and identified 10 devices. In addition, we searched all 546 product summaries for these keywords and identified 55 devices, with the exception of 7 as there was no direct connection to liver cancer care. In total, our search yielded 48 FDA-approved AI solutions (Supplemental Table S1, http://links.lww.com/XCL/A11). Thirty-three devices use a single data modality, and 15 devices are multimodal. Medical image management and processing systems dominate the market. Of the 48 devices, 34 analyze MRI, 25 analyze CT, 5 analyze positron emission tomography, 1 for single-photon emission computed tomography, 2 for not other specified nuclear medicine input data, 3 for ultrasound, and 1 for computed radiography and for x-ray angiography (Figure 2A). Only 1 device for pathology was identified, which quantifies the tumor similarity to a template of 15 cancer types with the RNA expression pattern from tumor specimens.

FIGURE 2.

FDA 510(k) cleared AI-based products for liver cancer care depicted by input data modality (A) and potential application category (B). The majority of products available take MRI as input data modality, followed by CT imaging (A). We grouped the products in potential application categories in liver cancer care and found the majority of products to be useful for diagnosis and therapy planning (B). Abbreviations: CR, computed radiography; FDA, Food and Drug Administration; PET, positron emission tomography; SPECT, single-photon emission computed tomography.

We categorized devices into prevention and risk group identification, diagnostics, therapy planning, and treatment (Figure 2B, Supplemental Table S1, http://links.lww.com/XCL/A11). Resonance Health Analysis Services has introduced HepaFatSmart, an AI-driven solution for quantifying liver fat using MRI data, which could ease the identification of patients at risk for HCC on the basis of metabolic dysfunction–associated fatty liver disease.⁶ Similarly, Shanghai United Imaging Intelligence’s uMR 680 and uMR Omega offer AI-enhanced MR-based liver spectroscopy for the same purpose. FerriSmart by Resonance Health utilizes AI to measure liver iron concentration, while their LiverSmart device integrates both fat and iron quantification technologies. Sonic Incytes’ HepaVelacur leverages DL for precise liver segmentation and enhanced shear wave measurements, critical for diagnosing liver diseases that could progress to cirrhosis and elevate HCC risk.^7,8

It is noteworthy that the devices we have cataloged are primarily utilized in diagnostics and therapy planning, highlighting a notable deficiency in other application areas. These include, among others, prediction, prevention, digital and drug companion therapeutics, lifestyle adaptation, disease management, and presurgical and postsurgical rehabilitation. AI can improve liver anatomy depiction and segmentation, providing support during ultrasound examinations with systems like GE Medical Systems’ Versana Balance. For CT and MRI, a suite of AI tools including GE Medical Systems’ Hepatic VCAR, Perspectum’s LiverMultiScan and Hepatica V1, and Canon Medical Systems Corporation’s Vantage series, streamline the process. Fujifilm Corporation’s Synapse 3D is particularly versatile, accepting a range of input data types such as CT, MRI, computed radiography and ultrasound, nuclear medicine, positron emission tomography, and x-ray angiography. In addition, MIM Software’s Contour ProtegeAI contributes to organ contouring and Siemens Medical Solutions’ Magnetom series allows an improved display of internal structures by constructing additional anatomical planes from MRI data. Change Healthcare Canada’s Anatomical AI significantly speeds up the identification of regions of interest by providing detailed anatomic descriptors and optimizing the diagnostic workflow. Hepatic VCAR aims at segmentation, vessel analysis, visualization, and quantitative evaluation of liver anatomy. All tools designed for segmentation and anatomical representation offer the capability to forecast the size of the future liver remnant, thereby aiding in identifying patients suitable for resection or radiation segmentectomy.⁹ Furthermore, these tools facilitate the easier identification of anatomically adjacent structures, which is crucial in assessing a patient’s eligibility for ablation procedures.

Additional applications offering segmentation capabilities are predominantly designed for radiation therapy planning. These include Siemens’ Syngo.Via RT Image Suite, which performs tumor segmentation using CT, MRI, or positron emission tomography scans; DeepVoxel’s DV.Target; Carina Medical’s INT Contour; MVision AI’s Segmentation; Radformation’s AutoContour Model; Limbus AI’s Limbus Contour; Therapanacea SAS’s ART-Plan; and Manteia Technologies Co., Ltd’s MOZI TPS. This is also of particular relevance for HCC, as selective internal radiation therapy (TARE,Y90) is increasingly utilized in HCC care, as both a bridging therapy resulting in excellent local tumor control and overall survival, as well as in more advanced cases when used in combination with immunotherapy to synergize overall treatment effect.¹⁰ Different radiation therapy options including selective internal radiation therapy but also stereotactic body radiation therapy are still used as treatment alternatives for local tumor control when other alternatives have been exhausted or are not feasible because of patient comorbidities or anatomical and functional characteristics.¹¹

GE’s FlightPlan for Liver is tailored to support embolization therapy planning, providing a depiction of liver vasculature from various imaging modalities. Arterys Oncology from Aretrys Inc. detects and characterizes lesions. Similarly, Ezra AI’s Ezra Plexo software provides detection, quantification, evaluation, and documentation of lesions. Perspectum Diagnostics’ MRCP+ offers a 3-dimensional representation of the biliary system, enabling regional volumetric analysis. In addition, GE’s Xeleris V Processing and Review System includes the Q.Liver application, which provides preparatory assistance for selective internal radiation therapy, such as liver segmentation, estimation of the liver-to-lung shunt value, and calculation of the body surface area. Finally, Canon Medical Systems Corporation’s Aquilion Exceed LB and Cartesion Prime, along with Shanghai United Imaging Healthcare’s DeepRecon, aim to use AI to provide clearer imaging, which could improve the accuracy of HCC detection. The majority of these applications attribute DL as the foundational technology behind their devices as described in the FDA product summaries.⁵ However, many manufacturers typically do not disclose detailed information about the specific technologies publicly. Over a third of AI/ML-based medical devices evolved from non-AI/ML first-generation devices,¹² others evolved from AI/ML-based medical devices (Figure 3). Radiology devices often changed to AI-guided tasks, initiating discussions on safety concerns.¹²

FIGURE 3.

FDA 510(k) cleared AI-based products for liver cancer care per year of clearance. (A) The FDA 510(k) clearance is a process that allows medical device manufacturers to legally market their products in the United States by demonstrating that their new device is “substantially equivalent” to a predicate device already on the market. A predicate device serves as a benchmark for comparison regarding intended use and technological characteristics. (B) Over time, medical devices often evolve, with manufacturers making iterative changes and improvements that lead to new 510(k) clearances for each version, even though they may be part of an original device’s lineage. (B) Depicts an increase in products with ML/AI applicable to liver diseases and HCC in recent years. Some of these devices have already been developed and serve as benchmarks (predicate devices) for newer devices. The number of products (y axis) is constantly rising starting from 2008 with the first product applying AI. Abbreviations: AI, artificial intelligence; FDA, Food and Drug Administration; ML, machine learning.

In summary, despite these numerous approved devices, there is a substantial lack of AI/ML methods in HCC care outside of radiology image processing. The use of clinical data has been crucial in enhancing AI applications in health care.¹³ As the availability of health care data expands, there is a clear rationale for the utilization of wider data types than radiology imaging data alone for the training of application of AI models, and there is increasing interest in the development of multimodal models.^14,15 This interest is driving the development of multimodal AI models, which promise not only improved predictive capabilities but also deeper insights into disease mechanisms. To date, tumor morphology, studied primarily through imaging data like radiology and histopathology, has been a key prognostic factor.^13,16 However, challenges remain, particularly in interpreting and generalizing results from diverse data sources.¹³

The FDA AI/ML-enabled Medical Devices database⁵ and the FDA product classification database¹⁷ provide some details about devices but are not optimally structured for clinical practitioners. Clinicians would benefit substantially from an enhanced search capability that filters devices by indication and application, thereby facilitating the selection of appropriate tools for distinct patient groups in clinical routines. Notably, for AI/ML-enabled devices approved in the EU, no accessible general structured database exists. EUDAMED is a medical device product database that is not fully operational, requires technical expertise to search, and does not allow searching based on whether devices are enabled by AI/ML. Furthermore, regulatory approval is not the final goal for Medical Devices, but these have to be reimbursed, evaluated in clinical trials, and ultimately be broadly implemented in clinical routine. To bridge the gap, close cooperation between medical practitioners, industry, and other stakeholders could help to identify areas in which these devices can be most beneficial and can be implemented.

TEACHING POINTS

• Deep learning excels in handling complex medical data sets, crucial also for liver cancer care.

• AI-enabled devices in HCC care require FDA clearance or approval and, in the EU, CE-marking after assessment.

• Medical image management and processing systems, predominantly using MRI and CT data, dominate the AI-enabled device market in liver cancer care.

• AI tools in liver cancer are mainly used for diagnosis and therapy planning, revealing a significant gap in other application areas.

• Exploiting the increasing volume of health care data can enhance AI models, broadening their utility across different stages of the journey of a patient with liver cancer.

• Improving the FDA database structure could aid in selecting the most suitable AI tools for specific patient groups.

Supplementary Material

xcl-23-e0136-s001.docx ^{(16.4KB, docx)}

CONFLICTS OF INTEREST

Stephen Gilbert owns stock in, is employed by, consults for, and advises Ada Health GmbH. He consults for and advises Flo Ltd. He consults for Una Health GmbH, Lindus Health Ltd, Thymia Ltd, and High-Tech Gründerfonds Management GmbH. He is on the speakers’ bureau for FORUM Institut für Management GmbH. He declares a nonfinancial interest as an Advisory Group member of the EY-coordinated “Study on Regulatory Governance and Innovation in the field of Medical Devices” conducted on behalf of the DG SANTE of the European Commission. He is the coordinator of a Bundesministerium für Bildung und Forschung (BMBF) project (Personal Mastery of Health & Wellness Data, PATH) on consent in health data sharing, financed through the European Union NextGenerationEU program. Jakob N. Kather owns stock in StratifAI GmbH, Germany. He consults for Owkin, France; DoMore Diagnostics, Norway, Panakeia, UK; Scailyte, Switzerland; and Histofy, UK. He is on the speakers’ bureau for AstraZeneca, Bayer, Eisai, MSD, BMS, Roche, Pfizer, and Fresenius. The remaining author has no conflicts to report.