Improving Adjuvant Liver-Directed Treatment Recommendations for Unresectable Hepatocellular Carcinoma: An Artificial Intelligence–Based Decision-Making Tool

PURPOSE

Liver-directed therapy after transarterial chemoembolization (TACE) can lead to improvement in survival for selected patients with unresectable hepatocellular carcinoma (HCC). However, there is uncertainty in the appropriate application and modality of therapy in current clinical practice guidelines. The aim of this study was to develop a proof-of-concept, machine learning (ML) model for treatment recommendation in patients previously treated with TACE and select patients who might benefit from additional treatment with combination stereotactic body radiotherapy (SBRT) or radiofrequency ablation (RFA).

METHODS

This retrospective observational study was based on data from an urban, academic hospital system selecting for patients diagnosed with stage I-III HCC from January 1, 2008, to December 31, 2018, treated with TACE, followed by adjuvant RFA, SBRT, or no additional liver-directed modality. A feedforward, ML ensemble model provided a treatment recommendation on the basis of pairwise assessments evaluating each potential treatment option and estimated benefit in survival.

RESULTS

Two hundred thirty-seven patients met inclusion criteria, of whom 54 (23%) and 49 (21%) received combination of TACE and SBRT or TACE and RFA, respectively. The ML model suggested a different consolidative modality in 32.7% of cases among patients who had previously received combination treatment. Patients treated in concordance with model recommendations had significant improvement in progression-free survival (hazard ratio 0.5; P = .007). The most important features for model prediction were cause of cirrhosis, stage of disease, and albumin-bilirubin grade (a measure of liver function).

CONCLUSION

In this proof-of-concept study, an ensemble ML model was able to provide treatment recommendations for HCC who had undergone prior TACE. Additional treatment in line with model recommendations was associated with significant improvement in progression-free survival, suggesting a potential benefit for ML-guided medical decision making.

INTRODUCTION

Hepatocellular carcinoma (HCC) is the third leading cause of cancer-related mortality and one of the most commonly diagnosed cancers worldwide.¹ In the United States, the incidence of HCC has tripled since the 1980s, with more than 40,000 new cases in 2020² despite availability of screening and improvements in the treatment of underlying causative risk factors.

CONTEXT

• Key Objective

• Liver-directed therapy after transarterial chemoembolization (TACE) can lead to improvement in survival for selected patients with unresectable hepatocellular carcinoma (HCC); however, there is uncertainty in the appropriate application and modality of therapy in current clinical practice guidelines. This study developed a proof-of-concept, machine learning (ML) decision support tool to assist in treatment recommendations for patients previously treated with TACE and select patients who may benefit from additional treatment with combination stereotactic body radiotherapy or radiofrequency ablation.

• Knowledge Generated

• An ensemble ML model was able to identify patients with HCC who may be candidates for combination stereotactic body radiotherapy or radiofrequency ablation after TACE. Patients treated in concordance with ML recommendations had improved progression-free survival.

• Relevance

• Given the lack of guideline consensus in optimal therapy and vast clinical heterogeneity of patients with HCC, ML support tools may provide complementary clinical value to approximate progression-free survival for different treatment modalities.

Management of HCC is complex with several viable treatment approaches, including surgery, catheter-based therapies (chemoembolization [TACE] and radioembolization [TARE]), external beam radiation therapy (EBRT) including stereotactic body radiotherapy (SBRT),³ and radiofrequency thermal ablation (RFA).⁴ Surgical resection, liver transplantation, and thermal ablation are standard treatment modalities for curative intent. Unfortunately, many patients with HCC are not candidates for liver transplantation or surgery, and use of locoregional therapies may be limited because of compromised liver function from underlying cirrhosis.

Liver-directed therapy (LDT) with TARE, TACE, RFA, or EBRT is an alternative treatment option for local tumor control. The American Society for Radiation Oncology (ASTRO) has recently proposed recommendations for the role of radiation in HCC for first-line treatment in nonsurgical candidates and in the clinical context of consolidative therapy after incomplete response to LDT.⁵ However, selection of appropriate LDT can be quite challenging because of patient heterogeneity in baseline liver function and tumor location and size. Furthermore, the benefits of retreatment with combination therapy (typically involving EBRT or RFA after TACE) need to be weighed against risks of further liver function compromise. There may be some guidance as to the most appropriate therapy on the basis of the forthcoming results from the phase III RTOG 1112 trial evaluating the benefit of SBRT in unresectable HCC; however, other large randomized evidence is lacking.⁶ Observations from real-world data suggest that consolidative combination therapy with SBRT is promising^7,8 although underutilized,⁹ in part because of a lack of consensus for LDT selection. In current practice, selection of therapy is often based on multidisciplinary evaluation. Given the potential for subjective and inconsistent recommendations from multidisciplinary discussions, artificial intelligence approaches with machine learning (ML) algorithms have the potential to provide improved predictive guidance in the determination of optimal intervention strategies for each patient.

In this study, we analyzed an expanded subset of a previously reported¹⁰ institutional cohort of HCC treated with TACE with and without consolidative SBRT or RFA. The overall aim is to assess the feasibility for a ML model to predict survival in the definitive management of patients previously treated with TACE, with the goal of identifying patients who may benefit from a specific modality of LDT using SBRT or RFA.

METHODS

Study Design and Data Source

This was a retrospective institutional cohort study evaluating the ability of a deep learning model to assist in treatment recommendations for patients with HCC who had previously received TACE as the sole modality for local therapy. At our institution, all patients with HCC are presented in a multidisciplinary tumor board, and treatment management is decided after discussion among providers from different specialties. The institutional review board–approved study was exempted from informed consent requirements because it was a retrospective medical record review of existing data among patients already treated at our institution. This study followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline for cohort studies.¹¹

Study Population and Covariates

We included patients diagnosed from January 1, 2008, to December 31, 2018, with stage I-III HCC who were treated with TACE. We excluded patients with nodal or metastatic disease at baseline or who underwent any liver-directed therapy (eg, surgical resection, microwave ablation, or radioembolization) aside from TACE, RFA, or SBRT. Patients who underwent both RFA and SBRT were also excluded.

Disease-specific factors included T stage, number of lesions, and location and size of the largest lesion. Treatment-specific factors included the number of prior TACE treatments, receipt of sorafenib, and treatment with SBRT or RFA. Clinical characteristics were applied to a feedforward ML neural network model to facilitate treatment recommendations, using individual pairwise models to evaluate each potential treatment in a logistic fashion before an ensemble evaluation (Fig 1).

FIG 1.

Schematic of the ML model approach. The ensemble ML approach uses a pairwise DeepSurv model, which is a configurable feedforward deep neural network composed of several hidden layers consisting of fully connected functions followed by dropout layers, which is trained on hundreds of clinical cases. The final layer is a single node, which performs a linear combination of the hidden features and calculates the recommendation which yields the largest improvement on hazard risk and outputs an ensemble treatment recommendation. ALBI, albumin-bilirubin; MELD, model for end-stage liver disease; RFA, radiofrequency ablation; SBRT, stereotactic body radiotherapy.

Model Development

The multivariable Cox proportional hazards model for overall survival (OS) using all available variables (clinicodemographic characteristics and disease-specific factors) was developed for initial feature determination to select between highly correlated clinical variables (eg, albumin-bilirubin [ALBI] score and grade and Child-Pugh class and grade), on the basis of the largest coefficient magnitude in the hazards model. The features were then applied to the ML model.

DeepSurv¹² is a deep feedforward neural network, which applies deep learning techniques to a nonlinear Cox proportional hazards model. The neural network predicts effects of a patient's covariates on the hazard rate parameterized by the weights of the Cox proportional model and includes modern techniques for deep learning such as weight decay regularization, batch normalization, dropout, learning rate scheduling, and optimizers for gradient descent.¹² DeepSurv models were constructed in Python version 3.10.2 (Python Software Foundation) using the survivalmodels package¹² optimized against time-to-event OS outcome. We used a 50:50 split of data for training and testing, with equal sampling from each treatment group. Optimal hyperparameters for each model were selected and tuned by a random hyperparameter search using the Python package Optunity.¹³ Training treatment groups were first compared in a pairwise fashion using the DeepSurv package. Pairwise comparison groups included the following: no additional treatment versus SBRT, no additional treatment versus RFA, and RFA versus SBRT. Each comparison group uses average negative log-likelihood with L2 regularization as the loss function for the model to output an estimated hazard risk score^12,14 independent of treatment modality (given that modality is not included as a covariate within individual models and the DeepSurv architecture enables model prediction without a priori specification of potential treatment covariate interactions). An optimal strategy was chosen on the basis of an assessment of the greatest improvement in estimated hazard risk score by treatment modality for each patient across all models. Thus, the ensemble model treatment recommendation was selected by the treatment, yielding the best hazard rate from all three pairwise models on a per-patient basis.

Statistical Analysis

Statistical analysis was performed in Python 3.10.2; code used for model development and evaluation has been previously reported and is available online.¹⁵ The primary outcome was either progression-free survival (PFS) or OS benefit associated with concordant treatment according to an ensemble model recommendation. We calculated the hazard ratio, PFS, OS, and significance using the log-rank test for receipt of treatment in concordance with model recommendations. We expected our data set to be imbalanced between those receiving combination therapy, compared with those receiving TACE alone, and those with better baseline liver function receiving more modalities of local ablative treatment. Model accuracy was compared using the concordance index (C index).¹⁶ Covariates important to ensemble model prediction were evaluated by the root mean square of z-normalized covariate coefficients of pairwise models. For univariate analysis, the χ² test for categorical variables and Student's t test for continuous variables were used. All comparisons used an alpha value of .05 for significance, and all statistical tests were two-sided. All data analysis was performed using SPSS version 24 (IBM Corporation) or Python version 3.10.2 (Python Software Foundation).

RESULTS

A total of 237 patients (138 [73%] males; 184 [78%] age < 70 years) met our inclusion criteria, of whom 54 (23%) received combination of TACE and SBRT and 49 (21%) received TACE and RFA, respectively. Approximately three quarters of patients (184 [78%]) had a history of hepatitis C infection. The median follow-up was 31 months, and the median OS was 5.7 years (95% CI, 3.6 to 7.8). Demographic, disease, and treatment characteristics of the patients, stratified by receipt of TACE alone or combination of TACE and SBRT or RFA, are summarized in Table 1. Most cancers were stage I (49%) or II (45%), with an average largest lesion size of 3.2 (range of 0.8-12.2) cm. Receipt of combination of TACE and SBRT or RFA was significantly associated with better baseline liver function with lower ALBI grade, Child-Pugh class, and model of end-stage liver disease scores. In addition, patients who underwent SBRT or RFA were associated with a lower stage cancer, fewer and smaller lesions, and fewer treatments of sorafenib.

TABLE 1.

Patient Demographic, Disease, and Treatment Characteristics

After initial Cox proportional hazards models using all available variables, two variables were removed (Child-Pugh grade, ALBI score), in favor of Child-Pugh class and ALBI grade for downstream ML model generation.

Our models were trained on 118 cases, with the remaining 119 reserved for validation. Results of the optimal hyperparameter search for each pairwise models are shown in Table 2. With a median follow-up in the validation set of 29.8 (4.9-152.1) months, treatment according to the ML model recommendations was associated with significantly improved PFS and a trend in benefit for OS (Fig 2), with hazard ratios of 0.50 (95% CI, 0.31 to 0.83; P = .007) for PFS and 0.63 (95% CI, 0.34 to 1.2; P = .16) for OS. Of the 52 patients who had received combination LDT after TACE, the ensemble model recommended an alternative liver-directed therapy in one third of cases. In patients who received combination of TACE and SBRT, RFA was recommended 36% of the time. Among patients who received combination of TACE and RFA, SBRT was recommended 29.5% of the time. Patients receiving recommended treatment had a mean OS of 7.5 years (95% CI, 5.6 to 9.4) compared with 5.3 years (95% CI, 4.1 to 6.3) among patients who did not receive recommended treatment. For patients receiving recommended treatment, PFS was significantly (P < .001) improved with a mean PFS of 5.3 years (95% CI, 3.4 to 7.3) for patients who received the recommended treatment compared with 1.8 years (95% CI, 1.2 to 2.3). Accuracy for all models was fairly robust with the average C index of 0.727 in the training set and 0.602 in the testing set.

TABLE 2.

DeepSurv Pairwise Optimal Hyperparameters

FIG 2.

Survival outcomes on the basis of concordance with ML treatment recommendations in the test cohort trained on OS. (A) The significant difference (HR 0.50, P = .007) in PFS between patients who were concordant with the ML recommendation and those that were not. ML models were trained on time-to-event OS data in the test cohort. (B) The difference in OS between the recommendation groups. HR, hazard ratio; ML, machine learning; OS, overall survival; PFS, progression-free survival; TACE, transarterial chemoembolization.

We identified features important to ensemble model accuracy by first evaluating the magnitude of covariates in each pairwise model and taking the root mean square to determine critical features in the ensemble model (Fig 3). In the ensemble model, the three most important features were cause of cirrhosis (typically hepatitis C), followed by stage, and an assessment of liver function by ALBI grade. Sex, number of liver lesions, evaluation of liver function by model of end-stage liver disease or Child-Pugh score, and location of the dominant lesion within the liver were also significant, although less critical, in ensemble model treatment recommendation.

FIG 3.

Plot of Z-normalized covariates in pairwise machine learning models. Plot of magnitude of normalized coefficients of model covariates. Each column represents a pairwise model, with the right most common composed of the root mean square of model coefficients. Values that have a magnitude more than two are considered significant in determining predictive accuracy of the ensemble deep learning model, with larger numbers denoting greater feature importance. ALBI, albumin-bilirubin; MELD, model of end-stage liver disease; RFA, radiofrequency ablation; SBRT, stereotactic body radiotherapy; TACE, transarterial chemoembolization; Tx, treatment.

DISCUSSION

ML tools have been instrumental in revolutionizing several industries with diverse applications in economics, linguistics, and the physical sciences, including oncologic diagnosis and management. Thus far, prospective uses include determining at-risk patients undergoing active cancer treatment for mid-treatment preventive intervention¹⁵ and identifying extranodal extension on pretreatment computed tomography imaging for head and neck cancer.¹⁷ Recent developments of survival models have facilitated novel approaches in ML to evaluate clinical treatment decisions¹⁸ through assessment of individual treatment effects.¹⁹ Previous attempts to leverage ML in guiding tumor board recommendations for HCC have been hampered primarily because of small sample sizes and lack of measurable outcome information to quantify accuracy and clinical benefit.²⁰ Here, we demonstrate an ensemble, proof-of-concept, ML model that demonstrates the feasibility of identifying patients who can safely benefit from therapy after TACE and furthermore determine which therapy is most suitable.

In current practice, the decision to treat HCC with locoregional treatment is typically best determined in a multidisciplinary setting. TACE is one of the most widely used modalities in initial management but has rates of treatment failure as high as 43%.²¹ The utilization of SBRT or RFA for consolidative treatment to improve local control and OS has been demonstrated in several studies^22-28 and is the subject of several ongoing trials.^29,30 However, given significant underlying liver disease among most patients with HCC, combination treatment with SBRT or RFA must be weighed against liver toxicity³¹ and their differing side effect profiles and costs.^32-35 Our putative model found the stage of disease, liver function by ALBI, and cause of liver cirrhosis to be the most important factors with regard to predictive accuracy of consolidative treatment for survival. For patients who had previously received surveillance after initial TACE, our ML model was consistent in recommending combination local modalities for PFS benefit. Our institution¹⁰ along with other groups^25,29 has previously demonstrated the potential advantage of additional ablative treatment modalities after TACE, which included patients with advanced tumors and tumor with complete response on imaging after initial TACE. In addition, the model recommended an alternative combination treatment in approximately one third of patients. Previous literature has suggested similar survival and local control data with both RFA and SBRT, especially for small tumors.³² However, SBRT may offer a potential comparable advantage in local control for larger tumors (> 2 cm) depending on an amenable treatment location, including subphrenic lesions near the diaphragm, or near the biliary tree, which are problematic for RFA.^33,36 By contrast, RFA may be beneficial in smaller lesions given the generally smaller ablative volume from RFA and then the irradiated volume from SBRT.^36,37 Given the highly significant improvement in PFS on the basis of concordance with treatment recommendation, the DeepSurv ensemble model could serve as a useful clinical decision tool in patients who have undergone TACE.

There are several limitations to our current study. The analysis only evaluates three potential treatments (two combination options, TACE with and without SBRT or RFA), which may limit its generalizability to other potential modalities of liver-directed therapy. Although we accounted for confounders in our data set, it is recommended that models are validated with an external independent data set to demonstrate generalizable and robustness. Furthermore, the recommendation of consolidative SBRT and RFA is limited by potential technical limitations on the basis of patient anatomy, which cannot be fully accounted for solely the size and location of the target lesion. There is therefore a bias toward recommendation for additional adjuvant therapy given the minimal number of adverse events in the training data.

Overall, we have demonstrated the feasibility of applying an ensemble ML model to identify patients with HCC who may be candidates for consolidative SBRT or RFA after TACE. Further analysis will incorporate additional modalities of liver-directed therapy to optimize the selection of appropriate treatment with the aim of maximizing survival while minimizing toxicity. In the future, we anticipate deploying this tool in a prospective trial setting where medical treatment decision making for complex clinical cases may be enhanced through personalized ML recommendations.

AUTHOR CONTRIBUTIONS

Conception and design: Allen Mo, Christian Velten, Nitin Ohri, Shalom Kalnicki, Parsa Mirhaji, Boudewijn Aasman, Chandan Guha, Rafi Kabarriti

Financial support: Rafi Kabarriti

Provision of study materials or patients: Nitin Ohri

Collection and assembly of data: Allen Mo, Christian Velten, Julie M. Jiang, Shalom Kalnicki, N. Patrik Brodin, Rafi Kabarriti

Data analysis and interpretation: Allen Mo, Christian Velten, Justin Tang, Nitin Ohri, Shalom Kalnicki, Kei Nemoto, Madhur Garg, N. Patrik Brodin, Rafi Kabarriti

Manuscript writing: All authors

Final approval of manuscript: All authors

Accountable for all aspects of the work: All authors

AUTHORS' DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/cci/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).