Abstract
Purpose
To develop and test a machine learning (ML)–based model that integrates preoperative variables for prediction of advanced-stage progression (ASP) after transarterial chemoembolization (TACE).
Materials and Methods
This multicenter retrospective study (ResearchRegistry.com identifier no. researchregistry9425) included patients with intermediate-stage hepatocellular carcinoma (HCC) who underwent TACE at seven hospitals from June 2008 to December 2022. Thirty-four preoperative clinical and CT imaging variables were input into six ML-based models for prediction of ASP, and model performances were compared. Furthermore, the best-performing ML model was compared with the major staging systems, and its utility in performing post-TACE therapies was assessed. The performances of the models were compared by using area under the receiver operating characteristic curve (AUC) with DeLong test. Kaplan-Meier survival curves were compared using the log-rank test.
Results
A total of 2333 eligible patients (mean age, 54 years ± 12 [SD]; 2051 male patients) were categorized into the training set (n = 1026), the internal test set (n = 257), and the external test set (n = 1050). ASP was found in 8.4% (86 of 1026), 8.2% (21 of 257), and 6.7% (70 of 1050) of patients in the three datasets, respectively. Among all ML models, the Categorical Gradient Boosting (CatBoost) model yielded the highest AUC: 0.97 (95% CI: 0.95, >0.99) for the training set, 0.94 (95% CI: 0.92, 0.97) for the internal test set, and 0.93 (95% CI: 0.90, 0.95) for the external test set. Furthermore, it yielded better discriminatory ability with higher concordance indexes than the five staging systems (all P < .001). The time-dependent AUC of the CatBoost model was also higher than that of the clinical staging systems at various time points (all P < .001). Moreover, post-TACE systemic therapy improved progression-free survival and overall survival for patients in the high-risk group (both P < .001) but not in the low-risk group.
Conclusion
The CatBoost model demonstrated higher predictive performance compared with existing staging systems in predicting ASP after TACE in patients with intermediate-stage HCC. This model effectively stratified patients by risk level and identified those who benefited from post-TACE systemic therapy.
Keywords: Liver, Oncology, Transarterial Chemoembolization, Hepatocellular Carcinoma, Advanced-stage Progression, Machine Learning, Risk Differentiation
ResearchRegistry.com identifier no. researchregistry9425
Supplemental material is available for this article.
© RSNA, 2025
See also commentary by Rouzbahani in this issue.
Keywords: Liver, Oncology, Transarterial Chemoembolization, Hepatocellular Carcinoma, Advanced-stage Progression, Machine Learning, Risk Differentiation
Summary
The CatBoost machine learning model accurately predicted advanced-stage progression of intermediate-stage hepatocellular carcinoma after transarterial chemoembolization (TACE) and identified patients who benefited from post-TACE systemic therapy.
Key Points
■ In this retrospective study of 2333 patients with intermediate-stage hepatocellular carcinoma who underwent transarterial chemoembolization (TACE), the Categorical Gradient Boosting (CatBoost) model including 11 clinical and CT imaging variables yielded area under the receiver operating characteristic curve values of 0.94 (internal test set) and 0.93 (external test set) for predicting advanced-stage progression (ASP), which occurred in 6.67%–8.37% of patients.
■ The CatBoost model yielded higher concordance index values (0.82, 0.70, and 0.75 on the training, internal test, and external test sets, respectively) than five clinical staging systems (all P < .001).
■ Post-TACE systemic therapy improved progression-free and overall survival (both P < .001) in the group at high risk of ASP.
Introduction
Hepatocellular carcinoma (HCC) accounts for 75%–85% of liver cancer and is the fourth leading cause of cancer deaths globally, with a dismal prognosis (1–3). Currently, transarterial chemoembolization (TACE) is the primary treatment for intermediate-stage HCC (4,5). However, the high recurrence rate after TACE (5-year recurrence rate: 54%–63%) remains a major obstacle to long-term survival (6,7). Notably, vascular invasion and extrahepatic metastasis after TACE in intermediate-stage HCC are two types of recurrence that are worse than intrahepatic recurrence and are referred to as advanced-stage progression (ASP), which indicates patients progress to Barcelona Clinic Liver Cancer (BCLC) stage C and limits the choice of curable treatment options (8). In recent years, molecular targeted agents and immune checkpoint inhibitors have been widely used for advanced-stage HCC and have been proven to be effective in improving patient survival (9–11). Evidence from randomized controlled trials and retrospective studies has implied that patients with HCC who underwent TACE at high recurrence risk might benefit from post-TACE systemic therapies such as sorafenib, lenvatinib, programmed cell death-1 inhibitors or programmed cell death ligand-1 inhibitors, plus molecular targeted therapies (12–15). Therefore, identifying ASP risk stratification for these patients may be useful for individualized decision-making leading to survival benefit.
Previously, histopathologic subtypes, radiologic features, and tumor burden have been associated with increased risk of metastasis (16–19). Moreover, researchers have attempted to establish predictive models using laboratory tests, tumor burden, and imaging features. A model integrating neutrophil count, tumor diameter, and hyperenhancement proportion in the arterial phase for prediction of advanced-stage recurrence after resection demonstrated better performance than current staging systems (8). However, due to limitations such as small sample sizes, varying radiologic scoring standards, and inadequate predictive ability, current models for recurrence prediction before TACE still urgently need improvement.
Machine learning (ML) is a promising approach that employs statistical, probabilistic, and optimization techniques to train a machine (20–22), with outstanding advantages such as standardized processing of large-scale multicenter data, interpolation of missing values, and efficient computing power. Logistic regression is a traditional supervised ML algorithm and is commonly used in the medical field. Previous studies demonstrated that three ML models outperformed logistic regression in predicting recurrence after microwave ablation of HCC (23). Moreover, we evaluated the ability of five other ML algorithms, eXtreme Gradient Boosting, Categorical Gradient Boosting (CatBoost), Gradient Boosting Decision Tree, Light Gradient-Boosting Machine, and random forest, to accurately stratify prognostic risk for patients with HCC who underwent intra-arterial therapies (24). In summary, these supervised ML algorithms predicted HCC recurrence and aided in decision-making with minimal interference from subjective factors.
To date, there is no noninvasive, convenient, and accurate model for guiding physicians in therapeutic decision-making for TACE. This study aimed to develop and test an ML model based on clinical data for predicting ASP before TACE. Furthermore, we compared this ML model with major staging systems that were well acknowledged and widely used (BCLC, European Association for the Study of the Liver, China Liver Cancer, Japan Society of Hepatology, and Hong Kong Liver Cancer) and assessed its utility in providing post-TACE therapies.
Materials and Methods
This multicenter, retrospective study was registered at ResearchRegistry.com (identifier no. researchregistry9425) and approved by the Ethical Review Committee of Sun Yat-sen University Cancer Center (approval no. B2022-694), which served as the lead ethics committee for all participating centers, with a waiver of the requirement to obtain written informed consent. The study was conducted following the 1975 Declaration of Helsinki and the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines (25).
Study Sample
For ML model development, consecutive patients with HCC who underwent initial TACE from June 2008 to December 2022 were reviewed retrospectively at seven tertiary hospitals in China. The diagnosis was established based on either pathology or clinical criteria of the American Association for the Study of Liver Diseases guidelines (4). The inclusion criteria for these patients were age 18–75 years; intermediate-stage HCC defined by the BCLC staging system; no vascular invasion or extrahepatic spread; candidates for TACE based on the European Association for the Study of the Liver guidelines (26); Eastern Cooperative Oncology Group performance status less than 2; Child-Pugh class A or B; and underwent CT or MRI within 2 weeks before TACE. Exclusion criteria were systemic treatment before or after TACE, any other current or previous malignancy, decompensated liver cirrhosis, missing image information, or lack of any follow-up information.
Patients from the Sun Yat-sen University Cancer Center were assigned to the derivation set, while patients from the other six centers were assigned to the external test set. Patients in the derivation set were randomly divided into the training set and the internal test set in a 4:1 ratio using the ML algorithm (fivefold cross-validation). The TACE procedure is described in Appendix S1.
To test the established ML model’s ability in identifying patients with HCC who might benefit from post-TACE systemic therapies, patients who underwent TACE and post-TACE therapies and met the aforementioned inclusion and exclusion criteria (except for receiving systemic treatment after TACE) were retrospectively included from June 2014 to December 2020 at Sun Yat-sen University Cancer Center. Patients who underwent TACE alone in the internal test set were used as the comparison group for propensity score matching (PSM) analysis. Post-TACE therapy was selected based on multidisciplinary discussions as described in Appendix S2.
A total of 34 clinical and imaging variables including demographic variables, tumor features, and laboratory findings before TACE were collected (Table S1). The CT scan protocol is described in Appendix S3. Two specialized abdominal radiologists with more than 5 years of hepatic imaging expertise from each participating center independently analyzed pre-TACE contrast-enhanced CT images. In cases of disagreement, a third abdominal radiologist with more than 10 years of hepatic imaging expertise adjudicated to resolve the discrepancy. While aware of the HCC diagnoses, the radiologists remained blinded to other clinical, histopathologic, therapeutic, or follow-up data. Imaging features were assessed on a per-patient basis, with the dominant lesion evaluated in cases of multifocal disease. Based on previous studies, we selected eight typical CT imaging features to develop the ML model (27). Detailed definitions of included CT features are summarized in Table S2, and examples of CT features are shown in Figure S1.
Follow-up and End Points
After initial TACE, all patients were monitored regularly at 1 and 3 months postoperatively for serum α-fetoprotein (AFP) levels and imaging examinations with intervals of 3–6 months until death or the date of last follow-up. The predictive window was the time from initial TACE to ASP. The primary end point was advanced-stage progression-free survival (ASPFS), defined as the time from initial TACE to ASP or last follow-up. Secondary end points were progression-free survival (PFS, defined as the time from initial TACE to intrahepatic tumor recurrence, tumor-in-vein, metastasis as per American Association for the Study of Liver Diseases, or all-cause death) and overall survival (OS, defined as the time from initial TACE to death from any cause or last follow-up).
Development of the ML Model
Six supervised ML algorithms, namely eXtreme Gradient Boosting, CatBoost, Gradient Boosting Decision Tree, Light Gradient-Boosting Machine, logistic regression, and random forest, were selected to develop the optimal ML model for predicting ASP, with detailed information provided in Appendix S3 and parameters presented in Table S3. Among 34 collected variables, those with a missing data rate of greater than 20% were excluded. Next, all included variables were trained for feature importance ranking to optimize the ML model. Based on recursive feature elimination with cross-validation, the three least important feature elimination processes were repeatedly performed to assess the discrimination ability of the features and rank all features by importance. Then the areas under the receiver operating characteristic curves (AUCs) of each ML model with different numbers of features were calculated and compared to determine the included features. Finally, based on the AUCs, six optimal ML models (each included a number of features) were compared to identify the most optimal ML model. The SHapley Additive exPlanations (SHAP) method was used to explain the predictive ability of the ML model, with the Shapley value of each variable being based on game theory to determine the relative importance of each variable (28,29).
Statistical Analysis
Statistical analysis was performed using SPSS version 23.0 and R software version 3.5.3 (http://www.r-project.org/). The cumulative survival outcomes of different groups were estimated using the Kaplan-Meier method and compared with the log-rank test after checking the proportional hazards assumption, and subgroup analyses were performed to adjust for known prognostic factors. For PSM analysis, the propensity score was estimated by logistic regression using a 1:1 ratio for optimal pair matching with a goal-to-achieve standardized mean difference less than 0.10. Patients were subjected to 1:1 PSM using the nearest neighbor method with a caliper of 0.1.
Based on the internal test set, the ML model’s discrimination for ASP was compared using the AUCs with the DeLong test using MedCalc version 19.7 (https://www.medcalc.org). The performance of the ML model for predicting ASPFS was measured with concordance index and time-dependent AUC from 12 to 60 months. Model prediction error was assessed by computing the integrated Brier score, and clinical utility was evaluated with decision curve analysis. The ML model was compared with major clinical staging systems using the Z test. To classify patients into high-risk and low-risk groups, optimal thresholds of the ML models were determined with X-tile software (version 3.6.1).
All tests of significance were two-sided, and a P value less than .05 was considered statistically significant.
Results
Patient Characteristics
The study design is detailed in Figure 1. From June 2008 to December 2022, 4812 consecutive patients with HCC who underwent TACE at Sun Yat-sen University Cancer Center were screened, of whom 1283 were included in the derivation set (Table S4). A total of 3529 patients were excluded due to the following: underwent treatment before TACE (n = 874), had a history of other malignancies (n = 382), or inadequate image quality for reliable assessment (n = 2273). In a ratio of 4:1, 1026 were assigned to the training set and 257 to the internal test set. Meanwhile, 2896 patients at the other six hospitals were screened, of whom 1050 were included in the external test set (Table S4). A total of 1846 patients were excluded due to the following: underwent treatment before TACE (n = 389), had a history of other malignancies (n = 212), or inadequate image quality for reliable assessment (n = 1245). In summary, a total of 2333 patients were included. Tables 1–3 detail the patients’ baseline characteristics. The male to female ratio was 2051 to 282, and the median age was 54 years ± 12 (SD). The correlation of 34 variables is shown on the heatmap (Fig S2). Median follow-up was 31.2 months (IQR: 17.5–55.6 months) for the derivation set and 27.5 months (IQR: 12.8–49.0 months) for the external test set. The incidence of ASP was 8.4% (86 of 1026) for the training set, 8.2% (21 of 257) for the internal test set, and 6.7% (70 of 1050) for the external test set.
Figure 1:
Study design. ASPFS = advanced-stage progression-free survival, CatBoost = Categorical Gradient Boosting, C-index = concordance index, HCC = hepatocellular carcinoma, IBS = integrated Brier score, ML = machine learning, SHAP = SHapley Additive exPlanations, TACE = transarterial chemoembolization, Td-AUC = time-dependent area under the receiver operating characteristic curve.
Table 1:
Demographics and History of Patients with HCC Who Underwent TACE in All Datasets
| Demographics and History (n = 11) | Training Set (n = 1026) | Internal Test Set (n = 257) | P Value | Total (Derivation Set) (n = 1283) | External Test Set (n = 1050) | P Value |
|---|---|---|---|---|---|---|
| Age (y) | <.001 | <.001 | ||||
| ≤65 | 808 (78.8) | 161 (62.6) | 969 (75.5) | 866 (82.5) | ||
| >65 | 218 (21.2) | 96 (37.4) | 314 (24.5) | 184 (17.5) | ||
| BMI | .28 | <.001 | ||||
| <21.3 | 54 (5.3) | 9 (3.5) | 63 (4.9) | 74 (7.0) | ||
| 21.3–22.8 | 402 (39.2) | 112 (43.6) | 514 (40.1) | 731 (69.6) | ||
| >22.8 | 570 (55.6) | 136 (52.9) | 706 (55.0) | 245 (23.3) | ||
| Alcohol consumption | .20 | .006 | ||||
| Absence | 944 (92.0) | 230 (89.5) | 1174 (91.5) | 992 (94.5) | ||
| Presence | 82 (8.0) | 27 (10.5) | 109 (8.5) | 58 (5.5) | ||
| Smoking history | .54 | <.001 | ||||
| Absence | 817 (79.6) | 209 (81.3) | 1026 (80.0) | 924 (88.0) | ||
| Presence | 209 (20.4) | 48 (18.7) | 257 (20.0) | 126 (12.0) | ||
| Sex | .10 | .79 | ||||
| Male | 896 (87.3) | 234 (91.1) | 1130 (88.1) | 921 (87.7) | ||
| Female | 130 (12.7) | 23 (8.9) | 153 (11.9) | 129 (12.3) | ||
| ECOG PS | .79 | <.001 | ||||
| PS 0 | 1001 (97.6) | 250 (97.3) | 1251 (97.5) | 849 (80.9) | ||
| PS 1 | 25 (2.4) | 7 (2.7) | 32 (2.5) | 201 (19.1) | ||
| Comorbidities | .72 | .66 | ||||
| Absence | 910 (88.7) | 230 (89.5) | 1140 (88.9) | 939 (89.4) | ||
| Presence | 116 (11.3) | 27 (10.5) | 143 (11.1) | 111 (10.6) | ||
| HBV | .95 | <.001 | ||||
| Absence | 57 (5.6) | 14 (5.4) | 71 (5.5) | 171 (16.3) | ||
| Presence | 969 (94.4) | 243 (94.6) | 1212 (94.5) | 879 (83.7) | ||
| Cirrhosis | .04 | <.001 | ||||
| Absence | 67 (6.5) | 8 (3.1) | 75 (5.8) | 126 (12.0) | ||
| Presence | 959 (93.5) | 249 (96.9) | 1208 (94.2) | 924 (88.0) | ||
| Ascites | .33 | <.001 | ||||
| Absence | 974 (94.9) | 240 (93.4) | 1214 (94.6) | 1029 (98.0) | ||
| Presence | 52 (5.1) | 17 (6.6) | 69 (5.4) | 21 (2.0) | ||
| Child-Pugh class | .30 | .02 | ||||
| A | 892 (86.9) | 217 (84.4) | 1109 (86.4) | 942 (89.7) | ||
| B | 134 (13.1) | 40 (15.6) | 174 (13.6) | 108 (10.3) |
Note.—Except where indicated, data are numbers of patients, with percentages in parentheses. BMI = body mass index (calculated as weight in kilograms divided by height in meters squared), ECOG = Eastern Cooperative Oncology Group, HBV = hepatitis B virus, HCC = hepatocellular carcinoma, PS = performance status, TACE = transarterial chemoembolization.
Table 3:
Laboratory Findings of Patients with HCC Who Underwent TACE in All Datasets
| Laboratory Findings (n = 13) | Training Set (n = 1026) | Internal Test Set (n = 257) | P Value | Total (Derivation Set) (n = 1283) | External Test Set (n = 1050) | P Value |
|---|---|---|---|---|---|---|
| α-Fetoprotein level (ug/L) | <.001 | <.001 | ||||
| ≤400 | 613 (59.7) | 51 (19.8) | 664 (51.8) | 744 (70.9) | ||
| >400 | 413 (40.3) | 206 (80.2) | 619 (48.2) | 306 (29.1) | ||
| Des-γ-carboxy prothrombin level (mAU/mL) | <.001 | <.001 | ||||
| ≤400 | 527 (51.4) | 45 (17.5) | 572 (44.6) | 678 (64.6) | ||
| >400 | 499 (49.6) | 212 (82.5) | 711 (55.4) | 372 (35.4) | ||
| Albumin level (g/L) | .58 | <.001 | ||||
| ≤35 | 270 (26.3) | 72 (28.0) | 342 (26.7) | 211 (20.1) | ||
| >35 | 756 (73.7) | 185 (72.0) | 941 (73.3) | 839 (79.9) | ||
| Aspartate aminotransferase level (U/L) | .003 | <.001 | ||||
| ≤40 | 286 (27.9) | 48 (18.7) | 334 (26.0) | 437 (41.6) | ||
| >40 | 740 (72.1) | 209 (81.3) | 949 (74.0) | 613 (58.4) | ||
| Alanine aminotransferase level (U/L) | .22 | <.001 | ||||
| ≤40 | 219 (21.3) | 64 (24.9) | 283 (22.1) | 456 (43.4) | ||
| >40 | 807 (78.7) | 193 (75.1) | 1000 (77.9) | 594 (56.6) | ||
| Total bilirubin level (μmol/L) | .30 | <.001 | ||||
| ≤17.1 | 544 (53.0) | 127 (49.4) | 671 (52.3) | 652 (62.1) | ||
| >17.1 | 482 (47.0) | 130 (50.6) | 612 (47.7) | 398 (37.9) | ||
| Platelet count (×109/L) | .39 | <.001 | ||||
| ≤100 | 241 (23.5) | 67 (26.1) | 308 (24.0) | 350 (33.3) | ||
| >100 | 785 (76.5) | 190 (73.9) | 975 (76.0) | 700 (66.7) | ||
| Neutrophils (×109/L) | <.001 | <.001 | ||||
| ≤4.5 | 655 (63.8) | 195 (75.9) | 850 (66.3) | 925 (88.1) | ||
| >4.5 | 371 (36.2) | 62 (24.1) | 433 (33.7) | 125 (11.9) | ||
| Lymphocyte count (×109/L) | .047 | .003 | ||||
| ≤1.5 | 655 (63.8) | 181 (70.4) | 836 (65.2) | 745 (71.0) | ||
| >1.5 | 371 (36.2) | 76 (29.6) | 447 (34.8) | 305 (29.0) | ||
| Prothrombin time (sec) | .12 | .052 | ||||
| ≤13 | 871 (84.9) | 208 (80.9) | 1079 (84.1) | 851 (81.0) | ||
| >13 | 155 (15.1) | 49 (19.1) | 204 (15.9) | 199 (19.0) | ||
| International normalized ratio | .91 | .12 | ||||
| ≤1.01 | 555 (54.1) | 138 (53.7) | 693 (54.0) | 533 (50.8) | ||
| >1.01 | 471 (45.9) | 119 (46.3) | 590 (46.0) | 517 (49.2) | ||
| C-reactive protein level (mg/L) | .92 | <.001 | ||||
| ≤14 | 718 (70.0) | 179 (69.6) | 897 (69.9) | 961 (91.5) | ||
| >14 | 308 (30.0) | 78 (30.4) | 386 (30.1) | 89 (8.5) | ||
| Creatinine level (μmol/L) | .31 | .85 | ||||
| ≤80 | 906 (83.3) | 221 (86.0) | 1127 (87.8) | 925 (88.1) | ||
| >80 | 120 (16.7) | 36 (14.0) | 156 (12.2) | 125 (11.9) |
Note.—Except where indicated, data are numbers of patients, with percentages in parentheses. HCC = hepatocellular carcinoma, TACE = transarterial chemoembolization.
Performance of ML Models and Clinical Staging Systems
We next constructed six ML models for predicting ASP and compared their performance. The variable pathology differentiation was excluded owing to a missing data rate of greater than 20%. Table 2 lists numbers of variables, AUC (Fig S3), sensitivity, specificity, positive predictive value, negative predictive value, and F1 score of six ML models. Their performances for predicting early (within 2 years) or late (after 2 years) ASP are shown in Table S5 and S6. The numerators of performance comparison of different ML models for prediction of all ASP, early ASP, and late ASP are shown in Table S7–S9. Among all ML models, the CatBoost model using the 11 most important variables achieved the best performance, with AUCs of 0.97 (95% CI: 0.95, >0.99) for the training set, 0.94 (95% CI: 0.92, 0.97) for the internal test set, and 0.93 (95% CI: 0.90, 0.95) for the external test set (Table 2). Figure S4 shows rank of important variables for all ML models.
Table 2:
Imaging Characteristics of Patients with HCC Who Underwent TACE in All Datasets
| Imaging Characteristics (n = 9) | Training Set (n = 1026) | Internal Test Set (n = 257) | P Value | Total (Derivation Set) (n = 1283) | External Test Set (n = 1050) | P Value |
|---|---|---|---|---|---|---|
| Tumor size (cm) | .003 | <.001 | ||||
| <5 | 187 (18.2) | 35 (13.6) | 222 (17.3) | 453 (43.1) | ||
| 5–10 | 516 (50.3) | 113 (44.0) | 629 (49.0) | 503 (47.9) | ||
| >10 | 323 (31.5) | 109 (42.4) | 432 (33.7) | 94 (9.0) | ||
| No. of tumors | <.001 | <.001 | ||||
| 1–3 | 487 (47.5) | 20 (7.8) | 507 (39.5) | 635 (60.5) | ||
| >3 | 539 (52.5) | 237 (92.2) | 776 (60.5) | 415 (39.5) | ||
| Capsule appearance | <.001 | .008 | ||||
| Absence | 265 (25.8) | 93 (36.2) | 358 (27.9) | 242 (23.0) | ||
| Presence | 761 (74.2) | 164 (63.8) | 925 (72.1) | 808 (77.0) | ||
| Blood products in mass | .75 | .70 | ||||
| Absence | 911 (88.8) | 230 (89.5) | 1141 (88.9) | 939 (89.4) | ||
| Presence | 115 (11.2) | 27 (10.5) | 142 (11.1) | 111 (10.6) | ||
| Fat in mass, more than liver | .25 | .002 | ||||
| Absence | 686 (66.9) | 162 (63.0) | 848 (66.1) | 758 (72.2) | ||
| Presence | 340 (33.1) | 95 (37.0) | 435 (33.9) | 292 (27.8) | ||
| Necrosis or severe ischemia | .94 | <.001 | ||||
| Absence | 600 (58.5) | 151 (58.8) | 751 (58.5) | 773 (73.6) | ||
| Presence | 426 (41.5) | 106 (41.2) | 532 (41.5) | 277 (26.4) | ||
| Corona enhancement | .08 | .01 | ||||
| Absence | 747 (72.8) | 173 (67.3) | 920 (71.7) | 802 (76.4) | ||
| Presence | 279 (27.2) | 84 (32.7) | 363 (28.3) | 248 (23.6) | ||
| Intratumoral artery | .005 | <.001 | ||||
| Absence | 775 (75.5) | 172 (66.9) | 947 (73.8) | 875 (83.3) | ||
| Presence | 251 (24.5) | 85 (33.1) | 336 (26.2) | 175 (16.7) | ||
| BCLC stage | .73 | <.001 | ||||
| A | 315 (30.7) | 76 (29.6) | 391 (30.5) | 471 (44.9) | ||
| B | 711 (69.3) | 181 (70.4) | 892 (69.5) | 579 (55.1) |
Note.—Except where indicated, data are numbers of patients, with percentages in parentheses. BCLC = Barcelona Clinic Liver Cancer, HCC = hepatocellular carcinoma, TACE = transarterial chemoembolization.
We next compared the CatBoost model’s performance with that of five clinical staging systems (BCLC, European Association for the Study of the Liver, China Liver Cancer, Japan Society of Hepatology, and Hong Kong Liver Cancer) (Table 3). The CatBoost model yielded the best discriminatory ability, with concordance index values of 0.82 (95% CI: 0.79, 0.90) in the training set, 0.70 (95% CI: 0.61, 0.74) in the internal test set, and 0.75 (95% CI: 0.70, 0.80) in the external test set. Its time-dependent AUC was higher than that of clinical staging systems at various time points in three sets (all P < .001, Fig 2A–2C). The integrated Brier scores for the CatBoost model were 0.08 (95% CI: 0.07, 0.11), 0.08 (95% CI: 0.07, 0.11), and 0.11 (95% CI: 0.09, 0.18) in three sets (Fig 2D–2F). The decision curve analysis demonstrated that the CatBoost model provided greater net benefit across the range of reasonable threshold probabilities than clinical staging systems in three sets (Fig 2G–2I). Calibration curves showed fair agreement between the predicted and observed 1–3-year ASPFS probabilities in three sets (Fig S5).
Figure 2:
The (A−C) td-AUCs, (D−F) IBS, and (G−I) DCAs of the CatBoost model and clinical staging systems in three sets (the training set: A, D, G; the internal test set: B, E, H; and the external test set: C, F, I). BCLC = Barcelona Clinic Liver Cancer, CatBoost = Categorical Gradient Boosting, CNLC = China Liver Cancer, DCA = decision curve analysis, EASL = European Association for the Study of the Liver, HKLC = Hong Kong Liver Cancer, IBS = integrated Brier score, JHS = Japan Society of Hepatology, ML = machine learning, td-AUC = time-dependent area under the receiver operating characteristic curve.
Interpretation Methods for the CatBoost Model
The CatBoost model using 11 variables performed better than the other ML models (Fig 3A). We employed the SHAP method to elucidate the predicted ASP after TACE by the CatBoost model, which mainly included a global explanation at the feature level and a local explanation at the individual level. For global explanation, the top 11 important variables associated with ASP are shown with SHAP values in Figure 3B. Intratumoral artery was most strongly associated with ASP. Other variables included corona enhancement, des-γ-carboxy prothrombin (DCP) level, fat in mass (more than liver) (27), international normalized ratio, neutrophils, tumor number, alanine aminotransferase level, body mass index, necrosis or severe ischemia, and AFP level.
Figure 3:
Interpretation methods for the ML models. (A) The CatBoost model using 11 variables performed better than the other models. (B) Top 11 important variables associated with SHAP values. (C) Data for a patient with HCC trends toward ASP occurrence after TACE with a probability of 5.6%. (D) Heatmap shows variables for the prediction of ASP with SHAP values. The red color represents a high SHAP value, and the blue color represents a low SHAP value. (E) Waterfall plot shows the actual measured values of features. (F) Box plots (median, upper quartile, lower quartile, maximum, and minimum) of interpretation for ASP based on 11 variables in the internal test set. The x-axis represents the classification of each variable, and the y-axis represents the expression of SHAP values. A higher SHAP value for each patient represents a greater probability of ASP. ASPFS = advanced-stage progression-free survival, AFP = α-fetoprotein, ALT = alanine aminotransferase, ASP = advanced-stage progression, AUC = area under the receiver operating characteristic curve, BMI = body mass index, CatBoost = Categorical Gradient Boosting, DCP = des-γ-carboxy prothrombin, GBDT = Gradient Boosting Decision Tree, HCC = hepatocellular carcinoma, INR = international normalized ratio, LGBM = Light Gradient Boosting Machine, LR = logistic regression, ML = machine learning, RF = random forest, Neu = neutrophil, SHAP = SHapley Additive exPlanations, TACE = transarterial chemoembolization, XGBoost = eXtreme Gradient Boosting.
For local explanation, individualized data were input to provide information on how a certain ASP risk prediction was developed. Figure 3C represents the data for a patient with HCC trending toward ASP occurrence after TACE with a probability of 5.6%. As observed, the values of intratumoral artery, DCP, international normalized ratio, tumor number, neutrophils, AFP level, and necrosis or severe ischemia pushed the decision toward ASP. In contrast, corona enhancement, fat in mass, body mass index, and alanine aminotransferase level pushed this case toward non-ASP. The above-mentioned variables for predicting ASP are shown with SHAP values in the heatmap (Fig 3D). The actual measured values of features are also shown in the waterfall plot (Fig 3E). Furthermore, box plots of interpretation for ASP based on 11 variables in the internal test set are illustrated in Figure 3F.
Survival Benefit of Post-TACE Systemic Therapy according to the CatBoost Model
Using X-tile software in the training set, an optimal threshold of 66.27 for the CatBoost model was determined to clarify patients into high- and low-risk groups. ASPFS, PFS, and OS were significantly higher in the low-risk group than in the high-risk group in all three sets (P < .001 for all, Fig 4). The forest plots for the prediction of ASPFS, PFS, and OS with univariable Cox analysis in all patients (Fig S6A–C), low-risk group patients (Fig S6D–F), and high-risk group patients (Fig S6G–I) are shown in Figure S6. The correlation analysis between the risk scores with ASPFS (Fig S6J), PFS (Fig S6K), and OS (Fig S6L) are also presented in Figure S6.
Figure 4:
Kaplan-Meier curves of (A, D, G) ASPFS, (B, E, H) PFS, and (C, F, I) OS for the low-risk group and the high-risk group in three sets (the training set: A−C; the internal test set: D−F; and the external test set: G−I). ASPFS = advanced-stage progression-free survival, OS = overall survival, PFS = progression-free survival.
Post-TACE systemic therapies were performed in 812 patients, of whom 309 were included. These 309 patients and 257 patients from the internal test set who did not undergo post-TACE therapy were subjected to 1:1 PSM using the nearest neighbor method with a caliper of 0.1 (Fig 1; Fig S7). Covariates included in PSM were as follows: age, sex, Eastern Cooperative Oncology Group performance status, comorbidity, hepatitis B virus, cirrhosis, ascites, Child-Pugh class, tumor size, number of tumors, BCLC stage, and AFP level (Fig S7). With baseline variables balanced, 209 patients with HCC were assigned to the combination therapy group and 209 patients with HCC from the internal test set were assigned to the TACE alone group (Table S10). There was no evidence of a difference in ASPFS between the two groups (Fig 5A, P = .92). However, after PSM, PFS (1-year PFS rate: 41.6% vs 22.4%, P < .001) and OS (1-year OS rate: 63.2% vs 35.4%, P < .001) were higher in the combination therapy group than in the TACE alone group (Fig 5B, 5C). In the high-risk group, we observed no evidence of a difference of ASPFS between patients who underwent post-TACE therapy and those who did not (Fig 5D, P = .13). However, both PFS (1-year PFS rate: 41.6% vs 10.7%, P < .001) and OS (1-year OS rate: 63.2% vs 20.5%, P < .001) were higher in the combination therapy group than in the TACE alone group (Fig 5E, 5F). In the low-risk group, ASPFS, PFS, and OS in the combination therapy group were not statistically significant from the TACE alone group (Fig 5G–5I). Therefore, the CatBoost model might be helpful in decision-making for post-TACE therapy for patients with HCC (Fig S8).
Figure 5:
Kaplan-Meier curves of (A−C) ASPFS, (D−F) PFS, and (G−I) OS for the combination group and all (A, D, G) PSM patients, (B, E, H) high-risk patients, and (C, F, I) low-risk patients who did not undergo post-TACE therapy. ASPFS = advanced-stage progression-free survival, AT = adjuvant therapy, OS = overall survival, PFS = progression-free survival, PSM = propensity score matching, TACE = transarterial chemoembolization.
Discussion
Previous studies have attempted to predict recurrence of HCC after surgery using conventional statistical analyses based on imaging features and laboratory findings (30–32). However, the use of ML algorithms for the prediction of ASP after TACE in HCC remains scarce and worth exploring. In this retrospective study of 2333 patients with intermediate-stage HCC who underwent TACE, we established a CatBoost ML model including 11 clinical and CT imaging variables with AUC values of 0.94 (internal test set) and 0.93 (external test set) for predicting ASP. The CatBoost model outperformed five clinical staging systems with higher concordance index values (0.82, 0.70, and 0.75 on the training, internal test, and external test sets, respectively; all P < .001). Post-TACE systemic therapy improved PFS and OS (both P < .001) in the group at high risk of ASP.
Although a large number of multidimensional features can provide abundant information for ML models, interference between variables and the difficulty of collecting these data may limit performance and application. An important finding of our study is that the CatBoost model achieved the optimal discrimination and performance among the six ML models when only 11 variables were input and outperformed clinical staging systems. Moreover, in certain risk subgroups of the CatBoost model, there were significant differences in PFS and OS between the combination therapy and TACE alone groups. These findings suggested that the CatBoost model allowed for more accurate risk stratification of intermediate-stage HCC and helped to personalize treatment. We also found that several imaging features play a crucial role in predicting ASP after TACE. Jiang et al (27) previously reported that some CT imaging features were independent risk factors for recurrence after surgery following TACE in unresectable HCC. We selected eight typical CT imaging features to develop the ML model (27). Among them, intratumoral artery had the highest feature importance score, suggesting that the presence of intratumoral artery in HCC has a very close association with ASP (6,8,19).
The ML algorithm has always been regarded as a “black box” with poor explanations, which may lead to reluctance on the part of interventional radiologists to accept it due to the opacity of medical decision-making. Here, we used SHAP values for ASP prediction to overcome this limitation, as follows: First, the SHAP method could interpret the CatBoost model using a global explanation that describes the overall functionality; and second, the local explanation of the SHAP method was used for individual patients by inputting individualized data that provide details on how a certain prediction of ASP was made. Eleven ASP risk variables were identified in the CatBoost model, and their relationship to ASP was explained using SHAP values. For example, intratumoral artery was an important imaging feature, with a significant SHAP value difference for ASP risk prediction. This result indicated that tumors with an intratumoral artery may be more likely to invade blood vessels or metastasize extrahepatic organs than tumors without this imaging feature. Mechanistically, intratumoral artery was found to be associated with more prominent angiogenesis, more frequent microvascular invasion, and P53 mutation, which were predictors of HCC recurrence (27). DCP is used as a tumor marker for HCC, which was found to be a better indicator than AFP, with a higher SHAP value in this study (33).
To the best of our knowledge, no post-TACE systemic therapy is well-acknowledged and recommended for intermediate-stage HCC. However, some patients with HCC at high risk of recurrence after TACE are potential candidates for post-TACE therapies, with selection criteria that adhere to those of the IMBRAVE 050 trial (34). Although post-TACE therapies have been widely discussed, the lack of a standardized application scheme and accurate prognostic assessment has hampered their use. Importantly, patients with HCC at high risk of ASP according to the CatBoost model have worse PFS and OS than patients at low risk of ASP. Post-TACE therapy was not associated with survival benefits in the whole population or in patients at low risk for ASP. However, post-TACE therapies in patients at high risk for ASP improved PFS and OS. These results may be related to the fact that poor condition and liver function in high-risk patients impact the survival outcomes of post-TACE therapies and suggest that the CatBoost model may be used for decision-making before TACE for individualized prognostication and post-TACE therapy selection. Therefore, identifying these patients at increased risk for ASP before TACE may contribute to individual PFS or OS risk stratification, especially the use of post-TACE therapies, and implementing precise monitoring strategies.
This study had several limitations. First, patients were recruited from seven different hospitals over a long time span, which may confound the results. For example, there were different post-TACE systemic therapies, variations in the choice of TACE medications, and evolved TACE techniques over recent years. Second, the ASP rate was relatively low for the whole population, which resulted in a class imbalance issue for developing predictive models. However, ML models had parameters that allowed us to deal with class imbalance. Third, this study mainly included HCC with hepatitis B (the main cause in China). Thus, whether our findings are applicable to HCC from other causes requires further investigation. Fourth, we focused on patients who did not undergo systemic therapies before TACE in this study, which might limit the applicability of the model. Finally, some pathologic or genetic factors may affect the prediction of ASP, which were not included in this study.
In conclusion, we developed and externally tested the CatBoost model to predict ASP after TACE in patients with intermediate-stage HCC. The CatBoost model could facilitate accurate stratification of ASP risk and prognostic prediction before TACE, thereby serving as a favorable tool for enhancing individualized TACE management. Future multicenter prospective studies are needed to further validate the developed model.
Table 4:
Performance Comparison of Different ML Models Based on Preoperative Clinical Variables
| Models | No. of Predictors in Model | AUC | NPV | PPV | Sensitivity | Specificity | F1 Score |
|---|---|---|---|---|---|---|---|
| CatBoost | 11 | ||||||
| Training set | 0.97 (0.96, 0.99) | 0.99 (0.98, 1.00) | 0.40 (0.33, 0.47) | 0.92 (0.86, 0.98) | 0.87 (0.85, 0.90) | 0.56 | |
| Internal test set | 0.94 (0.89, 0.99) | 0.99 (0.98, 1.00) | 0.37 (0.24, 0.51) | 0.90 (0.78, 1.00) | 0.86 (0.82, 0.91) | 0.53 | |
| External test set | 0.93 (0.89, 0.96) | 0.99 (0.98, 1.00) | 0.42 (0.34, 0.50) | 0.84 (0.76, 0.93) | 0.92 (0.90, 0.94) | 0.56 | |
| GBDT | 9 | ||||||
| Training set | 0.94 (0.92, 0.97) | 0.98 (0.97, 0.99) | 0.49 (0.41, 0.58) | 0.83 (0.75, 0.91) | 0.92 (0.91, 0.94) | 0.62 | |
| Internal test set | 0.94 (0.89, 0.98) | 0.98 (0.96, 1.00) | 0.41 (0.26, 0.55) | 0.81 (0.64, 0.98) | 0.89 (0.86, 0.93) | 0.54 | |
| External test set | 0.94 (0.91, 0.97) | 0.99 (0.98, 1.00) | 0.41 (0.33, 0.49) | 0.86 (0.78, 0.94) | 0.91 (0.89, 0.93) | 0.55 | |
| LGBM | 32 | ||||||
| Training set | 0.98 (0.97, 0.99) | 0.99 (0.98, 1.00) | 0.56 (0.48, 0.65) | 0.93 (0.88, 0.98) | 0.93 (0.90, 0.95) | 0.70 | |
| Internal test set | 0.93 (0.87, 0.99) | 0.99 (0.97, 1.00) | 0.39 (0.25, 0.53) | 0.86 (0.71, 1.00) | 0.88 (0.84, 0.92) | 0.54 | |
| External test set | 0.91 (0.86, 0.95) | 0.99 (0.98, 1.00) | 0.49 (0.40, 0.58) | 0.81 (0.72, 0.91) | 0.94 (0.93, 0.96) | 0.62 | |
| RF | 33 | ||||||
| Training set | 0.94 (0.91, 0.96) | 0.98 (0.97, 0.99) | 0.38 (0.31, 0.45) | 0.84 (0.76, 0.92) | 0.87 (0.85, 0.90) | 0.52 | |
| Internal test set | 0.94 (0.89, 0.98) | 0.99 (0.98, 1.00) | 0.35 (0.22, 0.48) | 0.90 (0.78, 1.00) | 0.85 (0.81, 0.90) | 0.51 | |
| External test set | 0.94 (0.91, 0.97) | 0.99 (0.98, 1.00) | 0.37 (0.30, 0.45) | 0.87 (0.79, 0.95) | 0.89 (0.88, 0.91) | 0.52 | |
| LR | 24 | ||||||
| Training set | 0.94 (0.93, 0.96) | 0.98 (0.97, 0.99) | 0.44 (0.37, 0.52) | 0.84 (0.76, 0.92) | 0.90 (0.88, 0.92) | 0.58 | |
| Internal test set | 0.94 (0.90, 0.98) | 1.00 (0.99, 1.00) | 0.40 (0.26, 0.54) | 0.95 (0.86, 1.00) | 0.87 (0.83, 0.92) | 0.56 | |
| External test set | 0.84 (0.78, 0.910 | 0.98 (0.97, 0.99) | 0.44 (0.35, 0.53) | 0.71 (0.61, 0.82) | 0.93 (0.61, 0.82) | 0.54 | |
| XGBoost | 25 | ||||||
| Training set | 0.92 (0.91, 0.93) | 0.98 (0.97, 0.99) | 0.38 (0.31, 0.45) | 0.81 (0.76, 0.87) | 0.88 (0.86, 0.90) | 0.52 | |
| Internal test set | 0.91 (0.87, 0,98) | 0.98 (0.96, 1.00) | 0.38 (0.24, 0.52) | 0.81 (0.64, 0.98) | 0.88 (0.84, 0.92) | 0.51 | |
| External test set | 0.91 (0.84, 0.95) | 0.99 (0.98, 1.00) | 0.35 (0.28, 0.43) | 0.81 (0.72, 0.91) | 0.90 (0.88, 0.92) | 0.49 |
Note.—Values in parentheses are 95% CIs. AUC = area under receiver operating characteristic curve, CatBoost = Categorical Gradient Boosting, GBDT = Gradient Boosting Decision Tree, LGBM = Light Gradient Boosting Machine, LR = logistic regression, ML = machine learning, NPV = negative predictive value, PPV = positive predictive value, RF = random forest, XGBoost = eXtreme Gradient Boosting.
Table 5:
Comparisons between ML Model and Currently Used Major Staging Systems for Prediction of ASPFS
| Models and Staging Systems | C-index | P Value* | Time-dependent AUC | IBS |
|---|---|---|---|---|
| Training Set | ||||
| CatBoost | 0.82 | Ref | 0.87 (0.94, 0.80) | 0.08 |
| BCLC | 0.57 | <.001 | 0.59 (0.75, 0.43) | 0.10 |
| JHS | 0.58 | <.001 | 0.57 (0.73, 0.41) | 0.10 |
| CNLC | 0.57 | <.001 | 0.58 (0.74, 0.41) | 0.10 |
| EASL | 0.58 | <.001 | 0.71 (0.86, 0.57) | 0.10 |
| HKCL | 0.57 | <.001 | 0.57 (0.73, 0.41) | 0.10 |
| Internal Test Set | ||||
| CatBoost | 0.70 | Ref | 0.98 (1.00, 0.94) | 0.08 |
| BCLC | 0.51 | <.001 | 0.48 (0.82, 0.13) | 0.11 |
| JHS | 0.52 | <.001 | 0.64 (0.99, 0.30) | 0.11 |
| CNLC | 0.51 | <.001 | 0.64 (0.99, 0.30) | 0.11 |
| EASL | 0.49 | <.001 | 0.48 (0.82, 0.13) | 0.11 |
| HKCL | 0.52 | <.001 | 0.45 (0.80, 0.11) | 0.11 |
| External Test Set | ||||
| CatBoost | 0.75 | Ref | 0.80 (0.86, 0.73) | 0.11 |
| BCLC | 0.56 | <.001 | 0.58 (0.65, 0.51) | 0.22 |
| JHS | 0.58 | <.001 | 0.58 (0.64, 0.51) | 0.22 |
| CNLC | 0.55 | <.001 | 0.58 (0.63, 0.52) | 0.22 |
| EASL | 0.55 | <.001 | 0.55 (0.62, 0.48) | 0.22 |
| HKCL | 0.51 | <.001 | 0.54 (0.61, 0.47) | 0.22 |
Note.—Data in parentheses are the 95% CIs. Performance estimates were computed 2 years after transarterial chemoembolization for concordance index (C-index), time-dependent area under the receiver operating characteristic curve (AUC), and the integrated Brier score (IBS). ASPFS = advanced-stage progression-free survival, BCLC = Barcelona Clinic Liver Cancer, CatBoost = Categorical Gradient Boosting, CNLC = China Liver Cancer, EASL = European Association for the Study of the Liver, HKLC = Hong Kong Liver Cancer, JHS = Japan Society of Hepatology, ML = machine learning, Ref = reference.
P values were computed by comparing with the CatBoost model.
R.W. and Z.L. contributed equally to this work.
C.A. and Z.P. are co–senior authors.
Funding: This study was supported by the National High-Level Hospital Clinical Research Funding (no. BJ-2023-096).
Data sharing: Data generated or analyzed during the study are available from the corresponding author by request.
Disclosures of conflicts of interest: R.W. No relevant relationships. Z.L. No relevant relationships. L.J. No relevant relationships. M.Z. No relevant relationships. W.Y. No relevant relationships. W. Li No relevant relationships. Y.F. No relevant relationships. W. Liu No relevant relationships. C.L. No relevant relationships. P.W. No relevant relationships. J.H. No relevant relationships. Y.Z. No relevant relationships. J.T. No relevant relationships. J.R. No relevant relationships. C.A. No relevant relationships. Z.P. No relevant relationships.
Abbreviations:
- AFP
- α-fetoprotein
- ASP
- advanced-stage progression
- ASPFS
- advanced-stage progression-free survival
- AUC
- area under the receiver operating characteristic curve
- BCLC
- Barcelona Clinic Liver Cancer
- CatBoost
- Categorical Gradient Boosting
- DCP
- des-γ-carboxy prothrombin
- HCC
- hepatocellular carcinoma
- ML
- machine learning
- OS
- overall survival
- PFS
- progression-free survival
- PSM
- propensity score matching
- SHAP
- SHapley Additive exPlanations
- TACE
- transarterial chemoembolization
References
- 1. Bray F , Laversanne M , Sung H , et al . Global cancer statistics 2022: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries . CA Cancer J Clin 2024. ; 74 ( 3 ): 229 – 263 . [DOI] [PubMed] [Google Scholar]
- 2. Vogel A , Meyer T , Sapisochin G , Salem R , Saborowski A . Hepatocellular carcinoma . Lancet 2022. ; 400 ( 10360 ): 1345 – 1362 . [DOI] [PubMed] [Google Scholar]
- 3. Vitale A , Trevisani F , Farinati F , Cillo U . Treatment of hepatocellular carcinoma in the precision medicine era: from treatment stage migration to therapeutic hierarchy . Hepatology 2020. ; 72 ( 6 ): 2206 – 2218 . [DOI] [PubMed] [Google Scholar]
- 4. Singal AG , Llovet JM , Yarchoan M , et al . AASLD practice guidance on prevention, diagnosis, and treatment of hepatocellular carcinoma . Hepatology 2023. ; 78 ( 6 ): 1922 – 1965 . [Published correction appears in Hepatology 2023;78(6):E105.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. European Association for the Study of the Liver . EASL clinical practice guidelines: management of hepatocellular carcinoma . J Hepatol 2018. ; 69 ( 1 ): 182 – 236 . [Published correction appears in J Hepatol 2019;70(4):817.] [DOI] [PubMed] [Google Scholar]
- 6. Ronot M , Chernyak V , Burgoyne A , et al . Imaging to predict prognosis in hepatocellular carcinoma: current and future perspectives . Radiology 2023. ; 307 ( 3 ): e221429 . [DOI] [PubMed] [Google Scholar]
- 7. Zhou J , Sun H , Wang Z , et al . Guidelines for the diagnosis and treatment of primary liver cancer . (2022 edition). Liver Cancer 2023. ; 12 ( 5 ): 405 – 444 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Jiang H , Yang C , Chen Y , et al . Development of a model including MRI features for predicting advanced-stage recurrence of hepatocellular carcinoma after liver resection . Radiology 2023. ; 309 ( 2 ): e230527 . [DOI] [PubMed] [Google Scholar]
- 9. Finn RS , Qin S , Ikeda M , et al. ; IMbrave150 Investigators . Atezolizumab plus bevacizumab in unresectable hepatocellular carcinoma . N Engl J Med 2020. ; 382 ( 20 ): 1894 – 1905 . [DOI] [PubMed] [Google Scholar]
- 10. El-Khoueiry AB , Sangro B , Yau T , et al . Nivolumab in patients with advanced hepatocellular carcinoma (CheckMate 040): an open-label, non-comparative, phase 1/2 dose escalation and expansion trial . Lancet 2017. ; 389 ( 10088 ): 2492 – 2502 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Kudo M , Finn RS , Qin S , et al . Lenvatinib versus sorafenib in first-line treatment of patients with unresectable hepatocellular carcinoma: a randomised phase 3 non-inferiority trial . Lancet 2018. ; 391 ( 10126 ): 1163 – 1173 . [DOI] [PubMed] [Google Scholar]
- 12. Fan W , Zhu B , Chen S , et al . Survival in patients with recurrent intermediate-stage hepatocellular carcinoma: sorafenib plus TACE vs TACE alone randomized clinical trial . JAMA Oncol 2024. ; 10 ( 8 ): 1047 – 1054 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13. Peng Z , Fan W , Zhu B , et al . Lenvatinib combined with transarterial chemoembolization as first-line treatment for advanced hepatocellular carcinoma: a phase III, randomized clinical trial (LAUNCH) . J Clin Oncol 2023. ; 41 ( 1 ): 117 – 127 . [DOI] [PubMed] [Google Scholar]
- 14. Zhu HD , Li HL , Huang MS , et al. ; CHANCE001 Investigators . Transarterial chemoembolization with PD-(L)1 inhibitors plus molecular targeted therapies for hepatocellular carcinoma (CHANCE001) . Signal Transduct Target Ther 2023. ; 8 ( 1 ): 58 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15. Qiao W , Wang Q , Hu C , et al . Interim efficacy and safety of PD-1 inhibitors in preventing recurrence of hepatocellular carcinoma after interventional therapy . Front Immunol 2022. ; 13 : 1019772 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Lee S , Kim SH , Lee JE , Sinn DH , Park CK . Preoperative gadoxetic acid-enhanced MRI for predicting microvascular invasion in patients with single hepatocellular carcinoma . J Hepatol 2017. ; 67 ( 3 ): 526 – 534 . [DOI] [PubMed] [Google Scholar]
- 17. Xu X , Zhang HL , Liu QP , et al . Radiomic analysis of contrast-enhanced CT predicts microvascular invasion and outcome in hepatocellular carcinoma . J Hepatol 2019. ; 70 ( 6 ): 1133 – 1144 . [DOI] [PubMed] [Google Scholar]
- 18. Hung YW , Lee IC , Chi CT , et al . Redefining tumor burden in patients with intermediate-stage hepatocellular carcinoma: the Seven-Eleven Criteria . Liver Cancer 2021. ; 10 ( 6 ): 629 – 640 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19. Yang L , Gu D , Wei J , et al . A radiomics nomogram for preoperative prediction of microvascular invasion in hepatocellular carcinoma . Liver Cancer 2019. ; 8 ( 5 ): 373 – 386 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20. Liu X , Lu J , Zhang G , et al . A machine learning approach yields a multiparameter prognostic marker in liver cancer . Cancer Immunol Res 2021. ; 9 ( 3 ): 337 – 347 . [DOI] [PubMed] [Google Scholar]
- 21. Heo J , Yoon JG , Park H , Kim YD , Nam HS , Heo JH . Machine learning-based model for prediction of outcomes in acute stroke . Stroke 2019. ; 50 ( 5 ): 1263 – 1265 . [DOI] [PubMed] [Google Scholar]
- 22. Chen T , Li X , Li Y , et al . Prediction and risk stratification of kidney outcomes in IgA nephropathy . Am J Kidney Dis 2019. ; 74 ( 3 ): 300 – 309 . [DOI] [PubMed] [Google Scholar]
- 23. An C , Yang H , Yu X , et al . A machine learning model based on health records for predicting recurrence after microwave ablation of hepatocellular carcinoma . J Hepatocell Carcinoma 2022. ; 9 : 671 – 684 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Liu W , Wei R , Chen J , et al . Prognosis prediction and risk stratification of transarterial chemoembolization or intraarterial chemotherapy for unresectable hepatocellular carcinoma based on machine learning . Eur Radiol 2024. ; 34 ( 8 ): 5094 – 5107 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25. Collins GS , Moons KGM , Dhiman P , et al . TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods . BMJ 2024. ; 385 : e078378 . [Published correction appears in BMJ 2024;385:q902.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26. Reig M , Forner A , Rimola J , et al . BCLC strategy for prognosis prediction and treatment recommendation: The 2022 update . J Hepatol 2022. ; 76 ( 3 ): 681 – 693 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 27. Jiang H , Zuo M , Li W , Zhuo S , Wu P , An C . Multimodal imaging-based prediction of recurrence for unresectable HCC after downstage and resection-cohort study . Int J Surg 2024. ; 110 ( 9 ): 5672 – 5684 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 28. Guo Z , Wang P , Ye S , et al . Interpretable machine learning models based on SHapley Additive exPlanations for predicting the risk of cerebrospinal fluid leakage in lumbar fusion surgery . Spine 2024. ; 49 ( 18 ): 1281 – 1293 . [DOI] [PubMed] [Google Scholar]
- 29. An C , Wei R , Liu W , et al . Machine learning-based decision support model for selecting intra-arterial therapies for unresectable hepatocellular carcinoma: A national real-world evidence-based study . Br J Cancer 2024. ; 131 ( 5 ): 832 – 842 . [Published correction appears in Br J Cancer 2025;132(5):492.] [DOI] [PMC free article] [PubMed] [Google Scholar]
- 30. Zhang XP , Chen ZH , Zhou TF , et al. ; Chinese National Research Cooperative Group for Diagnosis and Treatment of Hepatocellular Carcinoma with Tumour Thrombus . A nomogram to predict early postoperative recurrence of hepatocellular carcinoma with portal vein tumour thrombus after R0 liver resection: A large-scale, multicenter study . Eur J Surg Oncol 2019. ; 45 ( 9 ): 1644 – 1651 . [DOI] [PubMed] [Google Scholar]
- 31. Wang K , Liu G , Li J , et al . Early intrahepatic recurrence of hepatocellular carcinoma after hepatectomy treated with re-hepatectomy, ablation or chemoembolization: a prospective cohort study . Eur J Surg Oncol 2015. ; 41 ( 2 ): 236 – 242 . [DOI] [PubMed] [Google Scholar]
- 32. Wakabayashi T , Ouhmich F , Gonzalez-Cabrera C , et al . Radiomics in hepatocellular carcinoma: a quantitative review . Hepatol Int 2019. ; 13 ( 5 ): 546 – 559 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 33. Kudo M . Urgent global need for PIVKA-II and AFP-L3 measurements for surveillance and management of hepatocellular carcinoma . Liver Cancer 2024. ; 13 ( 2 ): 113 – 118 . [DOI] [PMC free article] [PubMed] [Google Scholar]
- 34. Llovet JM , Pinyol R , Yarchoan M , et al . Adjuvant and neoadjuvant immunotherapies in hepatocellular carcinoma . Nat Rev Clin Oncol 2024. ; 21 ( 4 ): 294 – 311 . [DOI] [PMC free article] [PubMed] [Google Scholar]