Prediction of prognostic risk factors in hepatocellular carcinoma with transarterial chemoembolization using multi-modal multi-task deep learning

Abstract

Background

Due to heterogeneity of hepatocellular carcinoma (HCC), outcome assessment of HCC with transarterial chemoembolization (TACE) is challenging.

Methods

We built histologic-related scores to determine microvascular invasion (MVI) and Edmondson-Steiner grade by training CT radiomics features using machine learning classifiers in a cohort of 494 HCCs with hepatic resection. Meanwhile, we developed a deep learning (DL)-score for disease-specific survival by training CT imaging using DL networks in a cohort of 243 HCCs with TACE. Then, three newly built imaging hallmarks with clinicoradiologic factors were analyzed with a Cox-Proportional Hazard (Cox-PH) model.

Findings

In HCCs with hepatic resection, two imaging hallmarks resulted in areas under the curve (AUCs) of 0.79 (95% confidence interval [CI]: 0.71–0.85) and 0.72 (95% CI: 0.64–0.79) for predicting MVI and Edmondson-Steiner grade, respectively, using test data. In HCCs with TACE, higher DL-score (hazard ratio [HR]: 3.01; 95% CI: 2.02–4.50), American Joint Committee on Cancer (AJCC) stage III+IV (HR: 1.71; 95% CI: 1.12–2.61), Response Evaluation Criteria in Solid Tumors (RECIST) with stable disease + progressive disease (HR: 2.72; 95% CI: 1.84–4.01), and TACE-course > 3 (HR: 0.65; 95% CI: 0.45–0.76) were independent prognostic factors. Using these factors via a Cox-PH model resulted in a concordance index of 0.73 (95% CI: 0.71–0.76) for predicting overall survival and AUCs of 0.85 (95% CI: 0.81–0.89), 0.90 (95% CI: 0.86–0.94), and 0.89 (95% CI: 0.84–0.92), respectively, for predicting 3-year, 5-year, and 10-year survival.

Interpretation

Our study offers a DL-based, noninvasive imaging hallmark to predict outcome of HCCs with TACE.

Funding

This work was supported by the key research and development program of Jiangsu Province (Grant number: BE2017756).

Research in context.

Evidence before this study

Due to etiological heterogeneity and lack of histopathological factors, there is still a challenge to preoperatively assess survival outcome of hepatocellular carcinoma (HCC) with transarterial chemoembolization (TACE). Although there are many studies for subtype identification of HCC patients, embedding survival outcome of HCC with TACE as part of the procedure of identified subtypes has been rarely reported before. Most reported stratification models have either no or very few deep imaging-based integrations, leading to inadequate predictive accuracy.

Added value of this study

We integrated multi-modal data from clinical, laboratory, imaging and histopathologic indicators, especially large-scale radiomics features from CT imaging, giving an edge by exploiting the improved signal-to-noise ratio. The propelled deep learning (DL)-based risk-stratification, showing its robust prediction ability, performed accurately in predicting survival with an accuracy (concordance index) of 0.73 and areas under the curve (AUCs) of 0.85 to 0.90 in predicting 3-year to 10-year survival.

Implications of the available evidence

The newly developed DL risk-stratification signature, is not only for prognostication, but also instrumental for improving risk-adapted therapy in HCC, which is an important step targeting for personalized prognostication.

Alt-text: Unlabelled box

1. INTRODUCTION

Hepatocellular carcinoma (HCC) is the fifth most common cancer and the second leading cause of cancer-related death globally [1]. Unfortunately, patients with HCC often present beyond curable eligibility, and transarterial chemoembolization (TACE) has been used widely in these individuals [2,3]. Barcelona Clinic Liver Cancer (BCLC) score and American Joint Committee on Cancer (AJCC) stage are the most widely accepted stage systems for HCC and also make treatment allocation and prognostic prediction [1,4]. However, the high level of heterogeneity in HCC, along with the complex etiologic factors such as liver function and tumor burden, and along with the absence of postoperative histopathologic factors, make the prognosis prediction very challenging [5,6].

The prognostic variability for HCC patients treated with TACE highlights the need to better define patients and disease factors that are associated with outcome. Studies investigated that vascular endothelial growth factor (VEGF) and basic fibroblast growth factor (b-FGF) are sensitive hallmarks of HCC invasiveness, postoperative recurrence, and prognosis [7,8]. A few works have studied these factors in the TACE setting, but none provided accurate results [9]. Moreover, the histopathologic findings, such as microvascular invasion (MVI), Edmondson-Steiner grade, and peritumoral opacities can be prognostic factors that reliably predict both postoperative recurrence and prognosis 10, 11, 12. For now, nothing is known about the relationship between these histopathologic factors and TACE, because all these factors are inaccessible in patients receiving TACE as first-line treatment due to lack of surgical specimen.

Imaging setting could be a promising tool for the detection, stage, and risk assessment of HCC [13,14]. Imaging findings such as tumor size, capsule appearance, and contrast enhancement are prognostic biomarkers of HCC after surgical resection [15,16]. And recently, studies showed that HCCs presented with MVI or advanced Edmondson-Steiner grade were described with significant different gray-level imaging patterns, which might be tracked by radiomic machine learning (ML) or deep learning (DL) analysis 17, 18, 19. ML and DL are newly emerging form of data analysis that use a series of data-mining algorithms or statistical tools to obtain predictive or prognostic information 20, 21, 22. By building appropriate model with refined features, ML and DL algorithms can provide a great alternative to conventional approach for such medical image interpretation and triage in various challenging clinical tasks [23,24]. But despite its strong potential, clearly, prospective clinical application of this technology remains stymied. Task challenge remains about how to preferably integrate multi-modal data such as clinical, laboratory, imaging, and genomic indicators into a clinically available tool that enables patient's outlook to be predicted accurately. The Cox-Proportional Hazard (Cox-PH) is the most commonly used regression analysis approach for survival data among semi-parametric survival models. The assumption of Cox-PH is referred to the linear proportional hazards condition [25]. However, the Cox analysis may provide unsatisfactory results under conditions of high-dimensional survival data [26]. Therefore, new application areas of survival analysis urgently call for novel tools that able to handle high-dimensional data.

To fill this gap, we aimed to develop multi-task DL algorithms for prognostic risk factors in HCC with TACE by a concept of integrating multi-modal large-scale data from clinical, laboratory, imaging, and histopathologic indicators.

2. METHODS

2.1. Patients

Ethics committee approval was granted by the local institutional ethics review board, and the requirement of written informed consent was waived. All procedures involving human participants were performed in accordance with the 1975 Helsinki declaration and its later amendments.

In this retrospective study, we used a total of two cohorts at a single tertiary medical institution (The First Affiliated Hospital of Nanjing Medical University). One cohort consisted of 494 HCCs who received hepatic resection as first-line therapy between Jan. 2009 and Aug. 2017. All patients in this cohort underwent standard preoperative contrast-enhanced CT examination (Supplemental data; S-text-1) within 1 month before surgery; they also did not have history of previous surgical or TACE therapies. The histologic examination of surgical specimens was performed by two pathologists through serially examining multiple pathologic specimens. The histologic parameters ordinarily included the Edmondson-Steiner grade (I-II vs III-IV) [27], size, number, surgical margin, and MVI (absent vs present) of the resected tumor. MVI was defined as the presence of tumor in a portal vein, hepatic vein, or a large capsular vessel of the surrounding hepatic tissue lined by endothelium that was visible only on microscopy.

Another cohort consisted of 243 HCCs who received TACE as first-line therapy between Jan. 2010 and Mar. 2019. Patients in this cohort were diagnosed as HCCs via biopsy or denoted liver imaging settings according to the European Association for the Study of the Liver (EASL) criteria [1]. All they underwent standard preoperative contrast-enhanced CT examination within 1 month before their first course of TACE, and did not have history of previous surgical or TACE therapies before CT examination. Patients undergoing TACE had multifocal disease or single node that was not eligible for surgery or percutaneous treatments. The pre-TACE CT images were interpreted retrospectively, blindly, and independently by two board-certified radiologists (Z.F.P. and Z.Y.D.) with more than 10 years of experience in liver imaging. Any disagreement between the readers was discussed until a final consensus was generated to minimize any interpretation bias. The pre-TACE investigations were used to stage the hepatic disease. TACE was performed as routinely done, with a first angiographic phase aimed at identifying abnormal neovascularization. Then the emulsion of lipiodol (5–20 mL) plus epirubicin (10 mg) was injected and the procedure was completed by gelfoam embolization. Finally, functional exclusion of the hypervascular areas was confirmed by a final angiographic study.

For patients with TACE, follow-up CT scan was obtained 1 month after the treatment. Tumor response was classified into four categories according to the modified Response Evaluation Criteria in Solid Tumors (mRECIST) as follows: complete response (CR), partial response (PR), progressive disease (PD), and stable disease (SD) [28]. Patients whose assessment was either CR or PR were classified as responders, and those whose assessment was either PD or SD were classified as non-responders. Patients after the first course of TACE were followed-up according to institutional practice, based on serum alpha-fetoprotein (AFP) 3-monthly and CT or MRI every 1 to 3 months. Patients with PR, PD or SD made the choice to receive repeated TACE or additional radiofrequency ablation (RFA), chemotherapy, and radiation therapy, and the time of disease-specific progression or death was recorded. Patients were censored at the time of emigration, or 30 June 2019, whichever came first.

We used the data of two cohorts in two steps: The first step was to develop a histologic-related scoring model, respectively, for predicting MVI status (designated MVI-score) and Edmondson-Steiner grade (designated Edmondson-score) by training dual-phase contrast-enhanced CT radiomics features using a ML classifier in 494 surgical datasets. In the next step, we built a prognostic biomarker (designated DL-score) of HCC treated with TACE for disease-specific survival using a multi-task deep learning algorithm in 243 datasets. The predicted MVI-score, Edmondson-score, DL-score and clinical factors were then integrated into a new Cox-PH model to obtain a precise estimation of the survival time of an individual patient with TACE. The entire architecture of our networks for multi-task prognostic models is shown in Fig. 1.

Fig. 1.

Overall workflow of (A) a RF feature selection and a SVM predictor used to develop MVI-score and Edmondson's score in 494 HCCs with surgical resection. (B) multi-task DL networks to build a prognostic score for HCC survival after TACE. First, a DAE is used to reduce and transform 2420 radiomics features from 243 HCCs with TACE into 70 new features from the bottleneck hidden layer of the networks; then six time-varying DL algorithms were used to train the obtained DAE-transformed features and the one perform best was used to build a prognostic score to compute the survival probabilities on the time grid. Finally, MVI-sore, Edmondson's score, DL-based survival score and evidence-based clinicoradiologic score were integrated into a Cox-PH model to obtain a precise estimation of the survival time of an individual patient with TACE. RF, random forest; SVM, support vector machine; MVI, microvascular invasion; HCC, hepatocellular carcinoma; DL, deep learning; DAE, deep auto-encoder; TACE, transarterial chemoembolization; Cox-PH, Cox-Proportional Hazard.

2.2. Histologic-related score

First, we built a histologic-related ML scoring model for predicting MVI and Edmondson-Steiner grade, respectively, in 494 surgical datasets. Tumor segmentation was performed separately by a radiology resident (L.Q.P.) and a fellow (X.X.) using an in-house software (ONCO IMAG ANLY v 2.0; Shanghai Key Laboratory of MRI, ECNU, Shanghai, China) written with Python 3.6.1 (https://www.python.org) on 1.5-mm late arterial phase and 1.5-mm portal-venous phase contrast-enhanced CT images. The software allows the semiautomatic identification of the volume of interest (VOI) of the tumor with a combination of automatic segmentation algorithm and a manual approach. Radiomics features were analyzed from target VOIs using an open-source python package (https://pyradiomics.readthedocs.io/en/latest/#). Image normalization was performed using a method that remapped the histogram to fit within µ ± 3σ (µ: mean gray-level within the VOI and σ: gray-level standard deviation). Total 1210 radiomics features were computed for target volume based on the texture analysis methods available in the software package (Supplemental data; S-text-2). To evaluate intra-observer reliability, reader L.Q.P. performed the segmentations on the same CT study twice in a 1-week period. Reader X.X. completed the remaining image segmentations, and the readout sessions were conducted over a period of two months. The reliability was calculated by using intra-class correlation coefficient (ICC). Radiomics features with both intra-observer and inter-observer ICC values greater than 0.9 (indicating excellent stability) were selected for subsequent investigation.

To obtain a predictive score connecting to the histologic MVI and Edmondson-Steiner grade of HCC, we designed a novel ML algorithm as shown in Fig. 1A that combines the concepts of a random forest (RF) feature selection and a support vector machine (SVM) prediction [29,30]. We first assessed the radiomics features using a mean decrease Gini index (MDGI) calculated by RF algorithm in R package (http://cran.r-project.org/). The MDGI represents the importance of individual features for correctly classifying a residue into linker and non-linker regions. The MDGI was calculated by classifying 200 randomly selected linker features and 200 non-linker features, and the mean MDGI was calculated as the averaged MDGI over 100 trials. Vectors with mean MDGI larger than 3.0 were selected as optimal feature candidates.

Next, these RF-ranked features were trained using a stepwise SVM classifier for predicting MVI and Edmondson-Steiner grade. The datasets were split into training, validation and held-out testing data (6:1:3) in a randomized fashion to avoid overfitting. We performed a random search over the SVM parameter (γ, C and E) configuration with a range of 0.01 to 10 and a stepping interval of 0.1, and chose the optimal parameters with the best score on the training, validation and test set. Best score was based on the evaluation of area under the curve (AUC). The outputs calculated from SVM predictor indicated the relative risk that the patient had MVI or poor Edmondson-Steiner grade. By this way, two new imaging hallmarks of HCC, i.e., MVI-score and Edmondson-score, were built from pretreatment CT imaging.

2.3. Prognostic score for survival

To build an imaging biomarker of survival, we first preprocessed 145 training data from a total of 243 HCC samples with TACE treatment and obtained 2420 radiomics features from dual-phase contrast-enhanced CT images. These radiomics features were used as input features to build a prognostic score related to patients’ survival status using a novel time-varying DL algorithm. One challenge is that the Cox analysis may provide unsatisfactory results under conditions of high-dimensional survival data. To fill this gap, we designed a novel DL framework as shown in Fig. 1B that combined the concepts of deep auto-encoder (DAE) for dimension reduction [31] and time-varying DL algorithms for survival data analysis. Our DL architecture consists of two separate coactivated networks. One is a DAE to reduce and transform 2420 radiomics features into 70 new features from the bottleneck hidden layer of the networks; another is time-varying DL-based algorithms to train the obtained DAE-transformed features into a DL-score that computes the survival probabilities on the time grid.

DAE is an unsupervised DL algorithm used for dimension reduction [31,32]. In its simplest representation, DAE is formed by two components: an encoder that maps the input vectors to some hidden representations, and a decoder which maps the hidden representations back to a reconstructed version of the input (Supplemental data; S-text-3). In the present work, 5 hidden layers composed by 2420, 500, 70, 500, and 2420 units, respectively, were used to construct a DAE-based deep network. The processing was first pre-trained in an unsupervised layer-wise manner and then fine-tuned by a back propagation. We used the bottleneck layer (70 units) of the DAE to produce new features. By this way, an imaging sequence containing 2420 radiomics features was reduced to 70 new components that matched to observations for time-varying DL-based survival modeling.

In contrast to most common regression problems, survival data analysis is much more complex because it examines the relationships between survival distributions and features; and models the time it takes for events to occur; and sometimes, the event we want to predict (such as time of death) is not always observed (censored). Therefore, it is difficult in practice to determine which algorithm, the linear-based, the nonlinear-based or the recently proposed DL algorithm, is optimal to obtain a precise estimation of the survival time of an individual patient. To fill this gap, the second part of our DL architecture consisted of 6 DL-based algorithms as shown in Fig. 1B for survival analysis. The presented DL algorithms are: 1) a non-proportional Cox model referred to as Cox-Time [33]; 2) a proportional version of the Cox-Time model referred to as Cox-CC [33]; 3) a linear Cox regression model referred to as Deep-Hit [34]; 4) a neural multi-task logistic regression model referred to as N-MTLR [35]; 5) a model parametrizing the probability mass function and optimizing the survival likelihood referred to as PMF [36]; 6) a piecewise constant hazard model assuming the continuous-time hazard function is constant in predefined intervals referred to as PC—Hazard [36].

We performed the referred DL algorithms using an open-source Python package for survival analysis and time-to-event prediction with PyTorch (https://github.com/havakv/pycox/). The DL networks were standard multi-layer perceptrons with the same number of nodes in every layer. We used rectified linear unit (ReLU) as activation function and batch normalization between layers. We used dropout, normalized decoupled weight decay [37], and early stopping for regularization to prevent overfitting. Learning rates were found using the methods proposed by Smith [38]. For all the neural networks, we performed a random search for hyper-parameter optimization over 400 parameter configurations. Finally, we chose the model performed best in survival prediction on the training and test set. Model performance was evaluated based on the Harrell's concordance index (C-index), brier score (BS) and binomial log-likelihood (BLL). And the desired model was selected to be the one resulting in the best C-index, the least integrated BS and the least integrated BLL over the training/test data. A list of the hyper-parameter search spaces can be found in supplemental data (Table s1).

2.4. Clinical evaluation and statistical analysis

Clinical and laboratory parameters including age, sex, history of hepatic virus infection, Child-Pugh class, AFP, serum aspartate aminotransferase (AST), albumin-bilirubin (ALBI) score [39], treatment histories, and tumor response (CR, PR, PD, and SD) after TACE were collected and reviewed from patients’ records. Radiologist's interpretations were performed by the two referring radiologists (Z.F.P. and Z.Y.D.). The following imaging features were independently evaluated: 1) number of tumors detected (solitary vs multiple); 2) the size of leading lesion; 3) satellite lesions (absent vs present); 4) peritumoral arterial contrast enhancement (absent vs present); 5) a two-trait predictor of venous invasion (TTPVI) (absent vs present) which was based on independent estimations from internal arteries and hypodense halos [13,40]; 6) margin appearance (smooth vs non-smooth); 7) capsule appearance (complete vs incomplete); 8) typical wash-in/wash-out feature (absent vs present), according to EASL criteria for nodules > 1 cm (hypervascularity in the arterial phase and wash-out of contrast media in the portal-venous/delayed phases) [1]; 9) growth pattern (intrahepatic vs extrahepatic).

We evaluated the patients’ specific survival using a standard Cox-PH algorithm with censored regression by combining the concepts of DL survival score, ML-based MVI score and Edmondson-score, as well as clinical laboratory indicators and radiographic-interpretation scores. The overall survival (OS) was computed from the date of first course of TACE to date of death or censored at the date of last follow-up. The discriminatory performance of Cox-PH model was test using Harrell's C-index and calibration curve analysis. Survival curves were generated with the Kaplan-Meier method and compared by two-sided log-rank tests. Statistical analysis was performed using the R software (version 3.4.4, R Project for Statistical Computing, http://www.r-project.org). Two-sided p-values less than 0.05 were considered statistically significant.

2.5. Role of funding source

The funding source had no involvement in study design, data collection, data analysis, or manuscript preparation or approval.

3. RESULTS

3.1. Patient characteristics

Of total 737 patients, 478 patients (64.9%) underwent surgical resection; 16 patients (2.2%) underwent liver transplantation and 243 patients (32.9%) underwent nonsurgical TACE treatment. Presence of histologic MVI was diagnosed in the explanted tissues of 149 patients (30.2%) among the surgical group. Well-differentiated (Edmondson-Steiner grade I), moderate-differentiated (Edmondson-Steiner grade II) and poor differentiated HCC (Edmondson-Steiner grade III-IV) were diagnosed in the explanted tissues of 54 (10.9%), 391 (79.1%) and 49 (9.9%) patients, respectively, among the surgical group. In 243 HCCs treated with TACE, the CR, PR, SD and PD after the first course of TACE were identified in 30 (12.3%), 73 (30.0%), 96 (39.5%) and 44 (18.1%) patients, respectively. The repeated TACE, RFA and radiation therapy/chemotherapy were determined in 58/243 (23.9%), 37/243 (15.2%), 13/243 (5.3%) patients, respectively. The details of clinicoradiologic characteristics in two cohorts are summarized in Table 1.

Table 1.

The clinicoradiologic characteristics of HCC in two cohorts.

Characteristics	Surgery n = 494	TACE n = 243
Age
≤ 60 yrs	323 (65.4)	139 (57.2)
> 60 yrs	171 (34.6)	104 (42.8)
Sex
male	417 (84.4)	192 (79.0)
female	77 (15.6)	51 (21.0)
ALBI grade †
grade 1 ≤ −2.60	293 (59.3)	75 (30.9)
grade 2–3 >−2.60	201 (40.7)	168 (69.1)
AFP
< 20 ng/ml	160 (32.4)	63 (25.9)
20–400 ng/ml	145 (29.4)	74 (30.5)
> 400 ng/ml	189 (38.2)	106 (43.6)
8th AJCC stage
stage I	277 (56.1)	55 (22.6)
stage II	97 (19.6)	21 (8.6)
stage III	115 (23.3)	93 (38.3)
stage IV	5 (1.0)	74 (30.5)
Child-Pugh class
Child A	458 (92.7)	186 (76.5)
Child B + C	36 (7.3)	57 (23.5)
Imaging detected No. of tumors
solitary	401 (81.2)	164 (67.5)
multiple	93 (18.8)	79 (32.5)
Imaging detected tumor size
< 2 cm	42 (8.5)	13 (5.3)
2–5 cm	205 (41.5)	51 (21.0)
> 5 cm	247 (50.0)	179 (73.7)
Imaging margin appearance
smooth	257 (52.0)	70 (28.8)
non-smooth	237 (48.0)	173 (71.2)
Imaging tumor growth pattern
intrahepatic	289 (58.5)	90 (37.0)
extrahepatic	205 (41.5)	153 (63.0)
Imaging capsule appearance
complete	164 (33.2)	62 (25.5)
incomplete	330 (66.8)	181 (74.5)
Imaging wash-in/wash-out
absent	68 (13.8)	38 (15.6)
present	426 (86.2)	205 (84.4)
Imaging peritumoral enhancement
absent	464 (93.9)	129 (53.1)
present	30 (6.1)	114 (46.9)
Imaging TTPVI
absent	248 (50.2)	95 (39.1)
present	246 (49.8)	148 (60.9)
Imaging detected star nodule
absent	421 (85.2)	145 (59.7)
present	73 (14.8)	98 (40.3)

Note. -Unless indicated otherwise, data are number of tumors, with percentages in parentheses.

†, The albumin-bilirubin (ALBI) score was computed by the formula, −0.085 × (albumin g/l) + 0.66 × log (bilirubin umol/l) [39]. ALBI grade 1 (≤−2.60), grade 2 (>−2.60 to −1.39) and grade 3 (>−1.39). HCC, hepatocellular carcinoma; TACE, transarterial chemoembolization; AFP, serum α-fetoprotein; AJCC, American Joint Committee on Cancer; TTPVI, two-trait predictor of venous invasion.

3.2. MVI-score and Edmondson-score

From 494 samples, we preprocessed the training data as described in the "Methods" section and obtained 2420 radiomics features as input features. The RF algorithm first selected 28 relevant features for histologic MVI status and 42 relevant features for Edmondson-Steiner grade (supplemental data; Figure s1, Table s2 and Table s3). The SVM predictors trained with the RF-selected features resulted in AUCs of 0.84 (95% CI: 0.79–0.87) and 0.79 (95% CI: 0.71–0.85) in training and test for predicting MVI, respectively; and resulted in AUCs of 0.79 (95% CI: 0.74–0.83) and 0.72 (95% CI: 0.64–0.79) in training and test for predicting Edmondson-Steiner grade (well/moderate- vs poor-differentiation), respectively. The details of development and test for scoring scheme of MVI and Edmondson-Steiner grade are summarized in Fig. 2 and Table 2.

Fig. 2.

Receiver operating characteristic curve (ROC) of prediction model and corresponding predicted MVI and Edmondson's grade distribution. (A) ROCs of predicted MVI in training and test data with areas under the curve (AUCs) of 0.837 and 0.788, respectively. (B) Risk stratification of MVI in predicted and actual proportion in each group. The error bar represents the 95% confidence intervals of the median probability. (C) ROCs for prediction of Edmondson's grade I, II, III with AUCs of 0.65, 0.63 and 0.79, respectively. (D) Risk stratification of Edmondson's grade in predicted and actual proportion in each group. The error bar represents the 95% confidence intervals of the median probability.

Table 2.

DL-based prediction performance of histological status in two cohorts with HCC.

	Histologic MVI			Histologic Edmondson-Steiner grade
	(+)	(-)		I	II	III
Surgical group (n = 494)
Predict (+)	94 (19.0)	37 (7.5)	Predict I	3 (0.6)	0 (0)	0 (0)
Predict (-)	55 (11.1)	308 (62.3)	Predict II	51 (10.3)	391 (79.1)	30 (6.1)
			Predict III	0 (0)	0 (0)	19 (3.8)
TACE group (n = 243)
Predict (+)	119 (51.0)		Predict I	0 (0)
Predict (-)	124 (49.0)		Predict II	237 (97.5)
			Predict III	6 (2.5)

Note. -Unless indicated otherwise, data are number of tumors, with percentages in parentheses.

DL, deep learning.

3.3. DL-score

We preprocessed the data as described in the "Methods" section. The 2420 radiomics features were first stacked using DAE framework as shown in Fig. 1B. We then used the activity of 70 nodes from the bottleneck hidden layer (third layer) as new input features. The survival data were trained using the proposed DL algorithms, such as Cox-Time, Cox-CC, PMF, N-MTLR, Deep-Hit, and PC-hazard model, respectively. The results of dynamic performance tuning of models are summarized in Table 3 and Fig. 3, wherein the Cox-Time model with sets of 3 hidden layers and 32 nodes outperformed all the other DL models regarding the best C-index (0.93), the least BS (0.02) and the least BLL (0.07) over the training and test data. The output of optimal Cox-Time model was obtained as a new hallmark of HCC for the estimation of the survival time of an individual patient.

Table 3.

The performance of six DL algorithms for survival analysis of DAE-derived imaging features.

Model	Layer	Node	Batch	PLL	C-index	BS	BLL
Cox-Time	3	32	256	−1.26	0.93	0.02	0.07
Cox-CC	1	32	256	−1.53	0.92	0.03	0.23
PMF	3	64	256	–	0.87	0.11	0.41
N-MTLR	3	32	256	–	0.88	0.05	0.16
Deep-Hit	4	16	256	–	0.93	0.04	0.16
PC-Hazard	4	32	256	–	0.90	0.04	0.13

Note. Cox-Time, a relative risk model that extends Cox regression beyond the proportional hazards. Cox-CC, a proportional version of the Cox-Time model. PMF, a model parametrizing the probability mass function (PMF) and optimizing the survival likelihood. Deep-Hit, a PMF method with a loss for improved ranking that can handle competing risks. N-MTLR, the neural multi-task logistic regression. PC-hazard, a piecewise constant hazard (PC—Hazard) model assuming that the continuous-time hazard function is constant in predefined intervals. C-index, Harrell's concordance index; PLL, partial log-likelihood; BS, brier score; BLL, binomial log-likelihood.

Fig. 3.

Comparison of the loss functions in the six proposed DL algorithms. Loss functions in training and validation datasets according to (A) Cox-Time model; (B) Cox-CC model; (C) PMF model; (D) N-MTLR model; (E) Deep-Hit model; (F) PC—Hazard model. Cox-Time model outperformed all the other DL models regarding the best C-index (0.93), the least BS (0.02) and the least BLL (0.07) over the training and test datasets. C-index, Harrell's concordance index; PLL, partial log-likelihood; BS, brier score; BLL, binomial log-likelihood; DL, deep learning.

3.4. Clinical evaluation

As of Dec. 2019, of total 243 patients with TACE treatment, the disease-specific recurrence was determined in 124/243 (51.0%) patients, with median disease-free survival of 13.0 (95% CI: 10.3 to 15.7) months. The disease-specific mortality was determined in 137/243 (56.4%) patients, with median OS of 23.4 (95% CI: 19.0 to 29.3) months.

Among all clinicoradiologic factors, 11 significant variables such as tumor size, TTPVI, tumor extrahepatic extension, imaging features of tumor wash-in, wash-out, tumor margin, AJCC stage, RECIST status, TACE course, predicted DL-score and predicted MVI-score were identified by a univariate Mantel-Cox analysis (Table s4). The 11 univariate variables were then analyzed by a stepwise multivariate Cox-PH regression analysis. The Cox-PH model showed that AJCC stage III+IV, RECIST score of SD or PD, TACE course of 1–3 and predicted DL-score (+) were independent prognostic predictors of OS of HCC after TACE (Table 4). The OS of patients stratified by AJCC stage (I+II vs III+IV), RECIST status (CR+PR vs SD+PD), TACE treatment (1–3 courses vs > 3 courses), and predicted DL-score (negative vs positive) reflected significant difference using log-rank test (Fig. 4).These four independent associated factors were used to form a Cox-score for HCC with TACE described by the formula (Supplemental data; S-text-4): Cox-score = 0.536 × AJCC stage (0: stage I+II; 1: stage III+IV) + 0.999 × RECIST status (0: CR+PR; 1: SD+PD) + 1.103 × DL-score (0: negative; 1: positive) - 0.427 × TACE course (0: 1–3 courses; 1: > 3 courses) (Fig. 5) [25]. The resulting TACE Cox-score demonstrated good accuracy in estimating the OS with a C-index of 0.733 (95% CI: 0.705–0.756) (Fig. 6A). The survival receiver operating characteristic curves (ROCs) analysis demonstrated that the TACE Cox-score predicted 3-year, 5-year, and 10-year survival with AUCs of 0.85 (95% CI: 0.81–0.89), 0.90 (95% CI: 0.86–0.94) and 0.89 (95% CI: 0.84–0.92), respectively (Fig. 6B).

Table 4.

Multivariate Cox-PH regression analysis of overall survival of patients with HCC after TACE.

Variable	Multivariate Cox analysis
	β	HR (95% CI)	p
AJCC stage
I+II	Ref	Ref
III+IV	0.536	1.709 (1.118–2.612)	0.013
RECIST
CR or PR	Ref	Ref
SD or PD	0.999	2.717 (1.842–4.008)	< 0.001
TACE course
≤ 3	Ref	Ref
> 3	−0.427	0.653 (0.445–0.757)	0.029
Predicted DLS
Low-risk	Ref	Ref
High-risk	1.103	3.014 (2.017–4.504)	< 0.001

Note. HR, hazard ratio; CI, confidence interval; RECIST, Response Evaluation Criteria in Solid Tumors; β, regression coefficient.

Fig. 4.

Kaplan-Meier survival curves of OS according to the four independent prognostic factors in multivariate regression model. (A) OS stratified by AJCC stage. (B) OS stratified by TACE treatment. (C) OS stratified by RECIST status. (D) OS stratified by predicted DLS. OS, overall survival; AJCC, American Joint Committee on Cancer; TACE, transarterial chemoembolization; RECIST, Response Evaluation Criteria in Solid Tumors; DLS, deep learning score.

Fig. 5.

Survival nomogram for HCC with TACE. The variables in the nomogram are independent survival predictors with a univariate Mantel-Cox analysis. *, independent predictors with a stepwise multivariate Cox-Proportional Hazard regression analysis. HCC, hepatocellular carcinoma; TACE, transarterial chemoembolization; TTPVI, two-trait predictor of venous invasion; AJCC, American Joint Committee on Cancer; RECIST, Response Evaluation Criteria in Solid Tumors; DLS, deep learning score; MVI, microvascular invasion.

Fig. 6.

Calibration and ROC curves of Cox-PH model developed from new prognostic risk factors. (A) Calibration curve for predicting overall survival with a C-index of 0.733. (B) ROCs for predicting survival at 3-year, 5-year, and 10-year, respectively. TP, true positive; FP, false positive. C-index, Harrell's concordance index; AUC, area under the curve.

4. DISCUSSION

The current study investigated the potential value of a novel DL-score deriving from pre-therapeutic dynamic CT for survival prediction in HCC patients treated with TACE. We concluded that dual-phase contrast-enhanced CT images quantitated by multi-task deep learning radiomics analysis could provide prognostic aspect to HCC patients treated by TACE. A risk assessment model integrating clinical factors and DL-based imaging hallmark allows marked improvement in the prediction of patients’ long-term outcome, which makes us believe our findings can play an important role in prognostication and improving risk-adapted therapy.

DL is a newly emerging form of data analysis that can handle high-throughput features to obtain predictive or prognostic information [21,22]. In current study, we combined a DAE and time-varying DL algorithms to establish a prognostic score for HCC survival. There are several ways for dimension reduction of large data sets to ensure computational efficiency [41]. Principal components analysis (PCA) is the most widely used conventional dimension reduction approach. PCA reduces the data frame by identifying a set of new variables using a linear combination of the original variables [26]. DAE is a relative new method and is quite similar to PCA, but it is much more flexible. DAE can represent both linear and non-linear transformation in encoding while PCA can only perform linear transformation. DAE can also be layered to form deep learning network and has been successfully used in several clinical tasks [42,43]. We also tested combination of DAE and state-of-the-art time-varying DL-based algorithms for survival analysis. The Cox-Time algorithm, a novel DL-derived extension of Cox-PH model, outperformed the others in terms of prediction accuracy. It provided an advance in modeling rich relationships between the covariates and event times by parameterizing the relative risk function of Cox-PH model with neural networks [33]. It performed well in 5 real-world data sets and was further invalided in our study. The current study provided a preliminary application of the novel multi-task DL algorithms and acquired a survival-sensitive risk stratification model.

Several studies tried to connect gene expression profiling or histological slices with the patients’ survival status using DL. Chaudhary et al. built a DL model on 360 HCC patients' data using RNA sequencing, miRNA sequencing, and methylation data [42]. The model provided two optimal subgroups of patients with significant survival differences [42]. Saillard et al. used DL algorithms based on whole-slide digitized histological slides to build models for predicting survival of HCC patients treated by surgical resection [44]. The C-index for survival prediction can reach 0.78 [44]. Both of the methods are characterized with invasiveness and retrospectiveness. Furthermore, the implementation of gene expression profiling technologies may be hampered in clinical work by their cost and the histological slices can hardly be acquired in patients receiving TACE as first-line treatment. Imaging examination, especially radiomics analysis, is a state-of-the-art quantitation of HCC in a non-invasive manner, which can provide invisible features such as texture, shape and heterogeneity [19,20]. The performance of our new hallmark derived from DL radiomics analysis was comparable to that of genetic biomarkers in previous studies but in a non-invasive and cost-effective manner, which can be an alternative tool in clinical application. To the best of knowledge, our study was the first to investigate the survival prediction with multi-task DL methods in HCC patients receiving TACE as first-line treatment.

Given the high level of heterogeneity of HCC, there is no single reliable factor that can predict survival accurately. Computational-assisted models combining different factors associated with survival may be a viable alternative. Xu et al. developed a nomogram including portal vein invasion, tumor number, tumor capsule, AFP, AST, and indocyanine green retention at 15 min for survival prediction in HCC patients after TACE and achieved a C-index of 0.755 [45]. Ni et al. proposed a nomogram including Eastern Cooperative Oncology Group performance status score, liver cirrhosis, AFP, tumor size, tumor number, ALBI grade, and treatment sessions of TACE or microwave ablation (MWA) to predict survival of patients with intermediate-stage HCC after TACE combined with MWA and achieved a C-index of 0.770 [46]. However, none of these models takes pathological factors into consideration due to lack of histological findings in patients treated with TACE. We attempted to build two new imaging hallmarks based on histologic-imaging correlation, using machine learning of volumetric CT radiomics features for predicting MVI and Edmondson-Steiner grade of HCC. And then, the two hallmarks were used to prospectively assess the prognostic risk of HCCs with TACE. In univariate analysis, MVI-score was a significant prognostic factor, but was not selected in the stepwise multivariate analysis. This may be explained by the collinearity between MVI-score and the DL-score; both of which are derived from CT radiomics features.

Our final Cox-PH model comprising of DL-score, AJCC stage, RECIST status and TACE course achieved a C-index of 0.733. AJCC stage is a commonly used staging system in many tumors and is associated with prognosis [4,45]. Treatment response is also an important prognostic factor and the survival rate is higher in responders compared with non-responders [47,48]. Repetition of TACE maximizes tumor response and may prolong survival in well selected patients [49]. However, repeated TACE cycles are also associated with increased side effects and liver damage, potentially preventing an even greater survival advantage. TACE course more than 3 was a favorable factor in our study, which was in agreement with Ni's nomogram [46]. However, results from a study in a cohort of patients treated with TACE as an initial oncologic therapy demonstrated that the median survival did not offer significant difference between patients who receive single or multiple TACE courses [50]. The prognostic role of TACE course needs further validation. Interestingly, unlike many other models, factors associated with liver function such as Child-Pugh class and ALBI grade were not selected in our Cox-PH model. Difference in the degree of heterogeneity regarding these factors within the current population may lead to the result. It should be acknowledged that our model did not devalue liver function as a prognostic factor, meanwhile, our model needs to be validated in a wider population, but still, it appears to be an alternative tool to help clinical decision support.

Our study had several limitations. First, our study was conducted in a retrospective manner in a single center. Potential selection bias may hamper the reliability of the results. Second, we did not include other possible prognostic factors, such as VEGF, b-FGF and genomic information. The model may be more accurate if combined with these factors. Third, multi-parametric MRI has been widely used in HCC patients in recently years, which may contain more tumor information, so further studies are needed about prognostic value of information extracted from MRI.

In conclusion, the DL-score from CT images could be a new prognostic hallmark of HCC in patients with TACE. The prognostic model based on DL-score may accurately predict the long-term survival. Therefore, our results may improve the assessment of patients’ prognosis and individualized precision medicine.

Declaration of Competing Interest

The authors declare no potential conflicts of interest in this study.