Deep learning-based radiomics allows for a more accurate assessment of sarcopenia as a prognostic factor in hepatocellular carcinoma

Hepatocellular carcinoma (HCC) is one of the most common malignancies and is a major cause of cancer-related mortalities worldwide (Forner et al., 2018; He et al., 2023). Sarcopenia is a syndrome characterized by an accelerated loss of skeletal muscle (SM) mass that may be age-related or the result of malnutrition in cancer patients (Cruz-Jentoft and Sayer, 2019). Preoperative sarcopenia in HCC patients treated with hepatectomy or liver transplantation is an independent risk factor for poor survival (Voron et al., 2015; van Vugt et al., 2016). Previous studies have used various criteria to define sarcopenia, including muscle area and density. However, the lack of standardized diagnostic methods for sarcopenia limits their clinical use. In 2018, the European Working Group on Sarcopenia in Older People (EWGSOP) renewed a consensus on the definition of sarcopenia: low muscle strength, loss of muscle quantity, and poor physical performance (Cruz-Jentoft et al., 2019). Radiological imaging-based measurement of muscle quantity or mass is most commonly used to evaluate the degree of sarcopenia. The gold standard is to measure the SM and/or psoas muscle (PM) area using abdominal computed tomography (CT) at the third lumbar vertebra (L3), as it is linearly correlated to whole-body SM mass (van Vugt et al., 2016). According to a “North American Expert Opinion Statement on Sarcopenia,” SM index (SMI) is the preferred measure of sarcopenia (Carey et al., 2019). The variability between morphometric muscle indexes revealed that they have different clinical relevance and are generally not applicable to broader populations (Esser et al., 2019).

Radiomics refers to the large-scale, algorithm-based, quantitative analysis of imaging features, which can reveal disease features and underlying pathophysiological features. Deep learning is a data-driven analytical method that allows for the mining of images with hidden clinical value, and has shown promising results in diagnosing HCC and determining its pathology, prognosis, and response to therapy (Jin et al., 2021). Coupled with radiomics, deep learning can automatically learn features from imaging labels and be used to plan individualized treatment of cancer patients (Bi et al., 2019). In this study, we assessed muscle mass and determined its prognosis significance in HCC patients using a deep learning-based radiomic survival model implemented by tensorflow (TFDeepSurv).

This study included HCC patients who had undergone hepatectomy (n=492, discovery cohort, divided into the training and internal test sets at a 7:3 ratio) and liver transplantation (n=173, external LT test set) at the Department of Hepatobiliary and Pancreatic Surgery of the First Affiliated Hospital, Zhejiang University School of Medicine (Hangzhou, China), and 161 patients who had undergone hepatectomy at Easter Hepatobiliary Surgery Hospital, Naval Medical University (Shanghai, China) (external test set). Patients were eligible for inclusion if they had primary HCC confirmed by pathology and had undergone abdominal CT within the month before surgery. Patients with unsatisfactory image quality or missing clinical data were excluded. The clinical characteristics of the subjects in the training and validation sets are summarized in Table 1. The discovery and the external test cohorts were similar in terms of baseline clinical characteristics. In the external LT test cohort, the majority of recipients were female and the tumors were mostly multiple. Radiomic features from CT scans at L3 were used to develop the TFDeepSurv model. A flowchart of the research process is shown in Fig. 1. The theories of AutoEncoder and the TFDeepSurv network are shown in the supplementary section materials and methods.

Table 1.

Subject characteristics of discovery and test sets

Characteristics	Discovery cohort (n=492)	External test (n=161)	External LT test (n=173)
Age (years)*	55.9±11.1	55.0±11.2	52.4±9.3
BMI (kg/m2)*	22.9±2.9	24.0±3.4	23.3±3.3
Gender
Male	418 (85.0%)	139 (86.3%)	23 (13.3%)
Female	74 (15.0%)	22 (13.7%)	150 (86.7%)
Tumor size
Total diameter≥5 cm	128 (41.3%)a	97 (60.2%)	56 (32.4%)b
Tumor number
Single	415 (84.3%)c	150 (93.2%)	77 (44.5%)d
Vascular invasion
Yes	70 (14.2%)e	NAf	46 (26.6%)

^* Data are shown as mean±standard deviation. ^a Miss data (n=35); ^b Miss data (n=1); ^c Miss data (n=35); ^d Miss data (n=1); ^e Miss data (n=24); ^f Miss data (n=161). BMI, body mass index; NA, not available.

Fig. 1. Overview of the research process. The left panel shows the overall workflow of the proposed study. The right panel shows the architecture of the deep neural network prediction model for the prognosis of hepatocellular carcinoma (HCC) based on AutoEncoder and TFDeepSurv. CT: computed tomography; DL: deep learning; TFDeepSurv: deep learning-based radiomic survival model implemented by tensorflow.

Plain abdominal CT images of the patients were obtained from the hospital’s picture archiving and communication systems in digital imaging and communications in medicine (DICOM) format. The details of the CT images can be found in the supplementary section materials and methods. After the optimal window width and position were set, region of interest (ROI) segmentation was performed with a three-dimensional (3D) slicer. ROIs in SM and PM were manually delineated at the L3 level of the CT in each patient by an experienced clinician, and the delineated images were reviewed by a senior clinician. The patients’ clinical data were obtained from the hospital’s medical records, such as survival time and recurrence-free time.

According to Carey et al. (2017), sarcopenia is indicated by an SMI of ≤50 cm²/m² in males and ≤39 cm²/m² in females, while according to Golse et al. (2017), the PM area (PMA) is <15.61 cm² in males and <14.64 cm² in females. According to Hamaguchi et al. (2016), the PM index (PMI) is <6.36 cm²/m² in men and <3.92 cm²/m² in women, and according to Yoo et al. (2017), a PM density (PD) of <38.5 Hounsfield unit (HU) indicates sarcopenia. Fujiwara et al. (2015) used mean muscle attenuation (MA) as an indicator of intermuscular fat (IMF) deposition and found that a positive MA result is ≤44.4 HU in men and ≤39.3 HU in women. The prevalence of preoperative sarcopenia in patients with HCC varies according to different definitions of sarcopenia. In our cohort, based on definitions by Golse et al. (2017), Hamaguchi et al. (2016), Yoo et al. (2017), Carey et al. (2017), and Fujiwara et al. (2015), sarcopenia was present in 216 (43.9%), 263 (53.5%), 20 (4.1%), 290 (58.9%), and 352 (71.5%) patients, respectively. Its prevalence among 173 LT cases ranged from 2.9% to 78.6% (Table 2).

Table 2.

Prevalence of sarcopenia in different definitions and its association with outcomes

Definition of sarcopenia and muscle mass assessment	Discovery cohort (n=492)			External test set (n=161)		External LT test set (n=173)
	n (%)	RFS (HR, 95% CI)	OS (HR, 95% CI)	n (%)	OS (HR, 95% CI)	n (%)	RFS (HR, 95% CI)	OS (HR, 95% CI)
Golse et al. (2017)M: PMA<1561 mm2F: PMA<1464 mm2	216 (43.9%)	1.04 (0.80‒1.35)P=0.76	1.58 (1.07‒2.35)P=0.02	54 (33.5%)	1.58 (0.81‒3.06) P=0.18	71 (41.0%)	1.24 (0.64‒2.38)P=0.52	1.59 (0.76‒3.31)P=0.22
Hamaguchi et al. (2016)M: PMI<6.36 cm2/m2F: PMI<3.92 cm2/m2	263 (53.5%)	1.14 (0.88‒1.48)P=0.31	1.57 (1.05‒2.35)P=0.03	81 (50.3%)	1.27 (0.66‒2.45) P=0.48	100 (57.8%)	1.37 (0.70‒2.65)P=0.36	1.47 (0.70‒3.13)P=0.31
Yoo et al. (2017) PD<38.5 HU	20 (4.1%)	1.27 (0.68‒2.40)P=0.45	2.11 (0.92‒4.85)P=0.08	0	NA	5 (2.9%)	5.98 (1.33‒26.90)P=0.02	5.47 (1.25‒23.92)P=0.02
Carey et al. (2017)M: SMI≤50 cm2/m2F: SMI≤39 cm2/m2	290 (58.9%)	1.10 (0.84‒1.43)P=0.49	1.52 (1.00‒2.31)P=0.05	97 (60.2%)	1.92 (0.92‒3.98) P=0.08	87 (50.3%)	1.08 (0.55‒2.11)P=0.83	1.23 (0.57‒2.66)P=0.60
Fujiwara et al. (2015)M: MA≤44.4 HUF: MA≤39.3 HU	352 (71.5%)	1.31 (0.97‒1.76)P=0.07	1.52 (0.96‒2.40)P=0.08	8 (5.0%)	0.52 (0.07‒3.82) P=0.52	136 (78.6%)	1.34 (0.61‒2.95)P=0.47	2.44 (0.84‒7.05)P=0.10

LT, liver transplantation; RFS, recurrence-free survival; OS, overall survival; HR, hazard ratio; CI, confidence interval; M: male; F: female; PMA, psoas muscle area; PMI, psoas muscle index; PD, psoas muscle density; SMI, skeletal muscle index; MA, mean muscle attenuation; HU, Hounsfield unit; NA, not available.

In the discovery cohort, patients with sarcopenia at the time of operation had significantly lower overall survival (OS) according to three muscle indices (PMI, SMI, and PMA), while based on other muscle indices (IMF and PD), sarcopenia was not associated with OS (Fig. 2). However, sarcopenia was not associated with recurrence-free survival (RFS) according to any muscle indices (Fig. S1). Additionally, only the patients with sarcopenia based on PMI had a trend of lower OS than those without sarcopenia in the external test cohort (Fig. S2), and the patients with sarcopenia based on PD had a significantly lower OS than the patients without sarcopenia in the LT cohort (Fig. S3). To summarize, sarcopenia was more associated with OS. Finally, the TFDeepSurv model was constructed to predict OS.

Fig. 2. Role of traditional sarcopenia definitions in the prognosis of overall survival in patients with hepatocellular carcinoma (HCC) who have undergone liver resection. (a) Skeletal muscle index (SMI); (b) Psoas muscle index (PMI); (c) Intermuscular fat (IMF); (d) Psoas muscle area (PMA); (e) Psoas muscle density (PD). Red, sarcopenia; blue, non-sarcopenia.

Radiomics feature extraction was carried out using PyRadiomics, a tool that extracts standardized radiomic features from imaging data. Firstly, we extracted 1343 radiomics features from muscle images, including three standardized feature classes: first-order statistics, shape descriptors, and texture features (including a grey level co-occurrence matrix, a grey level run length matrix (GLCM), a grey level size zone matrix (GLSZM), a grey level dependence matrix (GLDM), and a neighboring gray tone difference matrix (NGTDM)). Features not only contain the SM area but also its density, shape, and textural features. In total, 1343 radiomic features were extracted from SMs containing psoas at L3 level CT images, which comprised various muscle mass information. To assess the performance of numerous radiomics features, a 3D-vector was obtained after using principal component analysis (PCA) to reduce dimensionality. As expected, radiomics features also have good power to discriminate between various sarcopenia definitions, except for the definition by Yoo et al. (2017) (Fig. 3), indicating that radiomics features of muscle are associated with sarcopenia.

Fig. 3. Ability of radiomics features to discriminate between different sarcopenia definitions. (a) Body mass index (BMI); (b) Skeletal muscle index (SMI); (c) Psoas muscle index (PMI); (d) Intermuscular fat (IMF); (e) Psoas muscle area (PMA); (f) Psoas muscle density (PD).

Radiomic features have a lot of redundancy and many features are linearly related to each other. Therefore, we used AutoEncoder to reduce the dimensionality of original radiomics features to obtain deep learning features (relatively small and not related to each other), and TFDeepSurv to construct a prognosis prediction model. TFDeepSurv was constructed as follows: to speed up the analysis and fully use its features, 1343 radiomics features were compressed into 100 variables using AutoEncoder, and the TFDeepSurv survival network was trained based on those variables. Figs. 4a and 4b show the training process. The medium-based cut-off of the TFDeepSurv stratified the HCC patients in the training set into two risk groups (Fig. 4c), and its prognostic stratification power was confirmed in the internal (Fig. 4d), the external (Fig. 4e), and the LT test sets (Fig. 4f) via Kaplan-Meier survival analysis.

Fig. 4. Development of the TFDeepSurv and Kaplan-Meier survival curves of training and test sets. (a, b) The trend of the concordance index (a) and loss function (b) of the deep neural network training process. (c‍‒‍f) The TFDeepSurv divided patients with hepatocellular carcinoma (HCC) into high- and low-risk groups with a significantly different overall survival (OS) in the training group (c), the internal test set (d), the external test set (e), and the external liver transplantation (LT) test set (f). TFDeepSurv: deep learning-based radiomic survival model implemented by tensorflow.

The time-dependent area under the receiver operating characteristic curve (AUC) and the concordance index (C-index) values are depicted in Fig. 5. Compared to the other sarcopenia definitions, TFDeepSurv maintained higher AUC values in the internal test set (Fig. 5a), the external test set (Fig. 5c), and the external LT test set (Fig. 5e). As anticipated, TFDeepSurv exhibited a significant predictive ability in the internal test set (C-index, 0.730; 95% confidence interval (CI), 0.707‍–‍0.753; Fig. 5b), the external test set (C-index, 0.667; 95% CI, 0.651‍–‍0.682; Fig. 5d), and the external LT test set (C-index, 0.653; 95% CI, 0.633‍–‍0.677; Fig. 5f). This is the first attempt to decode SM mass with radiomics in tumor patients, independent of any definition of sarcopenia. The principal finding was that, despite substantial heterogeneity among patients from multiple centers, TFDeepSurv still exhibited a superior performance to the traditional definitions of sarcopenia in the test sets.

Fig. 5. Comparison of the predictive performance of TFDeepSurv vs. various sarcopenia definitions. (a, b) Time-dependent area under the receiver operating characteristic curve (AUC) (a) and C-index (b) in the internal test set. (c, d) Time-dependent AUC (c) and C-index (d) in the external test set. (e, f) Time-dependent AUC (e) and C-index (f) in the external liver transplantation (LT) test set. TFDeepSurv: deep learning-based radiomic survival model implemented by tensorflow; PMA: psoas muscle area; PD: psoas muscle density; PMI: psoas muscle index; SMI: skeletal muscle index; IMF: intermuscular fat; ANOVA: analysis of variance.

SM mass is an important predictor of HCC outcomes, but many questions, such as “what is the best modality for assessing muscle?” “what are the ideal timing and frequency of muscle mass assessment?” and “how to incorporate the method of assessment into clinical decision-making?” remain unanswered. Sarcopenia, which is a metric of SM depletion, is used to assess muscle mass in clinical practice. Despite the existence of multiple definitions of sarcopenia, there is no consensus on the threshold value associated with poor survival that accurately stratifies patients awaiting surgery. However, based on the traditional definitions, the prevalence of sarcopenia varied from 2.9% to 78.6% in our cohort. Until now, radiomics has rarely been used to assess sarcopenia. Kim (2021) successfully identified sarcopenia based on radiomic features and machine learning methods, which proved that radiomic features could decode the phenomenon in the muscle. However, this was a pilot study of only a limited number of patients and lacked sufficient validation. Chen et al. (2022) proposed a new CT radiomics-based method for diagnosing sarcopenia, which could effectively improve the predictive accuracy of prognosis compared to the traditional sarcopenia definitions. However, the above research defined sarcopenia using different criteria. There are no established evaluative indicators or cut-off values for each traditional definition of sarcopenia, which may limit their application to different populations. Hence, there is a desperate need for an objective muscle mass evaluation system assisted by deep learning and radiomics.

This study is limited by its retrospective nature. The use of varying CT imaging equipment in diverse patient populations may display different muscle composition phenotypes, potentially leading to biases during feature extraction. Thus, to validate the robustness and generalizability of our findings, larger prospective studies are warranted.

In conclusion, we developed a TFDeepSurv system for accessing SMs and predicting the survival of patients with HCC who have undergone liver resection and transplantation. The prognostic stratification power of TFDeepSurv was superior to other traditional definitions of sarcopenia, so it can serve as a visual prognostic tool that can assess muscles and help clinicians to identify patients with a high mortality risk and plan their treatment.

Materials and methods

Detailed methods are provided in the electronic supplementary materials of this paper.

Author contributions

Xiao XU, Shusen ZHENG, Ningyang JIA, and Zhikun LIU performed the conceptualization; Yichao WU, Lun LU, Ningyang JIA, Abid Ali KHAN, Jianguo WANG, and Jun CHEN contributed to the data curation; Xiao XU, Zhikun LIU, and Jun CHEN performed the funding acquisition; Zhikun LIU and Yichao WU performed the methodology; Zhikun LIU, Yichao WU, and Lun LU performed the investigation; Xiao XU and Zhikun LIU performed the project administration; Yichao WU, Zhikun LIU, and Abid Ali KHAN wrote the manuscript; Xiao XU, Shusen ZHENG, and Ningyang JIA contributed to writing ‒ review & editing. All authors have read and approved the final manuscript, and therefore, have full access to all the data in the study and take responsibility for the integrity and security of the data.

Compliance with ethics guidelines

Xiao XU is an editorial board member for Journal of Zhejiang University-SCIENCE B (Biomedicine & Biotechnology) and was not involved in the editorial review or the decision to publish this article.

Zhikun LIU, Yichao WU, Abid Ali KHAN, Lun LU, Jianguo WANG, Jun CHEN, Ningyang JIA, Shusen ZHENG, and Xiao XU declared that they have no conflict of interest.

All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (the Ethics Committee of Affiliated Hangzhou First People's Hospital) and with the Helsinki Declaration of 1975, as revised in 2013. The individual consent for this retrospective analysis was waived.

Data availability statement

The dataset used or analyzed during the current study is available from the corresponding author on reasonable request.