Predicting preoperative lymph node metastasis of hilar cholangiocarcinoma based on deep learning radiomics

Abstract

Background:

The objective of this study was to evaluate whether deep learning radiomic (DLR) models utilizing B-mode ultrasound (BUS) and contrast-enhanced ultrasound (CEUS) could improve the preoperative prediction of lymph node metastasis (LNM) in patients with hilar cholangiocarcinoma (HCCA).

Methods:

The study included 110 HCCA patients from two clinical centers, divided into primary and external validation cohorts. Pathological verification of lymph node status was performed, and the ResNet101 architecture was used to extract deep learning features (DLFs) from BUS and CEUS images. The Genetic Programming-based Symbolic Regression (GPSR) algorithm was applied to integrate radiomic features (RadFs) and DLFs, generating deep learning radiomic features (DLRFs). DLR models were subsequently constructed using the eXtreme Gradient Boosting (XGBoost) algorithm.

Results:

Lymph node metastasis was identified in 48 out of 110 patients (43.64%). No significant differences in clinical characteristics were observed between LNM-positive and LNM-negative groups (P-values ranging from 0.14 to 0.98). A total of 837 RadFs and 4095 DLFs were initially extracted from each tumor region of interest (ROI). After feature selection, 10 RadFs (4 from BUS, 6 from CEUS) and 27 DLFs (5 from BUS, 22 from CEUS) were retained. Using the GPSR algorithm, 5 BUS-DLRFs, 10 CEUS-DLRFs, and 15 Combination-DLRFs were generated, leading to the development of three corresponding DLR models. In internal validation, the AUC values were 0.70 for the BUS-DLR model, 0.77 for the CEUS-DLR model, and 0.83 for the Combination-DLR model. In external validation, the AUC values were 0.66, 0.68, and 0.72, respectively. These results indicate that the integration of multiphasic CEUS and BUS data is essential for more comprehensively identifying LNM and achieving precise preoperative staging.

Conclusions:

The DLR models based on DLRFs demonstrated an enhanced ability to preoperatively predict LNM in HCCA patients, indicating that the integration of deep learning and RadFs from BUS and CEUS may offer improved predictive performance for LNM.

Introduction

Cholangiocarcinoma is the second most common primary tumor of the liver and biliary tract, accounting for 10-15% of all hepatic and biliary tumors (1). Among its subtypes, hilar cholangiocarcinoma (HCCA) represents approximately two-thirds of biliary tract malignancies (2). Surgical resection remains the primary treatment modality for HCCA (3, 4). However, the therapeutic outcomes remain suboptimal with a median survival time of 12-24 months and postoperative recurrence rates exceeding 70% (2, 5). Notably, both incidence and mortality rates of HCCA have been rising over the past decade, underscoring the critical need for improved staging accuracy to optimize therapeutic strategies (6). Lymph node metastasis (LNM) plays a central role in tumor staging and prognostic evaluation for HCCA (7). The Tumor Node Metastasis (TNM)/ American Joint Committee on Cancer (AJCC) classification system emphasizes metastatic lymph node presence and burden as critical staging criteria (8, 9). Moreover, many studies have identified LNM as a major determinant of overall survival (OS) and an independent predictor of tumor recurrence (10, 11). Guglielmi et al. reported a 5-year survival rate less than 20% in patients with LNM, while Groot Koerkamp et al. documented near 100% recurrence in LNM-positive patients after surgical resection and LN dissection (11, 12). Another study suggested that lymph node ratio (LNR) and positive node count may be more prognostic than metastatic location (13). These findings highlight the importance of accurate preoperative LNM assessment.

Current imaging modalities for LNM detection in HCCA include computed tomography (CT), magnetic resonance imaging (MRI), and positron emission tomography (PET) (14). However, CT demonstrates limited accuracy (63-69%), and PET/CT has low sensitivity (42%) despite high specificity (95%) (15-17). Ultrasonography (US) offers advantages of cost-effectiveness and safety, making it a routine preoperative tool (18, 19). Given the angiogenic dependence of LNM, contrast-enhanced ultrasound (CEUS) with its pure blood pool agent provides unique insights into tumor vascularity compared to CT/MRI (20-22). Recent studies have explored indirect LNM prediction using primary tumor US features, such as CEUS based models demonstrating predictive value in thyroid cancer (23-25).

Recently, radiomics and deep learning have been widely applied in tumor differentiation, tumor subtyping, survival prediction, and treatment response assessment and shown excellent performance (26-28). However, a critical research gap persists due to the inherent rarity of HCCA, which leads to a scarcity of imaging data and renders conventional deep learning architectures prone to overfitting. Consequently, there is a lack of a non-invasive model capable of extracting high-level features while maintaining stability on small-scale medical cohorts. Deep learning radiomics (DLR) represents a potential solution by integrating automated feature extraction with hierarchical representation learning (29). By leveraging transfer learning (utilizing pre-trained networks as feature extractors) and integrating it with sparse machine learning classifiers, DLR can achieve robust predictive performance without severe overfitting, making it a highly feasible strategy even in cohorts with limited sample sizes (30, 31). Moreover, the integration of B-mode ultrasound (BUS) and CEUS within a DLR model is essential: BUS provides foundational anatomical morphology, while CEUS captures vital microcirculatory perfusion and vascularity—functional signatures of metastasis invisible on conventional ultrasound. DLR synergistically leverages these complementary perspectives, enabling a comprehensive characterization of tumor heterogeneity and ensuring robust diagnostic performance (32).

Therefore, the purpose of this study is to develop and validate DLR models using BUS and CEUS images to predict preoperative LNM in HCCA patients.

Materials and Methods

Patients and data collection

Patients were enrolled based on the following inclusion criteria: 1) patients underwent curative resection with LN dissection; 2) histopathologically confirmed as HCCA; 3) preoperative BUS and CEUS examinations performed within 1 month of surgery. Exclusion criteria included: 1) local or systemic neoadjuvant therapy; 2) poor-quality or missing ultrasound images; 3) non-DICOM formatted images. As shown in Figure 1, from January 2010 to January 2022, 100 consecutive HCCA patients from the First Affiliated Hospital of Sun Yat-sen University were screened. A 3:1 ratio (75 training/25 internal validation) was established. An independent external validation cohort (10 patients) was recruited from the Third Affiliated Hospital of Sun Yat-sen University (Center II) using identical criteria.

Figure 1.

Patient selection flowchart.

Demographic data (age, gender), clinical factors (Bismuth-Corlette classification, intrahepatic stones, biliary drainage, tumor size), and preoperative serum laboratory results were collected. Among the 110 enrolled patients, the median ages were 61.32, 60.60, and 60.10 years for the training, internal validation, and external validation cohorts, respectively (P>0.05). Male patients accounted for approximately 60-80% of the total population (P>0.05). Most patients presented with Bismuth-Corlette type III or IV. The mean tumor size in the training set was 2.65±1.04cm, showing no significant difference compared to the validation sets (P>0.05).

Image acquisition

Five US systems with abdominal probes were utilized (Supplementary Table S1), with examinations performed by four experienced radiologists (≥5 years’ experience). Following a comprehensive survey of the liver and gallbladder, the target tumor’s maximum cross-section was identified. A 2.4-mL ultrasound contrast agent (SonoVue®, Bracco, Italy) was injected via the cubital vein, followed by a 5-mL saline flush. Real-time monitoring began immediately upon injection. Dynamic videos were recorded for at least 90 seconds, with a total observation period of at least 3 minutes per patient. All images and videos were archived in the PACS.

Table S1.

Related parameters of ultrasonic instrument and probe.

Ultrasonic instrument, Producer	Contrast imaging mode	Probe, Frequency	Mechanical index
Acuson Sequoia 512®, Siemens, Mountain View, Calif.	Contrast Pulse Sequencing® (CPS)	Vector array® (model 4V1), 1-4MHz	0.07-0.19
Aloka SSD-Alpha 10®, Aloka,Japan	Contrast Harmonic Echo®(CHE)	Convex array® (model UST-9130), 2-6MHz	0.06-0.10
Toshiba Aplio XV/500®, Toshiba, Japan	Contrast Harmonic Imaging®(CHI)	Convex array® (model PVT375BT), 3.5MHz	0.07-0.12
Aixplorer, SuperSonic Imagine, Aix-en-Provence, France	Pulse inversion-power modulation techniques	Curved-array transducer (model SC6–1), 1.0–6.0MHz	0.05–0.13
Canon Aplio i900, Canon, apan	Contrast Harmonic Imaging®(CHI)	Convex array® (model PVT375BT), 3.5MHz	0.07-0.12

Images processing

Static BUS/CEUS images and CEUS videos were independently reviewed by a junior (H.S, 3 years’ experience of CEUS) and senior (BX. L, 10 years’ experience of CEUS) radiologist blinded to clinical data. Four representative images per patient (BUS, peak enhancement, venous, and delayed phase CEUS) were selected to achieve a comprehensive multi-parametric characterization (33, 34). This selection captures both the foundational anatomical morphology from BUS and the dynamic microcirculatory perfusion from CEUS, which are critical for identifying tumor heterogeneity associated with LNM. By focusing on these key temporal nodes, we balanced the inclusion of high-dimensional functional data with computational efficiency while ensuring robust multi-modal feature alignment.

Radiomic feature extraction

The images were loaded into the 3D Slicer 4.11 software (http://www.Slicer.org) by a researcher (H.S) to delineate the region of interest (ROI) (Supplementary Figure S1). The software package “pyradiomics” was employed for the extraction of radiomic features (RadFs). For reproducibility assessment, the selection of 50 random cases (approx. 45% of the total cohort) provided sufficient statistical power to assess reproducibility without redundancy (35). A 14-day interval was implemented to mitigate memory bias, ensuring the independence of the two segmentation sessions. Furthermore, an intraclass correlation coefficient threshold of 0.75 was adopted as the rigorous benchmark for excellent reliability, filtering out unstable features to ensure the robustness and generalizability of the model.

Figure S1:

This figure illustrates the ROIs for HCCA tumor lesions across four phases: A) BUS, B) peak enhancement CEUS, C) venous phase CEUS, and D) delayed phase CEUS.

Deep learning feature extraction

Firstly, ITK-SNAP software (http://www.itksnap.org) was used to crop 224×224 ROIs from maximum tumor cross-sections, which was verified by two radiologists (H.S and BX. L).

Secondly, the ResNet101 model was utilized in this study, which had been pre-trained on the ImageNet dataset (https://www.image-net.org) and exhibited exceptional recognition capabilities for natural images. The natural images refer to a vast collection of everyday photographic content—such as animals, objects, and landscapes—which enables the model to learn generalized low-level visual features like edges and textures. To adapt this pre-trained knowledge to the specific task of LNM prediction in HCCA, we implemented a transfer learning strategy using the Stochastic Gradient Descent (SGD) optimizer with a learning rate of 0.001, a batch size of 16, and a maximum of 100 epochs. A binary cross-entropy loss function was utilized, and data augmentation techniques, including random rotation, flipping, and brightness adjustment, were applied to mitigate the impact of data imbalance and prevent overfitting within our data-constrained cohort. A selective layer-freezing approach was executed where the foundational convolutional layers were frozen to preserve generalized imaging features from ImageNet, while deeper task-specific layers were fine-tuned using our dataset to capture high-level hierarchical representations of tumor heterogeneity. The complete dataset comprised 440 standardized images from 110 patients, representing four distinct imaging phases per patient in a 1:1:1:1 ratio: BUS, peak enhancement CEUS, venous phase CEUS, and delayed phase CEUS (>120s). All tumor ROIs were independently delineated and consensus-validated by two radiologists (H.S and BX. L), respectively, ensuring the diagnostic reliability and precision of the data annotation.

Under the supervision of image labels, the ResNet101 model was trained to update parameters and ultimately extract deep learning features (DLFs) that were most relevant to the image labels in the fully connected layer. The output of the penultimate fully connected layer was used as DLFs. Consequently, two types of DLFs were acquired: BUS-DLFs and CEUS-DLFs.

Feature selection and combination

A total of 837 quantitative RadFs were extracted using the “pyradiomics” package. Simultaneously, a pre-trained ResNet101 model served as a deep feature extractor for the tumor ROIs across four distinct imaging phases: BUS and three CEUS phases (arterial, venous, and delayed). For each ROI, 4,095 high-dimensional DLFs were extracted from the final fully-connected layer, capturing complex hierarchical patterns. To mitigate information redundancy and enhance predictive value, a multi-step feature selection process was employed. First, Spearman correlation coefficients were calculated between features; to ensure high dimensionality while minimizing collinearity, a rigorous threshold of 0.80 was applied, and any feature exceeding this coefficient was eliminated. This threshold of 0.80 is widely accepted in radiomic studies to balance the retention of diverse imaging information with the removal of highly redundant variables that could lead to model overfitting.

After feature selection using the Minimum Redundancy Maximum Relevance (mRMR) algorithm and the Least Absolute Shrinkage and Selection Operator (LASSO) regression, 10 RadFs (4 from BUS and 6 from CEUS) and 27 DLFs (5 from BUS and 22 from CEUS) were obtained (Supplementary Figure S2-S5). Subsequently, the Genetic Programming-based Symbolic Regression (GPSR) algorithm was implemented to amalgamate these remaining features into Deep Learning Radiomic features (DLRFs). This process yielded 5 BUS-DLRFs, 10 CEUS-DLRFs, and 15 Combination-DLRFs selected.

Figure S2.

Screening of BUS-RadFs. A) The feature coefficient decreases with the increase of λ value. B) The optimum λ value is selected by 10-fold cross-validation.

Figure S5.

Screening of CEUS-DLFs. A) The feature coefficient decreases with the increase of λ value. B) The optimum λ value is selected by 10-fold cross-validation

Model construction and evaluation

eXtreme Gradient Boosting (XGBoost) was used for model training. To ensure the stability of the model on a limited cohort, a repeated 10-fold cross-validation strategy was implemented during the training phase. Performance metrics included accuracy, sensitivity, specificity and area under receiver operating characteristic curve (AUC). DeLong test compared AUC between DLR and radiomic (Rad) models. Net Reclassification Index (NRI) and integrated discrimination improvement (IDI) evaluated clinical utility improvements. The overall workflow of the study is shown in Figure 2.

Figure 2.

Workflow of the study.

Statistical analysis

Statistical analyses were performed using R 4.2.2 (http://www.R-project.org) and Python3.9 (https://www.python.org), with convolutional neural network (CNN) architectures trained using the Keras. To address missing clinical data without compromising statistical power or reducing the sample size, the K-Nearest Neighbor (KNN) algorithm from the “DMwR2” package was employed. This methodology identifies the k most similar cases based on Euclidean distance in the feature space and estimates missing values using a weighted average for continuous variables or the most frequent value for categorical variables, thereby accounting for inherent correlations among clinical parameters. Feature engineering involved calculating Spearman correlation coefficients via the Python “pandas” package, followed by multi-step screening using the “mRMRe” and “glmnet” packages in R to mitigate redundancy. DLRFs were synthesized using GPSR via the “gplearn” package, and predictive models were constructed using the XGBoost algorithm. Model performance was rigorously evaluated using the “pROC” package to calculate the accuracy, sensitivity and specificity.

Results

Characteristics of the study population

The clinical and demographic profiles of the 110 enrolled patients across the training, internal validation, and external validation cohorts are summarized in Table 1. As shown in Table 2, univariate logistic regression analysis conducted within the training group indicated that none of the evaluated clinical indicators served as independent predictors of LNM (P=0.14-0.98). To rigorously assess model generalizability, an independent external validation cohort of 10 patients was incorporated. This dataset yielded a total of 40 standardized images—comprising 10 images each for the BUS, peak, venous, and delayed CEUS phases—with a balanced class distribution of 5 LNM-positive and 5 LNM-negative cases (1:1 ratio).

Table 1.

Clinical characteristics of patients with HCCA.

Clinical characteristics	Training Group (n=75)	Internal validation group (n=25)	External validation group (n=10)	P1	P2
Age (Mean ± SD)	61.32±10.89	60.60±12.21	60.10±14.08	0.78	0.75
Gender (Male/Female)	50/25	16/9	8/2	0.81	0.40
Bismuth-Corlette classification (I, II / III, IV)	23/52	8/17	1/9	0.90	0.17
Intrahepatic bile duct stones (Yes/No)	13/62	3/22	2/8	0.53	0.84
DBIL (≤7/>7μmol/L)	9/66	4/21	1/9	0.61	0.85
ALT (≤40/>40 U/L)	11/64	3/22	5/5	0.74	0.07
AST (≤37/>37 U/L)	8/67	2/23	2/8	0.70	0.39
ALP (≤110/>110 U/L)	4/71	1/24	2/8	0.80	0.09
GGT (≤50/>50 U/L)	4/71	2/23	1/9	0.63	0.56
CA19-9(≤35 />35u/ml)	19/56	5/20	3/7	0.59	0.75
CEA (≤5 />5ug/L)	49/26	15/10	7/3	0.63	0.77
Preoperative biliary drainage (Yes/No)	38/37	16/9	5/5	0.25	0.97
Tumor size (cm) (Mean ± SD)	2.65±1.04	2.98±1.22	2.98±1.17	0.24	0.39
LNM (Positive/Negative)	35/40	8/17	5/5	0.20	0.84

Note: SD, Standard Deviation; LNM, Lymph node metastasis; P1, Training Group vs. Internal validation group; P2, Training Group vs. External validation group

Table 2.

Univariate Logistic regression analysis of clinical variables in training group.

Clinical characteristics		LNM negative (n=40)	LNM positive (n=35)	P
Gender	Male	26	24	0.74
	Female	14	11
Bismuth-Corlette classification	I, II	12	11	0.89
	III, IV	28	24
Intrahepatic bile duct stones	No	33	29	0.97
	Yes	7	6
TBIL (μmol/L)	≤22	5	2	0.31
	>22	35	33
DBIL (μmol/L)	≤7	7	2	0.14
	>7	33	33
ALT (U/L)	≤40	7	4	0.46
	>40	33	31
AST (U/L)	≤37	5	3	0.58
	>37	35	32
ALP (U/L)	≤110	3	1	0.39
	>110	37	34
GGT (U/L)	≤50	3	1	0.39
	>50	37	34
CA19-9 (u/ml)	≤35	11	8	0.65
	>35	29	27
CEA (ug/L)	≤5	26	23	0.95
	>5	14	12
Preoperative biliary drainage	No	22	15	0.29
	Yes	18	20
Age, mean (±SD)	None	60.40±10.14	62.37±11.59	0.44
Tumor size, median [IQR]	None	2.20[1.90,3.30]	2.50[2.00,3.20]	0.98

Note: LNM, Lymph node metastasis; SD, Standard Deviation; IQR, Inter Quartile Range.

Model construction and evaluation

Three Rad (BUS-Rad /CEUS-Rad /Combination-Rad model) and three DLR models (BUS-DLR/CEUS-DLR/Combination-DLR model) were constructed using the XGBoost algorithm. The Combination models performed best because they merge two complementary perspectives: the structural details of the tumor and its dynamic blood supply. This approach enables the model to detect hidden patterns of metastasis with greater accuracy than single-modality models.

In the Rad cohort, the Combination-Rad and CEUS-Rad models outperformed the BUS-Rad model in the training set (AUC: 0.88 vs. 0.84) (Table S2 and Figure 3). Although the Combination-Rad model maintained the highest AUC in validation sets (internal: 0.70; external: 0.68), the inter-model differences lacked statistical significance (P=0.48-0.79), suggesting that traditional descriptors may have reached a diagnostic plateau in HCCA staging (Table 3).

Table S2.

Performance evaluation of Rad models.

Training group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
US-Rad	0.84 (0.75 - 0.93)	0.79 (0.69 - 0.87)	0.74 (0.60 - 0.89)	0.83 (0.70 - 0.93)
CEUS-Rad	0.88 (0.81 - 0.95)	0.80 (0.71 - 0.89)	0.86 (0.74 - 0.97)	0.75 (0.63 - 0.88)
Combination-Rad	0.88 (0.80 - 0.95)	0.81 (0.73 - 0.89)	0.94 (0.86 - 1.00)	0.70 (0.55 - 0.83)
Internal validation group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
US-Rad	0.64 (0.40 - 0.89)	0.60 (0.40 - 0.76)	0.88 (0.63 - 1.00)	0.47 (0.24 - 0.71)
CEUS-Rad	0.66 (0.58 - 0.93)	0.72 (0.52 - 0.88)	0.63 (0.55 - 0.88)	0.76 (0.53 - 0.94)
Combination-Rad	0.70 (0.42 - 0.98)	0.72 (0.52 - 0.88)	0.75 (0.58 - 1.00)	0.71 (0.47 - 0.94)
External validation group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
US-Rad	0.56 (0.15 - 0.97)	0.60 (0.30 - 0.90)	0.80 (0.40 - 1.00)	0.40 (0.30 - 0.80)
CEUS-Rad	0.64 (0.23 - 1.00)	0.70 (0.50 - 0.90)	0.80 (0.40 - 1.00)	0.60 (0.20 - 0.80)
Combination-Rad	0.68 (0.27 - 1.00)	0.80 (0.60 - 1.00)	1.00 (0.30 - 1.00)	0.60 (0.20 - 1.00)

Figure 3.

Performance evaluation of Rad models. (A) Predictive performance of BUS-Rad model, CEUS-Rad model, and Combination-Rad model in training group and internal/external validation group. (B) ROC curve of Rad models. Note: Rad, Radiomic; BUS, B-mode ultrasound; CEUS, Contrast-enhanced ultrasound.

Table 3:

The results of Delong test.

Training group	DeLong’s test	Internal validation group	DeLong’s test	External validation group	DeLong’s test
Combination-Rad vs. BUS-Rad (0.88 vs. 0.84)	P=0.37	Combination-Rad vs. BUS-Rad (0.70 vs. 0.64)	P= 0.64	Combination-Rad vs. BUS-Rad (0.68 vs. 0.56)	P=0.68
Combination-Rad vs. CEUS-Rad (0.88 vs. 0.88)	P=0.89	Combination-Rad vs. CEUS-Rad (0.70 vs. 0.66)	P=0.79	Combination-Rad vs. CEUS-Rad (0.68 vs. 0.64)	P=0.48
CEUS-Rad vs. BUS-Rad (0.88 vs. 0.84)	P=0.45	CEUS-Rad vs. BUS-Rad (0.66 vs. 0.64)	P=0.90	CEUS-Rad vs. BUS-Rad (0.64 vs. 0.56)	P=0.76
Combination-DLR vs. BUS-DLR (0.94 vs. 0.97)	P=0.43	Combination-DLR vs. BUS-DLR (0.83 vs. 0.70)	P= 0.17	Combination-DLR vs. BUS-DLR (0.72 vs. 0.66)	P=0.85
Combination-DLR vs. CEUS-DLR (0.94 vs. 0.99)	P=0.05	Combination-DLR vs. CEUS-DLR (0.83 vs. 0.77)	P=0.61	Combination-DLR vs CEUS-DLR (0.72 vs. 0.68)	P=0.88
CEUS-DLR vs. BUS-DLR (0.99 vs. 0.97)	P=0.32	CEUS-DLR vs. BUS- DLR (0.77 vs. 0.70)	P=0.61	CEUS-DLR vs. BUS- DLR (0.68 vs. 0.66)	P=0.95
BUS-Rad vs. BUS-DLR (0.84 vs. 0.97)	P<0.01	BUS-Rad vs. BUS-DLR (0.64 vs. 0.70)	P=0.71	BUS-Rad vs. BUS-DLR (0.56 vs. 0.66)	P=0.78
CEUS-Rad vs. CEUS-DLR (0.88 vs. 0.99)	P<0.01	CEUS-Rad vs. CEUS-DLR (0.66 vs. 0.77)	P=0.59	CEUS-Rad vs. CEUS-DLR (0.64 vs. 0.68)	P=0.88
Combination-Rad vs. Combination-DLR (0.88 vs. 0.94)	P=0.16	Combination-Rad vs. Combination-DLR (0.70 vs. 0.83)	P=0.29	Combination-Rad vs. Combination-DLR (0.68 vs. 0.72)	P=0.88

Note: Rad, Radiomic; BUS, B-mode ultrasound; CEUS, Contrast-enhanced ultrasound; DLR, Deep learning radiomics.

In contrast, DLR models—leveraging high-level hierarchical features—exhibited markedly superior discriminatory power. To demonstrate the stability of our DLR models, the performance was evaluated across repeated cross-validation. The DLR models exhibited highly consistent performance (Table S3). The Combination-DLR model achieved the highest overall performance (Table S4 and Figure 4A), with an AUC of 0.83 in the internal validation group and 0.72 in the external cohort. Notably, the CEUS-DLR model consistently outperformed the BUS-DLR model. This disparity indicates that dynamic hemodynamic signatures captured via deep learning provide more critical biological indicators for LNM than static structural information.

Table S3.

DLR models Performance Based on Repeated Cross-validation.

Model	Average AUC (95%CI)	Average Accuracy (95%CI)	Average Sensitivity (95%CI)	Average Specificity (95%CI)
BUS-DLR	0.84 (0.79 - 0.88)	0.87 (0.84 - 0.90)	0.90 (0.85 - 0.94)	0.85 (0.80 - 0.89)
CEUS-DLR	0.88 (0.85 - 0.92)	0.89 (0.85 - 0.92)	0.94 (0.90 - 0.98)	0.84 (0.79 - 0.89)
Combination-DLR	0.93 (0.90 - 0.97)	0.94 (0.92 - 0.97)	0.97 (0.95 - 1.00)	0.91 (0.87 - 0.95)

Table S4:

Performance evaluation of DLR models.

Training group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
BUS-DLR	0.97 (0.93 - 1.00)	0.93 (0.88 - 0.99)	1.00 (0.99 - 1.00)	0.88 (0.78 - 0.98)
CEUS-DLR	0.99 (0.97 - 1.00)	0.97 (0.93 - 1.00)	0.97 (0.91 - 1.00)	0.98 (0.93 - 1.00)
Combination-DLR	0.94 (0.89 - 0.99)	0.87 (0.79 - 0.93)	0.91 (0.83 - 1.00)	0.83 (0.70 - 0.93)
Internal validation group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
BUS-DLR	0.70 (0.57 - 0.93)	0.72 (0.52 - 0.88)	0.63 (0.55 - 1.00)	0.76 (0.53 - 0.94)
CEUS-DLR	0.77 (0.52 - 0.98)	0.80 (0.64 - 0.96)	0.75 (0.58 - 1.00)	0.82 (0.65 - 1.00)
Combination-DLR	0.83 (0.65 - 0.95)	0.88 (0.76 - 0.99)	0.88 (0.63 - 1.00)	0.88 (0.71 - 1.00)
External validation group
Model	AUC (95%CI)	Accuracy (95%CI)	Sensitivity (95%CI)	Specificity (95%CI)
BUS-DLR	0.66 (0.26 - 1.00)	0.70 (0.40 - 1.00)	0.80 (0.40 - 1.00)	0.60 (0.20 - 1.00)
CEUS-DLR	0.68 (0.28 - 1.00)	0.70 (0.40 - 0.90)	0.80 (0.40 - 1.00)	0.60 (0.20 - 1.00)
Combination-DLR	0.72 (0.34 - 1.00)	0.80 (0.60 - 1.00)	1.00 (0.50 - 1.000)	0.60 (0.20 - 1.00)

Figure 4.

The evaluation of DLR models. (A) Predictive performance of BUS-DLR model, CEUS-DLR model, and Combination-DLR model in training group and internal/external validation group. The DCA curve of the DLR models in the internal validation group (B) and the external validation group (C). Note: BUS, B-mode ultrasound; CEUS, Contrast-enhanced ultrasound; DLR, Deep learning radiomics; DCA, Decision Curve Analysis.

Comparison of DLR model and traditional Rad model

The superiority of DLR was further validated using the DeLong test (Table 3). In the training set, both BUS-DLR and CEUS-DLR models significantly outperformed their corresponding Rad models (P < 0.01). The Combination-DLR model yielded a higher AUC (0.94 vs. 0.88), though the difference was not statistically significant (P = 0.16), likely due to the already robust baseline performance of the Combination-Rad model. While the increase in AUC within validation cohorts did not reach statistical significance (P=0.29-0.88), this lack of significance is likely attributable to the inherent sample size constraints of the validation sets, which may have limited the statistical power to detect small yet clinically relevant differences.

As presented in Table 4, the Combination-DLR model demonstrated substantial improvements in both the NRI (0.79) and IDI (0.37) within the internal validation group (P<0.01). These results suggest that DLR features allow for a more precise risk stratification than manual radiomics. Finally, decision curve analysis (DCA) (Figure 4B-C) confirmed that the Combination-DLR model yielded the highest net benefit across a wide range of threshold probabilities, highlighting its potential to optimize preoperative management for HCCA patients.

Table 4:

Comparison of prediction ability between Rad models and DLR models.

Models	Internal validation group				External validation group
	NRI	P value	IDI	P value	NRI	P value	IDI	P value
BUS-Rad vs. BUS-DLR	0.41	0.14	0.28	0.10	-0.40	0.26	-0.29	0.22
CEUS-Rad vs. CEUS-DLR	0.68	<0.01	0.38	<0.01	0.20	0.62	0.11	0.66
Combination-Rad vs. Combination-DLR	0.79	<0.01	0.37	<0.01	0.20	0.55	0	0.97

Note: Rad, Radiomic; BUS, B-mode ultrasound; CEUS, Contrast-enhanced ultrasound; DLR, Deep learning radiomics.

Discussion and conclusion

In this study, the transfer learning strategy was employed to train deep learning models. The trained ResNet101 model served as a feature extractor to separately extract DLFs from BUS and CEUS images of HCCA. The GPSR algorithm was utilized for feature combination, yielding BUS-DLRFs, CEUS-DLRFs and Combination-DLRFs. Finally, three DLR models were constructed to predict the LNM. The results indicated that, in the training group, internal and external validation group, the diagnostic performance of DLR models integrating DLFs was superior to that of the corresponding Rad models.

By analyzing clinical factors, we found that common clinical indicators in HCCA patients lack potential value in predicting LNM, which was consistent with previous studies (36). Clinical parameters such as Bismuth-Corlette classification, TBIL, and CA19-9 are not independent risk factors for predicting preoperative LNM in HCCA patients (37). Meanwhile, our study treated clinical parameters as continuous or categorical variables without applying the Youden index—a statistical measure (J=sensitivity+specificity−1) used to determine the optimal cut-off value for maximizing diagnostic accuracy. This absence of threshold optimization might explain the lack of clinically significant LNM-associated indicators in our univariate analysis.

In clinical practice, it is difficult to obtain a sufficient number of medical images, leading to limited data availability (38). In our model training, a transfer learning strategy was adopted to reduce the impact of limited training data. Moreover, we used the ResNet101 model with transfer learning as a DLFs extractor rather than a classifier, thus circumventing the data scarcity problem and enhancing model accuracy (39). In this study, BUS-DLRFs, CEUS-DLRFs and Combination-DLRFs were constructed using the GPSR algorithm, which combined the RadFs and DLFs of BUS and CEUS. The results showed that the AUC of the Combination-DLR model exceeded 0.70 in both internal and external validation groups (internal validation group: 0.83, external validation group: 0.72). Moreover, DCA analysis also indicated that the Combination-DLR model had a higher predictive value.

In our comparative analysis of the validation cohorts, the Combination-DLR model consistently yielded a higher AUC than either the BUS-DLR or CEUS-DLR models; although these incremental improvements did not reach statistical significance (P > 0.05), we explicitly report these findings to underscore an observable clinical trend favoring multi-modal integration. This trend suggests that synergizing anatomical structures from BUS with the dynamic perfusion kinetics from CEUS—which captures microcirculatory blood flow across arterial, venous, and delayed phases—provides a more nuanced characterization of tumor heterogeneity (40, 41). The lack of statistical significance in these efficiency differences is likely attributable to the limited sample size, particularly within the external validation group (n=10), which constrains the statistical power to detect subtle performance gaps. Furthermore, the DLR models demonstrated varying degrees of improvement over their corresponding traditional Rad models, reinforcing the premise that DLFs capture complex, high-level image information that manual radiomics might overlook. Notably, while the Combination-DLR model showed exceptional generalizability, its relatively suboptimal performance within the training group compared to single-modality models likely indicates that the BUS-DLR and CEUS-DLR models experienced a degree of overfitting, whereas the multi-modal approach achieved a more balanced and robust representation of the clinical task.

There were some limitations of our study. First, we included only 110 HCCA patients. The small number of cases used for model construction and the insufficient external validation group, which consisted of only 10 cases, hindered an objective evaluation of the model’s generalization ability. The main reason could be the rarity of HCCA, despite it being the second-most common hepatobiliary tumor, which poses challenges in both diagnosis and obtaining accurate lesion images (42). Thus, prospective multi-center data are needed to confirm our result. Second, we used only one deep learning model for DLFs extraction and one machine learning model for model construction. With the development of the artificial intelligence field, new models and algorithms have emerged recently. In future research, other machine learning algorithms and deep learning networks should be explored to build a better prediction model. Finally, this study is primarily limited by the exclusion of other imaging modalities, such as CT or MRI, which may have restricted the breadth of the radiomic analysis. To further enhance predictive performance, future research should focus on developing multi-modal fusion frameworks that incorporate diverse imaging data. Integrating ultrasound with cross-sectional imaging or molecular diagnostics could provide a more comprehensive multi-dimensional characterization of HCCA, thereby optimizing the model’s clinical utility for LNM prediction.

This study developed DLR models with DLRFs for non-invasive prediction of LNM in HCCA patients before surgery. Compared to traditional Rad models, DLR models extracting and integrating intricate features from BUS and CEUS images are more efficient. Future work should validate these models in large-scale, multi-center studies to confirm clinical value.

Abbreviations

HCCA:

Hilar Cholangiocarcinoma

LNM:

Lymph Node Metastasis

TNM:

Tumor Node Metastasis

AJCC:

American Joint Committee on Cancer

OS:

Overall Survival

LNR:

Lymph Node Ratio

CT:

Computed Tomography

MRI:

Magnetic Resonance Imaging

PET:

Positron Emission Tomography

US:

Ultrasonography

CEUS:

Contrast-Enhanced Ultrasound

DLR:

Deep learning radiomics

BUS:

B-mode Ultrasound

ROI:

Region Of Interest

RadFs:

Radiomic Features

SGD:

Stochastic Gradient Descent

DLFs:

Deep Learning Features

mRMR:

Minimum Redundancy Maximum Relevance

LASSO:

Least Absolute Shrinkage and Selection Operator

GPSR:

Genetic Programming-based Symbolic Regression

DLRFs:

Deep Learning Radiomic Features

XGBoost:

eXtreme Gradient Boosting

AUC:

Area Under Receiver Operating Characteristic Curve

Rad:

Radiomic

NRI:

Net Reclassification Index

IDI:

Integrated Discrimination Improvement

CNN:

Convolutional Neural Network

KNN:

K-Nearest Neighbor

DCA:

Decision Curve Analysis

Authors’ Contributions:

HS and JMS conceived and designed the study. LW, KHL, MYX and LLW acquired the data. HS, JMS and YZ analyzed and interpreted the data. HS prepared figures 1-4. HS, YT and TYH drafted the manuscript. GLH, XYX and BXL revised it critically for important intellectual content. All authors read and approved the final version of the manuscript.

Funding:

This study was supported by the Guangdong Basic and Applied Basic Research Foundation (No.2020A1515111055, No.2023A1515220152) and National Natural Science Foundation of China (No.82102057, No.82102061).

Data Availability:

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Competing Interests:

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethics Approval and Consent to Participate:

This study was approved by the local ethics committee of the First Affiliated Hospital of Sun Yat-sen University and the informed consent requirement was waived. All methods were carried out in accordance with relevant guidelines and regulations.

Declaration on the Use of AI:

The authors declare that no generative AI was used in the preparation of this manuscript.