¹⁸F-FDG PET-based ensemble deep learning model for the prediction of lymphovascular invasion in colorectal cancer patients

Abstract

Objective

The aim of this study was to investigate the diagnostic performance of the 2.5-dimensional (2.5D) ensemble deep learning (DL) model based on ¹⁸F-fuorodeoxyglucose (FDG) positron emission tomography (PET) images in predicting lymphovascular invasion (LVI) in colorectal cancer (CRC) patients.

Methods

In this retrospective study, 177 CRC patients who underwent preoperative ¹⁸F-FDG PET/computed tomography were enrolled and assigned to the training cohort or the internal test cohort. Three inputs were determined according to the manually-delineated tumor volume of interest (VOI): the PrimaryLesion-2.5D input containing only the tumor VOI, the ProximalPeritumoral-2.5D input extending 10 mm outward from the VOI boundary, and the DistalPeritumoral-2.5D input extending 20 mm outward from the VOI boundary. Five common DL algorithm models, including VGG16, Googlenet, ResNet50, DenseNet201, and Vision Transformer were evaluated. Support vector machine was used to integrate the model outputs with good prediction performance to establish the Fusion model. The Radiomics and Clinical models were constructed for comparative analysis. The performance of the model was statistically analyzed by the area under the curve (AUC), accuracy, F1-score, parameter amount and inference time.

Results

The ProximalPeritumoral-DenseNet201 model (training cohort: AUC = 0.840, accuracy = 0.772, F1-score = 0.714; internal test cohort: AUC = 0.738, accuracy = 0.796, F1-score = 0.645; parameter amount = 18.097 M, inference time = 47.500 ms) and PrimaryLesion-ResNet50 model (training cohort: AUC = 0.746, accuracy = 0.740, F1-score = 0.628; internal test cohort: AUC = 0.733, accuracy = 0.704, F1-score = 0.619; parameter amount = 23.512 M, inference time = 38.200 ms) achieved an optimal balance between performance and computational efficiency. The performance of the Fusion model combined with the ProximalPeritumoral-DenseNet201 model and PrimaryLesion-ResNet50 model was further improved, with an AUC of 0.874, an accuracy of 0.821, and an F1-score of 0.766 in the training cohort. In the internal test cohort, the AUC was 0.824, the accuracy was 0.815, and the F1-score was 0.722. The Fusion model outperformed the Radiomics and Clinical models. Moreover, it showed good clinical utility and calibration.

Conclusion

The 2.5D ensemble DL model based on ¹⁸F-FDG PET images performed well for the prediction of LVI in CRC, proving its potential as a precision medical support tool for CRC patients.

Supplementary Information

The online version contains supplementary material available at 10.1007/s00261-025-05282-3.

Introduction

Colorectal cancer (CRC) is the third most common malignancy and the second most common cause of cancer-related death, with approximately 1.9 million new cases and 900,000 deaths globally in 2020 [1]. Currently, surgery is the primary treatment for early CRC, but 20%–30% of patients still have recurrence or metastasis after surgery [2]. Lymphovascular invasion (LVI) was identified when tumor cells or thrombi were located within an endothelium-lined space, or if tumor cells had compromised the lymphovascular wall [3]. LVI is a critical early step in local recurrence and distant spread of CRC, and it largely affects the clinical treatment decisions for CRC patients [4]. For example, preoperative adjuvant chemotherapy or radiotherapy reduces the incidence of LVI and improves the prognosis of patients with LVI-positive rectal cancer [4]. Therefore, preoperative determination of LVI status is of great clinical significance for formulating individualized treatment plans for CRC patients. However, the current diagnosis of LVI mainly relies on postoperative histopathological examination, which poses a challenge to the comprehensive preoperative evaluation of LVI in CRC.

¹⁸F-fuorodeoxyglucose (FDG) positron emission tomography (PET)/computed tomography (CT) has become an important clinical tool for the diagnosis, staging, and treatment evaluation of CRC. Although it is not possible to evaluate the LVI status of CRC patients based on visual imagery interpretation of ¹⁸F-FDG PET/CT images in clinical practice, a report suggested that radiomics features based on ¹⁸F-FDG PET/CT can quantify tumor heterogeneity and provide a potential tool for the comprehensive assessment of LVI status of CRC patients before noninvasive surgery [5]. Radiomics is a traditional machine learning approach that uses a manually selected input data representation to predict the target output, while deep learning (DL) uses an end-to-end mode of automatically learning hierarchical image features to eliminate the limitations of hand-crafted radiomics features [6]. Liu et al. found that the prediction model based on CT images using DL technology could be used to evaluate the LVI status of patients with stage ⅠA lung adenocarcinoma [7]. However, few studies have investigated the value of DL techniques based on ¹⁸F-FDG PET images to predict LVI status in CRC. In this context, combined with the advantages of ensemble learning [8], we constructed a 2.5-dimensional (2.5D) ensemble DL framework based on ¹⁸F-FDG PET for the comprehensive preoperative assessment of LVI status of CRC in this study. To the best of our knowledge, this study may be a preliminary study for the prediction of LVI status in CRC patients using ¹⁸F-FDG PET images combined with emerging DL techniques.

Materials and methods

Study cohorts

This was a retrospective single-center study. Data from a total of 296 patients with CRC confirmed by postoperative pathology in the First Affiliated Hospital of Harbin Medical University from February 2015 to May 2022 were retrospectively collected. The inclusion criteria were as follows: (1) patients underwent a ¹⁸F-FDG PET/CT scan 2 weeks before surgery; and (2) a primary adenocarcinoma with definite LVI status was diagnosed based on postoperative pathology. The exclusion criteria were as follows: (1) preoperative neoadjuvant therapy (radiotherapy, chemotherapy or targeted therapy); and (2) poor image quality preventing evaluation. A total of 177 patients were enrolled in the study, 70% of the study cohort (n = 123) was used as the training cohort to develop the DL model, and 30% of the study cohort (n = 54) was used as the internal test cohort to evaluate the DL model performance. This study was conducted under an institutional review board-approved protocol (NO.2023232) and in accordance with the principles of the Declaration of Helsinki. Due to the retrospective nature of this study, the ethics committee allowed participants to wave their informed consent. The cohorts used in this study were previously used to analyze the relationship between habitat features and LVI status in CRC [9].

Image acquisition and segmentation

The details of PET/CT image acquisition are provided in Appendix 1. Considering that it is difficult to determine the boundary of CRC lesions in plain CT images, only ¹⁸F-FDG PET images were analyzed in this study. Before volume of interest (VOI) annotation, all PET images were normalized to the ¹⁸F-FDG injection dose and attenuation-corrected patient weight, which were converted to standardized uptake value (SUV) units. To highlight the lesions, the images were observed within the SUV threshold range of 0–6. An nuclear medicine physician (Zhao, 4 years of experience) experience in LIFEx software (version 7.2.0; CEA-SHFJ, Orsay, France, www.lifexsoft.org) performed layer-by-layer tumor segmentation [10]. When the segmentation was uncertain, an expert nuclear medicine doctor (Lyu, 10 years of experience) was consulted for confirmation. The two physicians were blinded to the clinical and pathological information. Interobserver repeatability of lesion labeling was assessed in 20 randomly selected patients (see Appendix 2 for details).

Data preprocessing

First, the Nearest Neighbor algorithm was used to resample each raw image and VOI to an isotropic resolution of 1 × 1 × 1 mm³. The PET/CT image was multiplied with the tumor-segmented binary VOI mask. Then, the axial slice with the maximum cross-sectional area of the tumor (assumed to be layer n) was identified on the original 3-dimensional (3D) image, and the adjacent slices at the superior (n-1) and inferior (n + 1) in the axial direction were extracted from the principal slice. Subsequently, three slices were superimposed on the channel dimension. In addition, to compare and verify the potential value of the peritumoral area in evaluating tumor aggressiveness, we obtained 2.5D inputs containing different ranges of peritumoral areas during the process of cutting VOI: the PrimaryLesion-2.5D input containing only the tumor VOI, the ProximalPeritumoral-2.5D input extending 10 mm outward from the VOI boundary, and the DistalPeritumoral-2.5D input extending 20 mm outward from the VOI boundary. This schematic is reproduced in Fig. 1A.

Fig. 1.

A The scheme of obtaining 2.5D input data. B 2.5D DL ensemble model (Fusion model) structure diagram. 2.5-dimensional, 2.5D; deep learning, DL

DL model training

The 2.5D inputs were incorporated into the transfer learning strategy for the training of the model. Five common DL algorithm models, including VGG16, Googlenet, ResNet50, DenseNet201, and Vision Transformer were evaluated to directly distinguish the LVI status in CRC patients using an end-to-end method from the preprocessing of 2.5D input. We used the original network structures of the VGG16 [11], Googlenet [12], ResNet50 [13], DenseNet201 [14], and Vision Transformer [15] models, which were adapted for this research task. All model input channels were set to three to process 2.5D data, and the output layers were adjusted to binary classification to correspond to LVI negative and LVI positive statuses. All the input images of the VGG16, Googlenet, ResNet50, and DenseNet201 models were resized to 224 × 224 pixels. All images input to the Vision Transformer model were resized to 256 × 256 pixels, each image was divided into 64 × 64 pixel patches, the embedding feature dimension was 1024, the Transformer encoder contained 6 layers with 16 self-attention heads in each layer, and the hidden layer dimension of the multi-layer perceptron module was 2048. The DL model was initially pre-trained using the ImageNet dataset, and the weights of the pre-trained model were used as the initial state to start training. As part of the fine-tuning of the trained model, after each iteration, the weights of the DL model layers were updated according to the learning rate. We used the min–max transformation to normalize the gray value of the image slices to adjust the range to [− 1, 1], which was then used as the input to the model. The maximum training epoch was set to 100, with data augmentation applied during each epoch. The data augmentation consisted of a random left–right flip (with a probability 0.5), a random rotation (with a probability 0.5), and a random crop (with a probability 0.5). On each layer, the model with the smallest loss value was saved, and the performance of the model was evaluated on the internal test cohort at the end of training. For the images of the internal test cohort, the preprocessing procedure was restricted to only normalization without applying any data augmentation. We used a batch size of 16 and an SGD (learning rate = 0.001, weight_decay = 0.0001, momentum = 0.1) optimizer to train the model. Softmax was used as the output layer activation function and cross-entropy loss function as the loss.

Therefore, we trained 15 DL models based on three different 2.5D input data and five DL algorithms. In addition, based on the idea of ensemble learning, support vector machine (SVM) was used to integrate the output of the PrimaryLesion-ResNet50 model and the ProximalPeritumoral-DenseNet201 model, which had good performance among the 15 models, to build the Fusion model. Meanwhile, five-fold cross validation and random search were used to determine the optimal hyperparameter configuration (C = 9.112663083361808, gamma = 0.08192934757339093, kernel = poly, random_state = 42) in the SVM model training stage. To address the potential bias caused by class imbalance in the dataset (ratio of LVI negative to LVI positive: 77:46), we synthesized a virtual balanced training dataset (ratio of LVI negative to LVI positive: 77:77) using the synthetic minority oversampling technique (SMOTE) for modeling. The model structure is shown in Fig. 1B. To further validate the effectiveness of the 2.5D model in predicting LVI status, we constructed the DL model using 2-dimensional (2D) input and 3D input, respectively. The detailed process is shown in Appendix 3. For comparison, the development of both a Clinical model and a Radiomics model was performed, and the detailed process is presented in Appendix 4. The proposed model was built based on the PyTorch (version 1.11.0 + cu113, Meta, Menlo Park, CA, USA) framework and OnekeyAI (version 4.11.10, SingularityFlow Co. Ltd., Beijing, China) framework, and was trained on a machine equipped with an Intel CPU with Core (TM) i9-12900 K, 64 GB RAM, and NVIDIA GeForce GTX 3090ti GPU.

Interpretability of the model

To increase the transparency of the model’s decision process for identifying the LVI status, we used gradient-weighted class activation mapping (Grad-CAM) technology to visualize the model. The Grad-CAM algorithm was able to visualize the feature extraction during the learning process of the convolutional neural network as a heat map, thereby highlighting the input image regions that affected the prediction.

Model evaluation and statistical analysis

Receiver operating characteristic (ROC) curve analysis was used to evaluate the performance of the model, and the area under the curve (AUC), 95% confidence interval (CI), sensitivity, specificity, accuracy, and F1-score were calculated. The parameter amount of DL model and inference time were recorded to evaluate the model’s efficiency. The Delong test was used to test the differences in model performance. The calibration curve and Brier score were used to evaluate the differences between the predicted LVI results of the model and the actual LVI results to evaluate the accuracy of the model. Decision curve analysis (DCA) was used to evaluate the clinical utility of the model.

The Python programming language (version 3.9.7; Python Software Foundation, Amsterdam, Netherlands, www.python.org) and the R programming language (version 3.6.3; The R Foundation for Statistical Computing, Auckland, New Zealand, www.r-project.org) were used to perform statistical analyses and data visualization. For continuous variables, data following a normal distribution were expressed as the mean ± standard deviation (SD), and t-tests were used to determine statistical significance. Data with non-normal distributions were presented as the median (upper and lower quartiles) and analyzed using the Mann–Whitney U test. Categorical variables were expressed as rates or percentages using the Chi-square test or Fisher’s exact test. Two-tailed P-values < 0.05 were considered statistically significant.

Results

Baseline characteristics

A total of 177 CRC patients were retrospectively enrolled, of whom 34.5% (61/177) were females and 65.5% (116/177) were males. The average age of the cohort was 64.53 ± 10.84 years and the incidence of LVI positivity was 35.6% (63/177). The distribution of baseline characteristics of the patients in the training and internal test cohorts is shown in Table 1. Except for age, there were no statistically significant differences in any other clinical factors between the training and internal test cohorts.

Table 1.

Demographic data of patients in the training and internal test cohorts

Characteristics	Training cohort (n = 123)	Internal test cohort (n = 54)	P-value
Sex (%)			0.633a
Female	41 (33.3%)	20 (37.0%)
Male	82 (66.7%)	34 (63.0%)
Age (year, mean ± SD)	63.37 ± 10.80	67.17 ± 10.57	0.031b
Tumor location (%)			0.430a
Colon or sigmoid colon	65 (52.8%)	32 (59.3%)
Rectum or rectosigmoid junction	58 (47.2%)	22 (40.7%)
CEA
≤ 5 ng/mL	40 (32.5%)	16 (29.6%)	0.703a
> 5 ng/mL	83 (67.5%)	38 (70.4%)
TNM stage (%)			0.509a
Ⅰ-Ⅱ	59 (48.0%)	23 (42.6%)
Ⅲ-Ⅳ	64 (52.0%)	31 (57.4%)
Differentiation grade			0.137a
low differentiation	34 (27.6%)	21 (38.9%)
Medium or high differentiation	89 (72.4%)	33 (61.1%)
LVI (%)			0.449a
LVI negative	77 (62.6%)	37 (68.5%)
LVI positive	46 (37.4%)	17 (31.5%)

CEA carcinoembryonic antigen; LVI lymphovascular invasion; SD standard deviation

TNM stage: the 8th edition of the AJCC TNM staging system for CRC

^aChi-square test

^bStudent t-tests

2.5D DL model performance evaluation

Based on three different 2.5D input data and five DL algorithms, we trained 15 DL models. The performance of the DL models is summarized in Table 2. In the training cohort, the ProximalPeritumoral-DenseNet201 model, DistalPeritumoral-DenseNet201 model, and PrimaryLesion-ResNet50 model had the best performance. The AUC of the ProximalPeritumoral-DenseNet201 model was 0.840, the accuracy was 0.772, the F1-score was 0.714, the sensitivity was 0.761, and the specificity was 0.779. The AUC of the DistalPeritumoral-DenseNet201 model was 0.822, the accuracy was 0.772, the F1-score was 0.682, the sensitivity was 0.652, and the specificity was 0.844. The AUC of the PrimaryLesion-ResNet50 model was 0.746, the accuracy was 0.740, the F1-score was 0.628, the sensitivity was 0.587, and the specificity was 0.831. In the internal test cohort, the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model maintained relatively robust predictive performance. The AUC of the ProximalPeritumoral-DenseNet201 model was 0.738, the accuracy was 0.796, the F1-score was 0.645, the sensitivity was 0.588, and the specificity was 0.892. The AUC of the PrimaryLesion-ResNet50 model was 0.733, the accuracy was 0.704, the F1-score was 0.619, the sensitivity was 0.765, and the specificity was 0.676. Both models demonstrated good generalization ability and prediction stability between the training and internal test cohort. By contrast, the performance of the DistalPeritumoral-DenseNet201 model decreased in the internal test cohort, suggesting limited robustness. In addition, the other models did not achieve stable and good prediction performance in the training cohort and the internal test cohort.

Table 2.

Performance comparison of different DL models

The bold fonts denoted the best results

In terms of model complexity and inference efficiency, the ProximalPeritumoral-DenseNet201 model contained 18.097 M parameters and had an inference time of 47.500 ms, while the PrimaryLesion-ResNet50 model contained 23.512 M parameters and had an inference time of 38.200 ms. Because DenseNet and ResNet models achieve richer hierarchical feature extraction and information fusion through dense or residual connections, the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model performed worse in inference speed, but they were still at the millisecond level. Compared with VGG16 (134.269 M) and Vision Transformer (63.014 M) with a large parameter amount, DenseNet201 and ResNet50 were medium-scale models with lower computational costs. Although Googlenet (5.602 M) had the lowest parameter amount, it demonstrated poor prediction performance. Considering both overall performance and computational efficiency, the ProximalPeritumoral-DenseNet201 and PrimaryLesion-ResNet50 models were selected for the Fusion model construction. Details on the predictive performance of the DL models using 2D and 3D inputs can be found in Supplementary Table S1.

Prediction performance evaluation of the fusion model

The performance evaluation metrics of the model are shown in Table 3. The AUC of the Fusion model in the training cohort was 0.874 (95% CI: 0.812–0.936), the sensitivity was 0.783, the specificity was 0.844, the accuracy was 0.821, and the F1-score was 0.766. The AUC of the internal test cohort was 0.824 (95% CI: 0.700–0.947), the sensitivity was 0.765, the specificity was 0.838, the accuracy was 0.815, and the F1-score was 0.722. In this study, the effectiveness of the Fusion model structure was verified by ablation experiments. The DeLong test results showed that the AUC of the Fusion model was significantly better than that of the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model in the training cohort, and the AUC of the Fusion model was significantly better than that of the ProximalPeritumoral-DenseNet201 model in the internal test cohort. However, the difference from the PrimaryLesion-ResNet50 model did not reach statistical significance. In addition, the AUC of the ProximalPeritumoral-DenseNet201 model in the training cohort and the internal test cohort were higher than those of the PrimaryLesion-ResNet50 model, but none of them reached statistical significance. For further comparative analysis, we developed a Radiomics model and a Clinical model, respectively. The feature selection procedures for these models are detailed in Supplementary Table S2 and Supplementary Figure S2. The Fusion model performed better than the Radiomics model and Clinical model in the training and internal test cohorts, with statistical significance observed only in the training cohort. The ROC curve and DeLong test heat map are shown in Figs. 2 and 3.

Table 3.

The performance of prediction models in the training and internal test cohorts

Cohort	Model	AUC (95%CI)	Sensitivity	Specificity	Accuracy	F1-score
Training cohort	ProximalPeritumoral-DenseNet201	0.840 (0.768–0.912)	0.761	0.779	0.772	0.714
	PrimaryLesion-ResNet50	0.746 (0.657–0.835)	0.587	0.831	0.740	0.628
	Fusion model	0.874 (0.812–0.936)	0.783	0.844	0.821	0.766
	Radiomics model	0.649 (0.547–0.751)	0.652	0.649	0.650	0.583
	Clinical model	0.640 (0.553–0.727)	0.696	0.584	0.626	0.582
Internal test cohort	ProximalPeritumoral-DenseNet201	0.738 (0.581–0.895)	0.588	0.892	0.796	0.645
	PrimaryLesion-ResNet50	0.733 (0.594–0.872)	0.765	0.676	0.704	0.619
	Fusion model	0.824 (0.700–0.947)	0.765	0.838	0.815	0.722
	Radiomics model	0.661 (0.502–0.820)	0.647	0.676	0.667	0.550
	Clinical model	0.682 (0.558–0.806)	0.541	0.824	0.630	0.667

AUC area under the curve; CI confidence intervals

Fig. 2.

ROC curves of PrimaryLesion-ResNet50, ProximalPeritumoral-DenseNet201, Fusion model, Radiomics model and Clinical model in the training cohort (A) and internal test cohort (B). area under the curve, AUC; area under the curve, ROC

Fig. 3.

The heatmap of the DeLong test P-value for PrimaryLesion-ResNet50, ProximalPeritumoral-DenseNet201, Fusion model, Radiomics model and Clinical model in the training cohort (A) and internal test cohort (B)

The DCA results of the DL model, Radiomics model and Clinical model are presented in Fig. 4. DCA results showed that the Fusion model had a broader range of clinical benefit than the ProximalPeritumoral-DenseNet201 model, PrimaryLesion-ResNet50 model, Radiomics model and Clinical model, both in the training cohort and in the internal test cohort. The calibration curves for the prediction models are illustrated in Fig. 5. The calibration curves and Brier scores indicated that the Fusion model achieved better agreement between predicted and observed outcomes, reflecting higher predictive accuracy than the other LVI prediction models in the training cohort and the internal test cohort.

Fig. 4.

DCA curves of PrimaryLesion-ResNet50, ProximalPeritumoral-DenseNet201, Fusion model, Radiomics model and Clinical model in the training cohort (A) and internal test cohort (B). decision curve analysis, DCA

Fig. 5.

Calibration curves of PrimaryLesion-ResNet50, ProximalPeritumoral-DenseNet201, Fusion model, Radiomics model and Clinical model in the training cohort (A) and internal test cohort (B)

Interpretability of the model

Heat maps generated by Grad-CAM for the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model are shown in Fig. 6. The intensity magnitude in the heat map represents the importance of different regions in the ¹⁸F-FDG PET image for predicting the LVI status in CRC patients. The most activated regions in the ProximalPeritumoral-DenseNet201 model were mainly in the peritumoral region, while the most activated regions in the PrimaryLesion-ResNet50 model were mainly in the intratumoral region.

Fig. 6.

Grad-CAM heatmaps visualization results for the LVI positive (A) and LVI negative (B) images. In the Grad-CAM heatmaps, red indicates a higher predicted value and blue indicates a lower predicted value. gradient-weighted class activation mapping, Grad-CAM; lymphovascular invasion, LVI

Discussion

LVI is not only a subclinical marker of lymph node metastasis but also a strong risk factor for CRC recurrence and metastasis [5]. In clinical practice, preoperative prediction of LVI in CRC patients is essential for selecting appropriate treatment strategies. This study is the first to construct a hybrid model framework integrated with 2.5D DL for the non-invasive prediction of LVI status of CRC patients based on ¹⁸F-FDG PET images. The AUC of the model was 0.874 (95% CI: 0.812–0.936) in the training cohort and 0.824 (95% CI: 0.700–0.947) in the internal test cohort. This result highlights the simpler method to identify CRC patients at elevated risk of LVI, with potential value to further guide the personalized treatment of CRC patients.

DL and radiomics are quantitative methods for image analysis that are capable of extracting high-dimensional abstract digital information that goes beyond what can be perceived through the visual evaluation of a given image [16]. Therefore, the study by Yang et al. attempted to use ¹⁸F-FDG PET/CT radiomics features to evaluate LVI in CRC patients. They performed a retrospective analysis of ¹⁸F-FDG PET/CT images from 95 CRC patients with definite pathological LVI status. The results showed that the AUC of the ¹⁸F-FDG PET radiomics feature in the training cohort and validation cohort were 0.764 and 0.872, respectively [5]. The results of this study further confirmed the predictive value of radiomics for LVI status in CRC. The experimental results showed that the AUC values of the Radiomics model in the training cohort and the internal test cohort were 0.649 and 0.661, respectively. However, it should be noted that in the technical process of radiomics analysis, it is first necessary to extract radiomics features from the region of interest and conduct post-analysis through machine learning. This process involves many manually dependent feature engineering and selection algorithms, which poses challenges to the reproducibility of the model in new data sets [17]. DL can overcome these limitations by automatically extracting features from raw data. However, there is a lack of studies exploring the value of DL technology based on ¹⁸F-FDG PET images in predicting LVI status in CRC patients. As a preliminary exploratory study, we found that the Fusion model based on DL technology had a better prediction effect and clinical utility than the Radiomics model in comparative analysis. This finding further suggests that DL can provide more representative imaging information than traditional radiomics [18]. Although the results of this study cannot be directly compared to those of previous studies due to the use of different cohorts, the DL model constructed in this study had two significant advantages over prior radiomics studies. First, the Fusion model was based on a convolutional neural network, which can directly associate the input data with the target output, and enable the end-to-end prediction of LVI status from ¹⁸F-FDG PET images of CRC patients. Such a process can eliminate the limitation of the manual dependence of the radiomics technical process. Second, our study recruited more patients, and a larger sample size would guarantee better reliability of the classification model.

In addition, DL has been used to predict LVI status in a variety of malignancies. For example, Liu et al. reported that DL features extracted from CT images through 3D residual neural networks were used to predict LVI in patients with stage ⅠA lung adenocarcinoma. The AUC of the internal and external validation cohorts of the combined model constructed by DL features and clinical factors were 0.74 and 0.77, respectively [7]. Yang established a 3D ResNet-10 model based on magnetic resonance imaging to evaluate the LVI status of breast cancer patients. The AUC of that model was 0.740 [19]. A 3D DL model can comprehensively use the spatial information of medical tumor images, but the accuracy of the 3D DL model in predicting LVI status in lung cancer and breast cancer needs to be improved. The reason may be due to the fact that the availability of pre-trained 3D medical imaging models is rare, and most of the models are based on 2D images and are pre-trained on natural RGB images [17].

A DL model with a 2.5D structure was constructed in this study, which was different from previous studies. 2.5D DL combines the advantages of 2D and 3D convolutional neural networks in design. On the one hand, the 2.5D DL model can compensate for the lack of spatial information of single-layer 2D images through the joint input of adjacent slices, and can capture part of 3D spatial information [20]. This strategy enables the model to achieve a comprehensive representation of lesion morphology and local spatial continuity without significantly increasing the computational complexity. Our results also showed that the 2.5D DL model had better prediction results while having the same level of inference speed as the 2D DL model. On the other hand, the 3D DL model has a larger parameter amount than the same 2.5D DL model. Therefore, training 3D DL models effectively requires a higher computational cost and large-scale datasets [21]. The 2.5D model based on the architecture modification of the pre-trained 2D model can directly use the model weights trained by the large-scale natural image database such as ImageNet, which is more convenient for the application of transfer learning techniques than the 3D DL model [20]. This not only accelerates the convergence speed of the model, reduces the dependence on large sample data, but also improves its generalization performance and robustness. Our results also showed that under the condition of the same hardware device and small sample data, the prediction performance of the 3D DL model was lower than that of the same 2.5D DL model. Meanwhile, the running efficiency of the 3D DL model was much lower than that of the 2.5D DL model. However, the 2.5D DL model structure still has some limitations. Since the input only contains a limited number of adjacent layers, the model can only capture the local spatial continuity, and it is difficult to completely reflect the spatial topological characteristics of the tumor in the whole volume. Nevertheless, compared with the full 3D model, the 2.5D model is between the 2D and 3D models in spatial representation ability, but in the clinical context of limited training samples at the present stage, the 2.5D model achieves a good balance between computational cost and clinical usability.

When comparing 15 DL models with 2.5D input, we found that the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model had the most stable prediction performance in the training cohort and the internal test cohort, but the inference speed was slower than other models. This is mainly due to the deep network layer number and complex connection structure of DenseNet201 and ResNet50. The deep number of network layers and complex connection structure enable the model to fully learn the hierarchical features and subtle differences of tumor regions, thereby improving the discrimination performance, but the dense connections and residual connections introduce additional feature aggregation and memory access overhead in the inference stage, which leads to the good prediction performance of the model accompanied by higher computational complexity. However, the inference time (inference time < 50 ms) of the ProximalPeritumoral-DenseNet201 model and the PrimaryLesion-ResNet50 model was still within the acceptable range, and both had a moderate parameter amount, so we chose these two models as the base learners of the final ensemble model.

During the analysis of 2.5D inputs containing different ranges of peritumoral regions, we found that the ProximalPeritumoral-DenseNet201 model performed best, followed by the PrimaryLesion-ResNet50 model. Tumor cells are usually highly aggressive and destroy the normal structure of the surrounding parenchymal tissue, leading to cancerous infiltration of small blood vessels and lymphatic vessels around the lesion, which is often overlooked in studies focusing on the area within the tumor [22]. Our research confirmed that a peritumoral area with a certain range could better capture the invasive characteristics of CRC tumor cells than focusing only on the intratumoral region. The Grad-CAM results also indicated that the ProximalPeritumoral-DenseNet201 model focused most of its attention on the tumor margin area. Importantly, previous studies had also demonstrated that non-smooth tumor margins were an independent predictor of microvascular invasion in hepatocellular carcinoma [23, 24]. Meanwhile, the Grad-CAM results of the PrimaryLesion-Resnet50 model showed that there were high-risk areas reflecting the invasiveness of the tumor that required attention within the tumor. In addition, we noted that the DistalPeritumoral-DenseNet201 model performed well in the training cohort (AUC = 0.822, accuracy = 0.772, F1-score = 0.682); however, performance in the internal test cohort was lower (AUC = 0.685, accuracy = 0.630, F1-score = 0.583). Thus, when the outward expansion area is too large and contains too much irrelevant information, those confounding factors may interfere with the stable prediction of the model.

In this study, we integrated the output of two 2.5D DL models with different ranges of peritumoral regions using the SVM model. We believe that this hybrid model framework, based on ensemble learning technology, can focus on the heterogeneity of different regions of the tumor through different branches to achieve accurate assessments of LVI status and significantly improve the prediction of the model. The results of our ablation experiments also confirmed the validity of the model proposed in this study through multiple dimensions, such as the accuracy of the model prediction, its clinical utility, and calibration.

The results of our analysis of clinical factors showed that only TNM stage was an independent risk factor for LVI, suggesting that LVI positive CRC patients were more likely to enter a later TNM stage. Previous reports had stated that the larger the tumor and the deeper the invasion depth, the probability of tumor invasion into the surrounding lymphatic and vascular tissues increased [25, 26]. At the same time, the Clinical models based on TNM staging also demonstrated predictive value for LVI status, although the comprehensive performance of the Clinical models was lower than that of our Fusion model. In addition, age, sex, tumor location, carcinoembryonic antigen (CEA), and differentiation grade had limited value in assessing LVI status in CRC patients, which was consistent with previous results [4, 5].

This study had the following limitations. Firstly, this study was a single-center retrospective study with a relatively limited sample size and a lack of an external validation cohort, which may have led to the risk of selection bias and insufficient generalization ability of the model. Although we introduced transfer learning and data augmentation strategies in the training process to alleviate the impact of insufficient samples, the external generalization ability of the model still needs further verification in a multi-center and prospective study. Second, there was a certain imbalance in the distribution of LVI negative and positive samples. To alleviate this problem, we adopted the SMOTE technique in the model training phase to construct a balanced training set by generating virtual samples. In addition, we used F1-score as a performance evaluation metric to comprehensively evaluate the accuracy and stability of the model in the context of imbalanced data. However, the inclusion of a more balanced real-world cohort would facilitate a better trade-off between sensitivity and specificity. Third, manual delineation of lesion volume was used for VOI definition in the study, although the results of our interobserver repeatability assessment suggested that this method is relatively robust. However, manual labeling introduced variability. In the future, automatic or semi-automatic segmentation algorithms can be combined to reduce human interference and improve scalability. Fourth, this study mainly focused on the diagnostic performance of the model for LVI status, while its prognostic value and potential in clinical decision-making have not been explored. Future prospective studies are warranted to further evaluate the utility and potential guiding significance of the model in a real clinical setting. Finally, it was not possible to explore the effect of different scanner platforms on the model. Although the preprocessing process prior to model training helped standardize the model establishment, differences in image quality and acquisition parameters must be evaluated.

In conclusion, we showed that the 2.5D ensemble DL model based on preoperative ¹⁸F-FDG PET images provided a non-invasive and practical method to predict LVI in CRC patients. The predictive model provided valuable information for patient care and contributes to the personalized treatment of CRC patients, although additional prospective multicenter validation studies are needed to obtain high-level evidence for clinical translation.

Supplementary Information

Below is the link to the electronic supplementary material.

Abbreviations

2D

2-Dimensional

2.5D

2.5-Dimensional

3D

3-Dimensional

AUC

Area under the curve

CRC

Colorectal cancer

CT

Computed tomography

CI

Confidence interval

CEA

Carcinoembryonic antigen

DL

Deep learning

DCA

Decision curve analysis

FDG

Fuorodeoxyglucose

Grad-CAM

Gradient-weighted class activation mapping

LVI

Lymphovascular invasion

PET

Positron emission tomography

ROC

Receiver operating characteristic

SUV

Standardized uptake value

SVM

Support vector machine

SD

Standard deviation

SMOTE

Synthetic minority oversampling technique

VOI

Volume of interest

Author contributions

All authors contributed to the study conception and design. Hongyue Zhao and Zhehao Lyu provided the overall design of the experiments and contributed equally to the experiments. Ziyue Zhang and Zhehao Lyu were responsible for the collection of clinical and imaging data. Yexin Su, Hongyue Zhao and Zhehao Lyu completed tumor segmentation. Hongyue Zhao, Yexin Su, Huiting Zhang and Peng Xu performed the data analysis. Hongyue Zhao, Zhehao Lyu and Yexin Su wrote and edited the manuscript. Peng Fu and Zhongwei Lv supervised the study. All authors read and approved the final manuscript.

Funding

National Natural Science Foundation of China, 82371994, Cultivation Project of the Joint Fund, Heilongjiang Provincial Natural Science Foundation, PL2024H077.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Declarations

Conflict of interest

The authors declare no competing interests.