A deep-learning radiomics-based lymph node metastasis predictive model for pancreatic cancer: a diagnostic study

Abstract

Objectives:

Preoperative lymph node (LN) status is essential in formulating the treatment strategy among pancreatic cancer patients. However, it is still challenging to evaluate the preoperative LN status precisely now.

Methods:

A multivariate model was established based on the multiview-guided two-stream convolution network (MTCN) radiomics algorithms, which focused on primary tumor and peri-tumor features. Regarding discriminative ability, survival fitting, and model accuracy, different models were compared.

Results:

Three hundred and sixty-three pancreatic cancer patients were divided in to train and test cohorts by 7:3. The modified MTCN (MTCN+) model was established based on age, CA125, MTCN scores, and radiologist judgement. The MTCN+ model outperformed the MTCN model and the artificial model in discriminative ability and model accuracy. [Train cohort area under curve (AUC): 0.823 vs. 0.793 vs. 0.592; train cohort accuracy (ACC): 76.1 vs. 74.4 vs. 56.7%; test cohort AUC: 0.815 vs. 0.749 vs. 0.640; test cohort ACC: 76.1 vs. 70.6 vs. 63.3%; external validation AUC: 0.854 vs. 0.792 vs. 0.542; external validation ACC: 71.4 vs. 67.9 vs. 53.5%]. The survivorship curves fitted well between actual LN status and predicted LN status regarding disease free survival and overall survival. Nevertheless, the MTCN+ model performed poorly in assessing the LN metastatic burden among the LN positive population. Notably, among the patients with small primary tumors, the MTCN+ model performed steadily as well (AUC: 0.823, ACC: 79.5%).

Conclusions:

A novel MTCN+ preoperative LN status predictive model was established and outperformed the artificial judgement and deep-learning radiomics judgement. Around 40% misdiagnosed patients judged by radiologists could be corrected. And the model could help precisely predict the survival prognosis.

Background

Highlights

• A novel modified multiview-guided two-stream convolution network preoperative lymph node status predictive model was established and outperformed the artificial judgement and deep-learning radiomics judgement.

• Around 40% misdiagnosed patients judged by traditional diagnostic method could be corrected.

Pancreatic cancer (PC) is one of the most lethal malignancies with a 5-year survival rate lower than 10%^1,2. It is characterized by early lymphatic metastasis, rapid progression, and a poor prognosis². Most patients have lymph node (LN) metastases when diagnosed³. The positive LN status is independently associated with a worse prognosis⁴. And the neoadjuvant chemotherapy can effectively ameliorate the prognosis among the LN positive population^5,6. Thus, the preoperative LN status evaluation is vital, which further influences the prognosis prediction and the subsequent treatment strategy formulation. However, there lacks a precise and effective way to predict the LN status till now.

The ‘gold standard’ of LN evaluation was histological, mostly reporting postoperatively⁴. Contrast-enhanced computed tomography (CT) has long been preferred for the preoperative staging of PC⁷. MRI and positron emission tomography (PET) are also considered effective imaging methods. Nevertheless, it is still hard to judge the nature of an enlarged LN. Besides the LN metastasis, the inflammation and the lymphatic obstruction could also result in similar radiological phenomena⁷. Current radiological diagnosis strongly relied on the observation of radiologists, which is time-consuming and subjective^8–11.

Models based on deep-learning techniques can be helpful for image analyses^8,12,13. LN metastasis is mainly driven by primary tumors. Therefore, the imaging analysis of the primary tumor can reveal subtle differences that relate to the risk of LN metastasis¹⁴. We sought to establish a deep-learning model predicting the preoperative LN. The model was established with primary tumor and peri-tumor radiomics features in contrast-enhanced CT images. And based on the radiomics model, clinical factors were integrated to establish a multivariate model with better performance.

Method

Patients

All eligible patients were consecutively enrolled from the authors’ affiliated hospital during January 2017 to December 2020. The inclusive criteria included: pathologically diagnosed as PC, preoperatively diagnosed as resectable PC. (Without American Joint Committee on Cancer T4 patients)¹⁵, the number of the harvested LN was more than 15¹⁶. The exclusive criteria included: without raw data of contrast-enhanced CT scan, without complete oncological data, heterogenous carcinoma. Finally, a total cohort of 363 PC patients were enrolled in this study. The subjects were randomly divided into the train cohort and the test cohort by 7:3 for the model establishment. A total of 83 patients from Weifang People’s Hospital and Huadong Hospital affiliated with Fudan University were enrolled for external validation. And finally, the cohort with 28 patients fitted the criteria.

The study protocol was approved by the institutional review board at the authors’ affiliated hospital. The local ethics committee waived the need for informed consent because the study was observational and retrospective. The study was undertaken according to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines¹⁷ and in accordance with the latest version of the Declaration of Helsinki. The work has been reported in line with the STARD (Standards for the Reporting of Diagnostic accuracy studies) criteria¹⁸, Supplemental Digital Content 1, http://links.lww.com/JS9/A556.

Image collection

All scans were performed on a dual-layer spectral detector CT (IQon spectral CT; Philips Healthcare, Best). A tube voltage of 120-kVp with tube current automatic modulation technology was used for each scanning, collimation of 64×0.625 mm, the pitch of 0.894 and a rotation time of 0.27 s. The imaging matrix was 512×512. The pancreatic parenchyma scan and portal vein scan were performed 37 s and 70 s after the contrast agent bolus. The contrast agent was iopromide (Ultravist370; Bayer). All images were reconstructed with 1 mm thickness and 1mm increment. The CT scan parameters for all 363 enrolled patients were consistent. The images were annotated with itksnap software¹⁹.

Deep-learning scheme

We proposed a multiview-guided two-stream convolution network (MTCN) to extract the deep-learning features (Fig. 1). To make full use of the tumor information and alleviate the difficulty of a small sample size, we designed a multiview-guided data augmentation scheme, where each CT case was sampled with multiviews centered on the tumor to obtain nine slices of size 64×64. Besides, an L1 norm loss function in the first fully connected layer was designed for feature sparsity constraints. The data and model-driven scheme focus the network on extracting the most relevant feature. ResNet-18 was used as the backbone architecture. The features extracted from each stream were concatenated and fused in the first fully connected layer to integrate two-phase information. Finally, the fused feature passed through another fully connected layer to predict whether lymphatic metastasis occurs. Note that the output from the first fully connected layer is the desired deep-learning feature. Using the cross-entropy loss function, we trained the model with known labels as supervision. After training the model, the deep-learning features were extracted using the horizontal slices. A total number of 512 deep-learning features were extracted from the tumor area of each case.

Figure 1.

Schematic diagram of the two-stream convolution network.

To remove the redundant and irrelevant features to the maximum extent, we used a two-step feature selection method based on the fivefold cross-validation. First, we performed feature selection on each fold based on the DX score method to obtain five feature subsets $\{f_1,f_2,f_3,f_4,f_5\}$ . The DX score is calculated as follows:

$$D(X)=\frac{{\left(m_{\text{positive}}-m_{\text{negative}}\right)}^2}{d_{\text{positive}}^2+d_{\text{negative}}^2}$$

where $m_{\mathit{positive}}$ and $d_{\mathit{positive}}$ ( $m_{\mathit{negative}}$ and $d_{\mathit{negative}}$ , respectively) are the mean value and the SD of the feature in the positive (negative, respectively) training dataset. DX score can effectively measure the feature difference between positive and negative samples. A higher score indicates a greater difference between positive and negative samples, that is, a greater ability to discriminate between positive and negative samples. Specifically, we ranked the data from highest to lowest based on the DX score. The top n (1 ≤ n ≤ N) features were selected and fed into the SVM classifier model with initial parameters in turn. Its classification performance was evaluated based on the accuracy rate. The value of n with the highest accuracy was selected.

Next, we took a concatenation of the five feature subsets and obtained an initial set of deep-learning features F ~. At this point, F ~ still had a large number of redundant features, so we used a forward stepwise approach to do feature selection again. Specifically, the SVM classifier was trained to start with one feature, and the number of features was increased sequentially. If the test accuracy decreased when the ith feature was added, the feature was removed until all features were judged to be finished.

Definition

The T and N stages were referred to in the 8^th American Joint Committee on Cancer staging system⁴. The N+ and N- referred to the LN positive and negative status, respectively. The gold standard of metastatic LN is pathological, which is based on the postoperative histological reports. MTCN scores were the characteristic value transformed from the predicting probabilities of the MTCN radiomics model. Radiologist judgement were the LN status was judged independently by two experienced radiologists who majored in the hepatopancreatobiliary radiology. When divergence occurred, a radiologic supervisor would host a discussion with both two radiologists and reach the consensus.

Statistical analysis

Normally distributed continuous variables were presented as mean±SD, and analyzed using Student’s t-test. Non-normally distributed continuous variables were presented as median (Q1–Q3), and analyzed using the Mann–Whitney U test. The Kolmogorov–Smirnov test was utilized for normality tests of continuous variables. Categorical variables were presented as percentage, and analyzed using the Pearson test.

The clinical model establishment was based on the logistic regression analysis. The nomogram was utilized for model visualization. Restricted cubic spline analysis was performed regarding continuous factors in the multivariate model. The receiver operative curve (ROC), the calibration curve, and the decision curve analysis (DCA) were utilized for the model evaluations and comparisons. Area under curve (AUC) and accuracy (ACC) were utilized for the quantitative comparisons. Overall survival (OS) was assessed using Kaplan–Meier curves and log-rank Tests. Association between categorical data was assessed using $\mathit{\chi }$ ² Test, while association between continuous data using the Spearman rank test, and association between categorical data and continuous data using the Kruskal–Wallis test.

The statistical analysis of this study was performed by R Studio software. P< 0.050 was regarded as statistically significant.

Results

The demographic and baseline data of the train and test cohorts were displayed in Table 1. No significant differences were observed. The baseline data of the external validation cohort were displayed in Supplemental Table 1, Supplemental Digital Content 2, http://links.lww.com/JS9/A557. Univariate logistic regression models regarding LN status were established for factor screening in the Train cohort (Table 2). Finally, the multivariate model was established based on the radiologist’s judgement, MTCN scores, CA125, and age (Table 2). A restricted cubic spline analysis was performed regarding CA125 in the multivariate model, and CA125 manifested a positive linear relationship with the odd ratio of LN metastasis. (Supp. Figure 1, Supplemental Digital Content 3, http://links.lww.com/JS9/A558). A nomogram was plotted for the convenience of clinical use (Fig. 2).

TABLE 1.

Demographic and baseline data in the train and test cohorts.

	Test cohort	Train cohort
N	109	254	P
Age (median [IQR])	62.00 [56.00, 71.00]	63.00 [57.00, 69.00]	0.934
Sex = Female (%)	39 (35.8)	93 (36.6)	0.974
BMI (median [IQR])	22.28 [20.69, 24.09]	22.21 [20.58, 24.01]	0.956
Tumor size (median [IQR])	3.00 [2.50, 4.00]	3.30 [2.50, 4.00]	0.576
LN metastasis (%)	56 (51.4)	140 (55.1)	0.589
AJCC T stage (%)			0.238
1	22 (20.2)	40 (15.7)
2	61 (56.0)	166 (65.4)
3	26 (23.9)	48 (18.9)
AJCC N stage (%)			0.799
0	53 (48.6)	114 (44.9)
1	33 (30.3)	84 (33.1)
2	23 (21.1)	56 (22.0)
LN metastasis (radiologist judged preoperatively) (%)	26 (23.9)	68 (26.8)	0.652
CA125 (median [IQR])	17.00 [11.80, 27.60]	18.70 [12.40, 36.15]	0.147
CA199 (median [IQR])	178.50 [45.30, 738.20]	181.90 [50.05, 733.27]	0.901
CEA (median [IQR])	4.08 [2.18, 5.73]	4.00 [2.39, 6.32]	0.566
AFP (median [IQR])	2.80 [2.01, 4.30]	2.88 [1.97, 4.14]	0.885

LN, lymph node; IQR, interquartile range.

TABLE 2.

Univariate and multivariate logistic regression analysis in the train cohort.

	Univariate factor screening			Multivariate model establishment
	OR	95% CI	P	OR	95% CI	P
MTCN score	121.24	65.47–224.51	<0.001	140.7	36.14–547.69	<0.001
Radiologist judgement	2.69	1.45–4.99	0.001	2.36	1.18–4.72	0.014
Tumor size	1.12	0.61–2.08	0.222
T1	Ref
T2	2.01	0.99–4.06	0.052
T3	2.29	0.97–5.40	0.058
Age	0.97	0.52–1.8	0.040	0.97	0.94–1	0.050
Sex	1.21	0.65–2.24	0.473
BMI	1.03	0.55–1.9	0.552
CA125	1.01	0.55–1.87	0.035	1.01	1–1.02	0.019
CA199	1	0.54–1.85	0.270
CEA	1.01	0.54–1.86	0.469
AFP	0.9	0.49–1.66	0.092

MTCN, multiview-guided two-stream convolution network model; OR, odd ratio.

Figure 2.

The nomogram of the MTCN+ model.

The models were established based on the radiologist’s judgement only (the artificial model), the MTCN scores only (the MTCN model), and the screened factors (modified MTCN [MTCN+] model), respectively, in the Train cohort. Then the models were compared via the ROC, the calibration curve, and the DCA in the Train and Test cohorts, respectively (Fig. 3). In addition, the accuracies of different models were displayed (Table 3). Moreover, external validation was performed by comparing three models (Fig. 4, Table 3).

Figure 3.

The ROC curves of Artificial, MTCN, and MTCN+ models with AUC displayed in Train cohort (A) and Test (B) cohort, respectively. The DCA comparing Artificial, MTCN, and MTCN+ models in Train (C) and Test (D) cohorts. The calibration curves comparing Artificial, MTCN, and MTCN+ models in Train (E) and Test (F) cohorts.

TABLE 3.

The ACC and AUC comparisons among different models.

	Train cohort			Test cohort			External validation
Models	Artificial	MTCN	MTCN+	Artificial	MTCN	MTCN+	Artificial	MTCN	MTCN+
ACC	56.7	74.4	76.3	63.3	70.6	76.1	53.5	67.9	71.4
AUC	0.592	0.793	0.823	0.640	0.749	0.815	0.542	0.792	0.854

ACC, accuracy; AUC, area under curve; Artificial, radiologist judging LN status; MTCN, multiview-guided two-stream convolution network model; MTCN+, modified multiview-guided two-stream convolution network model with clinical factors integrated.

Figure 4.

The ROC curve of Artificial, MTCN, and MTCN+ models with AUC displayed in External validation cohort.

The MTCN+ was also verified in evaluating the LN metastatic burden, in another word, to discriminate N1 and N2 status. (P=0.753 to discriminate between N1 and N2 stage; P=0.373 to assess the association between positive LN numbers and the predicted probabilities). In the total cohort, the survival analysis was performed as well. (N- vs. predicted N- OS: P=0.77, DFS: P=0.96; N+ vs. predicted N+ OS: P=0.86, DFS: P=0.97) (Fig. 5). Setting 2 cm as the tumor size cutoff, the total cohort was divided into small tumor and large tumor groups. And the models were compared in both groups to verify the model robustness in small tumor model group (Fig. 6). The survival curves of traditionally-judged N status and actual N status were displayed in Figure 7.

Figure 5.

The Kaplan–Meier curves comparing predicted LN status and actual LN status regarding OS (A) and PFS (B) in the total cohort.

Figure 6.

The ROC curves of Artificial, MTCN, and MTCN+ models with AUC displayed in the small tumor cohort (A) and the large tumor cohort (B) cohort, respectively.

Figure 7.

The Kaplan–Meier curves comparing traditional judged LN status and actual LN status regarding OS (A) and PFS (B) in the total cohort.

Discussion

The preoperative LN status of PC is important for the following treatment strategy formulation. The current mainstream option to assess the LN status is still the radiologist’s judgement of radiological examinations. However, the accuracy of the artificial work was not satisfactory enough^9–11. (Train cohort: 56.7%, Test cohort: 63.3%, External validation: 53.5%). There lacks an effective and accurate method till now.

The radiomics model, a way to improve radiological analysis efficiency and avoid the subjective observational errors, was already applied in the preoperative evaluations of several malignant diseases^{7,8,12,20–23}. Considering the vital role of preoperative LN status, many studies built models to predict LN metastasis based on radiomics models and clinical characteristics for diseases including gastric cancer²⁰, colorectal cancer²², intrahepatic cholangiocarcinoma²¹, and PC⁷. Traditional radiomics feature extraction relies on the precise pixel-level annotations, which is especially challenging for PC due to its small size and unclear boundaries. Moreover, the traditional radiomics-based methods are likely to ignore the potential information surrounding the tumors¹³. The deep-learning radiomics was then proposed. The deep-learning techniques can outperform the traditional radiomics model and help overcome the limitations above in the model establishment, which was proven in breast cancer¹³, thyroid cancer²³, and so on. Thus, we established the deep-learning MTCN radiomics model with satisfactory discriminative performance. (ACC: Train cohort: 74.4%, Test cohort: 70.6%).

The PET examination relies on the metabolic characteristic of tissues, and it is proposed to be able to precisely diagnose LN metastasis. However, the sensitivity of PET is reported as unsatisfactory previously as well²⁴. Also, considering the radioactivity and high price, the PET examination was actually not commonly used in clinical. CT, the mainstream preoperative examination for PC, manifested superiority over PET because of its convenience and economy.

However, considering the clinical and biochemical behaviors of PC, we sought to further improve the deep-learning radiomics model. The new model was established based on the CA125, age, radiologist judgement, and the MTCN scores. The radiologist judgement was the most common way at present⁸. CA125 was a traditional tumor marker of PC, which was reported closely associated with occult metastasis^25–29. And elder age was discovered a protective factor for LN metastasis. The similar prognostic effect of elder age was also reported regarding distant metastasis of PC in previous studies, which was probably due to the biological behaviors of the malignance^30,31. The MTCN model was discovered superior to the mainstream radiologist judgement (Train cohort AUC: 0.793 vs. 0.592; Train cohort ACC: 74.4 vs. 56.7%; Test cohort AUC: 0.749 vs. 0.640; Test cohort ACC: 70.6 vs. 63.3%; External validation AUC: 0.792 vs. 0.542; External validation ACC: 67.9 vs. 53.5%). And the MTCN+ model consisting of the MTCN scores and clinical features furtherly outperformed the MTCN model (Train cohort AUC: 0.823 vs. 0.793; Train cohort ACC: 76.3 vs. 74.4%; Test cohort AUC: 0.815 vs. 0.749; Test cohort ACC: 76.1 vs. 70.6%; External validation AUC: 0.854 vs. 0.792; External validation ACC: 71.4 vs. 67.9%). In other words, regarding the LN status, around 40% [(76.2–58.6%)/(100–58.6%)] misdiagnosed patients could be corrected by the MTCN+ model, which is clinically valuable. The ROC, calibration curve, and DCA furtherly certified the superiority of the new model. Considering the difficulties for radiomics in analyzing small tumors, the ACC and AUC were analyzed regarding different primary tumor size (ACC: small vs. large: 79.0 vs. 75.7%; Fig. 4). Though the model performed well in distinguishing LN positive and LN negative patients, it was not associated with the LN metastatic burden. (P=0.373). The new model was also verified in the long-term prognostic prediction. And the predicted LN status fitted well with the actual LN status, indicating the model’s preciseness in predicting the survival prognosis.

Some limitations in the study warrant emphasizing. Firstly, the external validation sample size was limited. There is no public dataset with both fine CT scan raw data and clinical data nowadays, so it is hard to obtain the external validation data. A prospective cohort should be built and utilized in the future. Multicenter design with space heterogeneity would help more. Secondly, the sample size of the Train cohort should be further increased for better model fitting.

In conclusion, in the study, we verified a new deep-learning algorithms and developed a novel MTCN+ predictive model, which outperformed the artificial judgement and deep-learning radiomics judgement. Around 40% of misdiagnosed patients judged by radiologists could be corrected by the MTCN+ model, which is essential in applying a suitable treatment schema to the right patients. The model performed steadily even among patients with small primary tumor size. And the model could help precisely predict the long-term survival prognosis, whose survivorship curve is almost identical to the corresponding pathological LN status.

Ethical approval

The ethical approval was exempted, because the study was observational and retrospective.

Source of funding

This study was supported in part by the Jiaotong University Medical-engineering Cross Fund (Grant number: YG2022QN007) and Shanghai Municipal Health Commission Clinical Research Youth Special Fund (Grant number: 20214Y0186).

Authors’ contribution

Y.J., W.W., B.S.: study concepts; N.F., W.W., Y.J.: study design; W.C., W.F.: data acquisition; N.F., W.F., X.Q.: quality control of data and algorithms; N.F., H.C.: data analysis and interpretation; N.F., W.F.: statistical analysis; N.F., H.C., X.Q.: manuscript preparation; W.C., Y.J., W.W.: manuscript editing; X.Q., W.W., B.S.: manuscript review.

Conflicts of interest disclosure

The authors report no conflicts of interest.

Research registration unique identifying number (UIN)

1. Name of the registry: Radiomics in Pancreatic Cancer.

2. Unique Identifying number or registration ID: NCT05658679.

3. Hyperlink to your specific registration (must be publicly accessible and will be checked): https://clinicaltrials.gov/ct2/show/NCT05658679?cond=NCT05658679&draw=2&rank=1.

Guarantor

Yu Jiang, Weishen Wang, Baiyong Shen.

Data availability statement

The data where our results derived from were from Pancreatic Disease Center, Shanghai Jiao Tong University School of Medicine Affiliated Ruijin Hospital. The original data were not publicly available and could only be shared with the permission of the ethics committee of Ruijin Hospital.

Provenance and peer review

Not commissioned, externally peer-reviewed.

Acknowledgements

We have to thank Dr. Qingrou Wang and Dr. Naiyi Zhu, who helped the preoperative lymph node judgement as the radiologists. Also, we're grateful for the CT scan parameter conclusion contributed by Dr. Yanzhao Yang. The Pathological Department of Ruijin Hospital helped output high-quality pathological reports. Moreover, we feel grateful for the help of external validation from General Surgery Department, Weifang People's hospital, and General Surgery Department, Huadong Hospital Affiliated to Fudan University. And we ought to thank Dr. Chenghong Peng, Dr. Xiaxing Deng, Dr. Zhiwei XU, Dr. Jiancheng Wang and Dr. Jiabin Jin for their dedications as surgeons. Last but not the least, we honestly thank all the subjects enrolled in the study.

Supplementary Material

js9-109-2196-s001.pdf ^{(343.6KB, pdf)}

js9-109-2196-s002.docx ^{(144.1KB, docx)}

js9-109-2196-s003.docx ^{(57.2KB, docx)}