Skip to main content
NCBI home page
eClinicalMedicine logoLink to eClinicalMedicine
. 2023 Oct 24;65:102269. doi: 10.1016/j.eclinm.2023.102269

Development and validation of CT-based radiomics deep learning signatures to predict lymph node metastasis in non-functional pancreatic neuroendocrine tumors: a multicohort study

Wenchao Gu a,b,c,o, Yingli Chen b,d,o, Haibin Zhu e,o, Haidi Chen d,f,g,h,i, Zongcheng Yang j, Shaocong Mo k, Hongyue Zhao l, Lei Chen m, Takahito Nakajima a, XianJun Yu d,f,g,h,i, Shunrong Ji d,f,g,h,i,∗∗∗∗, YaJia Gu b,d,∗∗∗, Jie Chen d,f,n,∗∗, Wei Tang b,d,∗
PMCID: PMC10725026  PMID: 38106556

Summary

Background

Lymph node status is an important factor for the patients with non-functional pancreatic neuroendocrine tumors (NF-PanNETs) with respect to the surgical methods, prognosis, recurrence. Our aim is to develop and validate a combination model based on contrast-enhanced CT images to predict the lymph node metastasis (LNM) in NF-PanNETs.

Methods

Retrospective data were gathered for 320 patients with NF-PanNETs who underwent curative pancreatic resection and CT imaging at two institutions (Center 1, n = 236 and Center 2, n = 84) between January 2010 and March 2022. RDPs (Radiomics deep learning signature) were developed based on ten machine-learning techniques. These signatures were integrated with the clinicopathological factors into a nomogram for clinical applications. The evaluation of the model’s performance was conducted through the metrics of the area under the curve (AUC).

Findings

The RDPs showed excellent performance in both centers with a high AUC for predicting LNM and disease-free survival (DFS) in Center 1 (AUC, 0.88; 95% CI: 0.84–0.92; DFS, p < 0.05) and Center 2 (AUC, 0.91; 95% CI: 0.85–0.97; DFS, p < 0.05). The clinical factors of vascular invasion, perineural invasion, and tumor grade were associated with LNM (p < 0.05). The combination nomogram showed better prediction capability for LNM (AUC, 0.93; 95% CI: 0.89–0.96). Notably, our model maintained a satisfactory predictive ability for tumors at the 2-cm threshold, demonstrating its effectiveness across different tumor sizes in Center 1 (≤2 cm: AUC, 0.90 and >2 cm: AUC, 0.86) and Center 2 (≤2 cm: AUC, 0.93 and >2 cm: AUC, 0.91).

Interpretation

Our RDPs may have the potential to preoperatively predict LNM in NF-PanNETs, address the insufficiency of clinical guidelines concerning the 2-cm threshold for tumor lymph node dissection, and provide precise therapeutic strategies.

Funding

This work was supported by JSPS KAKENHI Grant Number JP22K20814; the Rare Tumor Research Special Project of the National Natural Science Foundation of China (82141104) and Clinical Research Special Project of Shanghai Municipal Health Commission (202340123).

Keywords: Non-functional pancreatic neuroendocrine tumor, Radiomics, Deep learning, Lymph node metastasis, CT

Research in context.

Evidence before this study

We conducted a literature search on Web of Science and PubMed using the search term “(radiomics OR deep learning) AND (prediction OR predict) AND (lymph node metastasis) AND (non-functional pancreatic neuroendocrine tumors)” for the time period up to June 30, 2023, without any language restrictions. No studies were found that met these criteria. However, when we broadened the search to include “pancreatic neuroendocrine tumors,” we identified one original study that employed radiomics analysis to predict the aggressive characteristics of pancreatic neuroendocrine neoplasms. One of the outcomes of this study was the prediction of lymph node metastasis (LNM) with an area under the curve (AUC) of 0.72 with 101 samples. It is important to note that this single study, which used radiomics to predict LNM in pancreatic neuroendocrine neoplasms (panNEN), is not directly comparable to studies focusing on non-functional pancreatic neuroendocrine tumors (NF-PanNETs), as panNEN encompasses all types of tumors, including both functional and non-functional. These are distinct types of tumors with different biological behaviors.

Added value of this study

To the best of our knowledge, no study has utilized imaging techniques based on radiomics and deep learning methods to predict LNM in NF-PanNETs. We have developed a novel radiomics deep learning signature (RDPs) that can accurately predict LNM in patients with NF-PanNETs. This signature has the potential to address the limitations of clinical guidelines regarding the 2-cm threshold for tumor lymph node dissection and to provide precise therapeutic strategies. Additionally, it exhibits a strong association with disease-free survival.

Implications of all the available evidence

Our RDPs serves as a non-invasive tool that supports clinical decision-making for precision surgical treatment in patients with NF-PanNETs. The clinical significance of our study lies in two points. First, for NF-PanNETs ≤ 2 cm, if the predicted risk of LNM is low, follow-up or minimally invasive enucleation can be more assured. On the contrary, active intervention is required and standard oncologic resection is performed. Second, for tumors >2 cm, minimally invasive enucleation may be performed if the risk of LNM is predicted to be low. Instead, standard oncologic resection is performed. Additionally, it provides recommendations for NF-PanNETs patients regarding the utilization of neoadjuvant therapy before surgical resection. Future prospective multicenter studies are imperative for the validation of our findings.

Introduction

Pancreatic neuroendocrine tumors are rare heterogeneous pancreatic neoplasms, which are mostly non-functional and often found incidentally.1, 2, 3 In recent years, there has been a rise in the occurrence of non-functional pancreatic neuroendocrine tumors (NF-PanNETs) due to the rapid development of medical imaging techniques. Surgery is the primary treatment for NF-PanNETs; however, the treatment method still needs to improve, especially, the optimal lymphadenectomy procedures remain unestablished and inconsistent.4

Lymph node metastasis (LNM) is a significant prognostic factor for NF-PanNETs.5, 6, 7 Furthermore, the National Comprehensive Cancer Network (NCCN) and European Neuroendocrine Tumor Society (ENETS) guidelines, which generally recommend LN dissection for PanNET patients either with a tumor diameter larger than 2 cm or with potential risk of LNM.8,9 However, it is crucial to acknowledge that a substantial proportion of tumors ≤2 cm also exhibit LNM accompanied by a considerable risk of recurrence.10, 11, 12 This could precipitate in distant metastases, ultimately leading to reduced disease-related survival (DFS).13, 14, 15 Therefore, identifying patients at a higher risk of LNM prior to surgery is crucial, as it enables identifying those who could potentially benefit from lymphadenectomy.

However, the accuracy of diagnosing LNM before surgery using traditional medical imaging is challenging as it relies on evaluating the minimal axis diameter of the lymph node and contrast enhancement on CT and MRI or tracer uptake in PET scans. This approach has inherent limitations, as it poses the potential risk of underestimating the extent of the disease by missing smaller micro-metastatic lesions and overestimating them in the presence of benign inflammatory processes that can result in the enlargement, enhancement and uptake of lymph nodes on images. Therefore, there is an urgent need to develop a new method to predict preoperative LNM.

Radiomics is a useful technique for quantitative medical imaging using handcrafted high-throughput features of tumor regions.16,17 It has been applied to predict prognosis and LNM in serval cancers.18, 19, 20, 21 Leveraging artificial intelligence technologies such as deep learning (DL) to autonomously derive quantitative representations from medical images is an evolving direction in radiomic research. The DL approach, based on convolutional neural networks, is an emerging method for LNM prediction and pathological classification.22,23 Recently, deep transfer learning technique has gained popularity as a research topic as it involves fine-tuning a pretrained DL network to perform a new task, making it possible to apply DL to small datasets. Both radiomics and deep learning are rapidly evolving technologies that have been used recently. But radiomics remains the predominant approach for high-throughput imaging data analysis, largely due to its interpretability advantages over DL. To date, there is no authoritative statement or consensus suggesting that deep learning is inherently superior to radiomics. Therefore, the combination of DL with radiomic features may lead to exceptional performance in predicting LNM.

Despite the increasing number of publications on NF-PanNETs, no studies have reported the use of radiomics and DL for LNM prediction. In this study, we developed radiomic deep learning signatures (RDPs) based on enhanced CT imaging for preoperative LNM discrimination in NF-PanNETs.

Methods

Patients’ data acquisition

This two-center retrospective study received approval from the Research Ethics Committee of the Institutional Review Boards from all participating hospitals, and the need for informed consent was exempted. We retrospectively gathered data from 320 patients with histopathology-validated NF-PanNETs across two centers between January 2010 and March 2022. The training and internal validation cohorts of Center 1 (Fudan university of Shanghai Cancer Center) were composed of 142 and 94 patients, respectively. And the external validation cohort of Center 2 (Peking University Cancer Hospital and Institute) was composed of 84 patients. The inclusion and exclusion criteria for this study are shown in the Supplementary File and Fig. 1. The initial clinical and pathological information for each patient, such as age, sex, tumor size, vascular invasion, differentiation grade, liver metastasis, and perineural invasion, was obtained from medical records. The study design and pipeline are shown in Fig. 2.

Fig. 1.

Fig. 1

Open in a new tab

Flowchart depicts the patient enrollment process in the study of Center 1 (A) and the Center 2 (B).

Fig. 2.

Fig. 2

Open in a new tab

Schematic representation of the study's design and procedural steps. (A) Segmentation was conducted on atrial phase CT images. (B) Radiomics and deep learning features were extracted separately. (C) Feature selection was based on the U-test, ICC, and LASSO. (D) A radiomics deep learning signature (RDPs) was developed from the combined radiomics and deep learning scores. This RDPs was further integrated with clinical factors to create the RDPs-nomogram model. (E and F) The predictive capability of the RDPs for lymph node metastasis (LNM) was assessed and externally validated in a separate center’s cohort.

CT imaging acquisition and image segmentation

All the patients performed CT scan before surgical resection. The details of CT scan protocol and imaging processing could be seen in Supplementary File. For this study, manual segmentation of the arterial phase in CT images of NF-PanNET tumors was performed by radiologists. Two senior radiologists (WT and HBZ have over than 10 years of experience in the field of pancreatic imaging) drew the regions of interest with the tumor using ITK-snap version: 3.6.0.24 The completed delineation images were saved as mask files in nifti format. A month later, a random selection of 30 patients was made, and the regions of interest were re-delineated by the two radiologists. Intraclass correlation coefficients (ICCs) were calculated as a measure of consistency in tumor delineation.

Radiomics features extraction

We utilized the package of Pyradiomics (https://github.com/Radiomics/pyradiomics)25 which is based on the Python 3.7 platform, to extract the radiomics features. The standardization of the feature extraction algorithm was conducted based on the guidelines proposed by the Image Biomarker Standardization Initiative.26 The radiomics features included first-order features, shape, grey-level run length matrix, grey-level size zone matrix, grey-level co-occurrence matrix, grey-level dependence matrix, and neighboring grey-tone difference matrix. Details of the radiomics features are mentioned in the Supplementary File.

Deep learning features extraction

We cropped the region of interest with the largest transverse area which contains the whole tumor. The grayscale images were transformed using windows of WL = 50 and WW = 350 to normalize the background information and minimize noise interference prior to imaging. Using linear interpolation, the input image was resized to the dimensions of 224 × 224 pixels while normalizing the mean and standard deviation of the pixel intensity to 0 and 1, respectively. Next, we used the Resnet101 pre-trained model based on the PyTorch package and fine-tuned our DL models to improve the performance of the convolutional neural networks in detecting LNM. The used backbone was the original pre-trained ImageNet (http://www.image-net.org). We referred to the cosine-annealing learning rate decay algorithm when setting the parameters for the task-specific part. The details can be found in the Supplementary Files.

Features selection and model construction

We established a three-stage approach within the training cohort for dimensionality reduction and the selection of robust features pertaining to both deep learning and radiomics features, respectively. We first selected 30 random CT images for the process of ROI segmentation and feature extraction, then evaluated the stability and reproducibility of these features by calculating ICCs. Features exhibiting ICCs exceeding 0.8 were considered acceptably reproducible and selected for further analysis. Next, all subsequent steps were based on training cohort (n = 142). Subsequently, we conducted the Student's t-test or Mann–Whitney U-test. Features with an adjusted p-value below 0.05 were selected and normalized using the z-score. Ultimately, we employed the least absolute shrinkage and selection operator (LASSO) logistic regression method to identify the most predictive features with non-zero coefficients, using penalty parameter tuning via 10-fold cross-validation. Following the LASSO analysis, the selected features were incorporated into 10 machine learning methods, including glmBoost, GBM, Enet, SVM, plsRglm, XGBoost, LDA, Ridge, NaiveBayes, and LASSO, to construct the radiomics and DL models based on the training cohort. Finally, an RDPs was developed based on the radiomics and DL signatures using logistic regression. The RDPs was developed utilizing a training cohort and its validity was confirmed through internal and external validation cohorts. Additional details of this process can be found in the Supplementary File.

Evaluation and development of the clinical-based RDPs nomogram

The significance of clinical factors in predicting LNM was evaluated using both univariate and multivariable logistic regression analyses in center 1. A p-value < 0.05 was chosen for the construction of the clinical nomogram. Finally, the RDPs-nomogram was developed by incorporating these clinical factors. In addition, the area under the curve (AUC) acquired through receiver operating characteristic curve (ROC) analysis, accuracy, sensitivity, specificity, negative predictive value (NPV), positive predictive value (PPV), calibration curves, and decision curve analysis were employed for the evaluation of each model. The DeLong test was utilized to compare the AUCs of different models.

The potential of prediction for the recurrence in NF-PanNETs

To assess the capability of the RDPs to predict the recurrence of NF-PanNETs, we conducted Kaplan–Meier analysis based on DFS. The log-rank test was employed to ascertain the significance of the observed discrepancies, median was set as cutoff.

Statistics analysis

All figures were generated with the R software (http://www.R-project.org; Version 4.2.1). The criteria of statistical significance was set as a p-value < 0.05 or adjust p-value < 0.05. The parametric test of Student’s t-test and non-parametric of Mann–Whitney U test were performed to compare the differences between two groups. The adjust p value was calculated by the Benjamini-Hochberg correction and Chi-square test was used to analyze the categorical variables. The 95% confidence interval (CI) of the AUC was determined using the bootstrapping method (1000 intervals). All machine learning model analysis was conducted utilizing the “glmnet” and “caret” package, while the generation of radar plots was accomplished using the “ggplot2” package. The plotting of ROC curves was facilitated through the implementation of the “pROC” package. Univariant, multivariable logistic regression, Kaplan–Meier curve analysis, nomogram development, and calibration curve creation were performed employing the “rms”, “survminer” and “survival” packages. The calibration curves were derived through bootstrapping utilizing 1000 resamples and were concurrently subjected to the Hosmer–Lemeshow goodness-of-fit test for evaluation by “Resource Selection” package.

Role of the funding source

The research fundings for the study did not influence the study design, data gathering, data processing, data interpretation, or the composition of the manuscript. All researchers reviewed, deliberated, and gave their approval for the final version of this manuscript. Every author had complete access to the data and endorsed the final manuscript for submission.

Results

Patients’ characteristics

Between January 2010 and March 2022, a total of 1002 patients were diagnosed with NF-PanNETs in Center 1 and Center 2. A total of 236 and 84 patients with NF-PanNETs from Center 1 and 2, respectively, were enrolled in this study. In Center 1, the patients were randomly allocated into a training (n = 142/236, 60%) and internal validation cohorts (n = 94/236, 40%). The patients from Center 2 were set as the external validation cohort. The training cohort composed of 142 patients (mean age, 52.8 ± 12.4, 72 women), the internal validation cohort comprised 94 patients (mean age, 50.8 ± 12.5, 38 women), and the external validation cohort included 84 patients (mean age, 52.4 ± 12.2, 42 women). There were no substantial variations observed were found in the age, grade, sex, vascular invasion, perineural invasion, or tumor size between all the cohorts (p > 0.05). In all cohorts, 67 (47.2%), 46 (48.9%), and 27 (32.1%) patients, respectively, had LNM (p < 0.05). The clinical characteristics of the patients are outlined in Table 1.

Table 1.

Clinical characteristics.

Characteristics Training cohort (n = 142)
 Internal validation cohort (n = 94)
 External Validation cohort (n = 84)
 p value
No. (%) No. (%) No. (%)
Sex
 Female 72 (50.7%) 38 (40.4%) 42 (50.0%)
 Male 70 (49.3%) 56 (59.6%) 42 (50.0%) 0.26
Age (Years)
Mean (SD) 52.8 (12.4) 50.8 (12.5) 52.4 (12.2)
Median [MIN, MAX] 53.5 [20,77] 52.5 [23,76] 55 [21,79]
Lymph node metastasis
 Yes 67 (47.2%) 46 (48.9%) 27 (32.1%)
 No 75 (52.8%) 48 (51.1%) 57 (67.9%) 0.04a
WHO classification
 G1 46 (32.4%) 27 (28.7%) 29 (34.5%)
 G2 84 (59.2%) 58 (61.7%) 43 (51.2%)
 G3 12 (8.5%) 9 (9.6%) 12 (14.3%) 0.52
Liver metastasis
 Yes 39 (27.5%) 19 (20.2%) 16 (19.0%)
 No 103 (72.5%) 75 (79.8%) 68 (81.0%) 0.25
Vascular invasion
 Yes 55 (38.7%) 35 (37.2%) 25 (29.8%)
 No 87 (61.3%) 59 (62.8%) 59 (70.2%) 0.38
Perineural invasion
 Yes 54 (38.0%) 32 (34.0%) 23 (27.4%)
 No 88 (62.0%) 62 (66.0%) 61 (72.6%) 0.26
Recurrence
 Yes 46 (32.4%) 24 (25.5%) 27 (32.1%)
 No 96 (67.6%) 70 (74.5%) 57 (67.9%) 0.49
Tumor size
 >2 cm 95 (66.9%) 61 (64.9%) 47 (56.0%)
 ≤2 cm 47 (33.1%) 33 (35.1%) 37 (44.0%) 0.24
Open in a new tab
a

Significant.

The feature extraction and selection

A total of 1834 radiomics and 2045 DL features were extracted from NF-PanNETs CT imaging. After deleting the features that showed poor reproducibility (ICCs < 0.8), a total of 1487 radiomics and 1552 DL features were selected for the Student’s t-test or Mann–Whitney U test. In total, 772 radiomics and 1292 DL features were selected for LASSO to shrink and determine the optimized feature numbers for the machine learning model construction. Finally, nine radiomics and 11 DL features were determined to construct the radiomics and DL signature (Supplementary Fig. S1 A–D). The Pearson correlation results demonstrated that the correlation for each feature was lower than 0.5 in both radiomics and DL features, respectively (p > 0.05, Fig. 3A and B). The radiomic features of R1 (Exponential_GLCM_LDN) and D8 (DP19) showed the highest coefficient weights in the radiomics and DL models, respectively (Fig. 3C and D and Supplementary Table S2).

Fig. 3.

Fig. 3

Open in a new tab

The Person correlation (r) and the coefficient weight of radiomics features (A, C). The Person correlation (r) and the coefficient weight of DL features (B, D).

Radiomics and DL signatures validation

Among the 10 machine learning methods, we selected glmBoost as our algorithm for constructing the model owing to its superior performance in training cohort, as evidenced in both the radiomic (AUC, 0.85; 95% CI: 0.79–0.91) and DL models (AUC, 0.84; 95% CI: 0.78–0.90) (Fig. 4A–F; Supplementary Table S3 and S4). It shows similar AUCs of 0.84 (95% CI: 0.76–0.92) for the radiomics model and 0.85 (95% CI: 0.77–0.92) for the DL model in the internal validation cohort. In both the radiomics and DL models, the training and internal validation cohorts showed not significant differences in predicting LNM (DeLong test, p > 0.05; Fig. 4C, F). Moreover, both in center 1 and center 2, the radiomic and DL models did not show significant differences (DeLong test, p > 0.05; Fig. 4G and H).

Fig. 4.

Fig. 4

Open in a new tab

The radar plot showing the area under the curve (AUC) value of each machine learning algorithms in training and internal validation cohorts for the radiomics (A and B) and DL signatures (D and E). The ROC curves represent the training and internal validation cohort for both radiomic and DL signatures, respectively (C, F). The ROC curves of radiomic and DL signatures in Center 1 and Center 2 (G and H).

RDP signature performance and validation

Finally, an RDPs was developed based on radiomics and DL models. The AUC was 0.89 (95% CI: 0.83–0.94) in the training cohort, 0.87 (95% CI: 0.80–0.94) in the internal cohort, and 0.91 (95% CI: 0.85–0.97) in the external validation cohort (Fig. 5A, Table 2). The calibration curve plot indicated all cohorts were well-calibrated with the actual observation (Supplementary Fig. S2 A–C; p > 0.05). The decision curves for the RDPs in predicting LNM across the three cohorts suggest that it provides a good net benefit (Fig. 5B). The AUC value of RDPs was 0.88 (95% CI: 0.84–0.92) in Center 1 (Supplementary Fig. S3A), and the decision curve analysis plot is shown in Supplementary Fig. S3B. Moreover, based on the univariate and multivariable logistic regression results, the vascular invasion, perineural invasion, tumor grade, and RDPs were independent factors for the prediction of LNM (Table 3; p < 0.05). Therefore, the RDPs-nomogram model was constructed based on these factors and showed an AUC value of 0.93 (95% CI: 0.89–0.96) for predicting LNM (Fig. 5C and D). The calibration curves for the RDPs nomogram revealed a well-calibrated alignment between the model-predicted LNM and actual observations (Fig. 5E). In an endeavor to address the controversy surrounding the clinical guidelines on whether to perform lymph node dissection for tumors at the 2 cm threshold, we assessed the predictive capability of LNM in relation to tumors size at 2 cm threshold. Of note, the predictive performance for tumor sizes ≤2 cm exhibited AUC values of 0.90 (95% CI: 0.82–0.99), 0.87 (95% CI: 0.75–0.99), and 0.93 (95% CI: 0.83–1.00) in three cohorts, respectively (Fig. 5F–H). Similarly, for tumor sizes >2 cm, the AUC values were 0.87 (95% CI: 0.80–0.94), 0.86 (95% CI: 0.77–0.95), and 0.91 (95% CI: 0.85–0.98) in all cohorts, respectively (Fig. 5F–H). For both tumor size ≤2 cm and >2 cm, the AUC values of Center 1 were 0.90 and 0.86, respectively (Supplementary Fig. S4). To elucidate the correlation between the relative RDPs and other features, a correlation triangle plot was constructed (Fig. 5I). The plot revealed significant correlations between the RDPs and each feature, with p < 0.05 and a correlation coefficient (r) greater than 0.4 (Fig. 5I).

Fig. 5.

Fig. 5

Open in a new tab

The ROC curves demonstrate the prediction performance of the RDPs in the training, internal validation, and external validation cohorts (A). In a wide range of decision threshold probability, the net benefit RDPs using the decision curve analysis (DCA) decision curves of training, internal validation and external validation cohort (B). The RDPs-nomogram and corresponding ROC curve (C and D). The calibration curve for RDPs-nomogram is presented (E). The RDPs prediction performance at the threshold of 2 cm of tumor size in the training, internal validation and external validation cohorts, respectively (F–H). The triangular correlation plot illustrates the relationships among the RDPs, radiomics, and DL features (I).

Table 2.

The diagnostic performance of RDP signature in predicting LNM in NF-PanNETs in the training, internal and external validation cohort.

AUC 95% CI Accuracy Sensitivity Specificity PPV NPV p value
Training cohort 0.89 0.83–0.94 0.77 0.82 0.73 0.73 0.82
Internal validation cohort 0.87 0.80–0.94 0.76 0.76 0.75 0.74 0.77
External validation cohort 0.91 0.85–0.97 0.87 0.91 0.79 0.97 0.82
Training cohort vs Internal validation cohort 0.75
Training cohort vs External validation cohort 0.43
Internal validation cohort vs External validation cohort 0.56
Open in a new tab

AUC, area under the curve; CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value.

Table 3.

Univariate and multivariable logistic regression.

Univariate analysis
 p value Multivariable analysis
 p value
OR (95% CI) OR (95% CI)
Recurrence 5.51 (2.93–10.36) <0.001
Liver metastasis 6.35 (3.14–12.84) <0.001
Vascular invasion 18.10 (9.01–36.37) <0.001 6.74 (2.67–17.02) 0.000
Perineural invasion 8.82 (4.71–16.48) <0.001 2.66 (1.05–6.73) 0.04
Sex 0.89 (0.54–1.47) 0.64
Age 0.99 (0.97–1.01) 0.47
Grade 7.58 (2.17–26.49) 0.002 6.74 (1.36–33.52) 0.02
Tumor size 4.28 (2.16–8.50) <0.001
RDPs 237.72 (65.86–858.09) <0.001 93.18 (19.23–451.50) 0.000
Open in a new tab

OR, odds ratio; RDPs, radiomics deep learning signature.

RDP signature associated with DFS

We examined the prognostic significance of the RDPs in two Centers based on the follow-up information of the patients. In Center 1, the median follow-up duration was 326 days with 70 endpoint events (42.1%) related to DFS. In Center 2, the median follow-up duration of 488 days with 27 endpoint events (32.1%) were associated with DFS. Our RDPs demonstrated a strong association with DFS in both Center 1 and Center 2 (log-rank test, p < 0.0001; HR, 8.4; 95% CI: 3.6–19 and p < 0.0001; HR, 14; 95% CI: 5–42, respectively; Fig. 6A and B). The heatmap displays the distribution of the chosen radiomics and DP features for constructing the RDPs within the high- and low-risk groups in Center 1 and Center 2, respectively (Fig. 6C and D). The Sankey diagram presents the patient distribution among the high- or low-risk groups, recurrence, tumor size, and LNM status (Fig. 6E and F).

Fig. 6.

Fig. 6

Open in a new tab

The KM plot show the RDPs associated with DFS in Center 1 and Center 2 (A and B). The heatmap shows the features distribution between high and low risk group (C and D). The Sankey diagram of the correspondence relationships of RDPs subgroup among the all patients with recurrence, tumor size, and lymph node status in Center 1 and Center 2 (E and F).

Discussion

The pancreas has important internal and external secretory functions. And the pancreas is a non-regenerative organ. Excessive resection of the pancreas may lead to hyperglycemia and affect the quality of life of patients, especially the average age of onset of pancreatic neuroendocrine tumors is relatively young (<50 years). Besides, indiscriminate lymph node dissection can also increase surgical risk. Surgical resection is the foremost treatment option according to the NF-PanNETs treatment guidelines. Recently, with the development of minimally invasive techniques, surgery to preserve pancreatic parenchyma, such as enucleation, is becoming more and more mature. The size of the tumor is no longer a decisive factor in whether enucleation can be performed.27 Whether the tumor is associated with high risk factors, especially LNM, is the key to carry out pancreatic parenchyma-sparing surgery and influencing patient prognostic outcomes.28,29 Nonetheless, consensus regarding the extent of the surgical approach, especially lymph node dissection, is lacking, partly because the preoperative prediction of LNM is challenging. Primarily, it is difficult to define LNM using current conventional imaging, and it largely depends on the experience of the radiologist. Therefore, it is necessary to find a new, more accurate method for detecting LNM, such as radiomics and deep learning approaches. To date, there are no studies on the application of radiomics and/or deep learning features for predicting LNM in NF-PanNETs. Consequently, this study presents an innovative radiomics deep learning-based model to predict LNM and recurrence in the patients with NF-PanNETs prior to surgical resection using preoperative CT images. Our RDPs demonstrated AUC of 0.88 and 0.91 in Center 1 and Center 2, respectively, and was strongly associated with DFS. Our follow-up analysis findings are aligned with numerous studies which showed associations between LNM and inferior DFS outcomes.6,30,31 Moreover, the combination of clinical factors and RDPs improved the prediction of LNM (AUC: 0.93). However, there were no observable significant differences between the RDPs and the RDPs-nomogram models. Some studies reported that the tumor grade is an independent predictor of LNM, which is consistent with our results.32,33 Nevertheless, some clinical factors rely on pathological findings which are often not reliably accessible prior to surgery, thus limiting their practical application. Consequently, the only RDPs were more reliable for the translation to clinical.

For most tumors, pathology serves as the gold standard for diagnosing LNM. However, it is difficult to obtain preoperative pathological results because of the anatomical position of the pancreas. Biopsy results may also be unreliable due to the heterogeneity of NF-PanNETs.34 Therefore, LNM is always predicted using preoperative imaging. Numerous investigations have explored the precision of CT and MRI in differentiating LNM using imaging characteristics such as imaging enhancement and tumor size.35, 36, 37, 38 Among these studies, tumor size showed the cutoff value of 1.5–4 cm to be associated with LNM.28,39, 40, 41 These findings demonstrate the challenges and inconsistencies in identifying LNM through appraisal of tumor dimensions. Our study proposed that tumor size did not function as a self-sufficient risk indicator for LNM in the context of multivariable analysis, aligning with the conclusion of prior studies.26,35,42 On the other hand, some studies show the low sensitivity 65% and 12% for prediction the LNM by the CT and even the target molecular imaging such as 68Ga-DOTATOC.43,44 However, our results indicate that the RDPs exhibit higher sensitivities of 82%, 76%, and 91% in each cohort, respectively. One of the primary factors that may contribute to these results is that nodal lesions are not directly visible in images, and micro-metastases may have already occurred. Therefore, in our study, the RDPs were developed based on the primary tumor rather than direct examination of the lymph nodes.

In clinical guidelines, there is no unified standard for determining the tumor size that dictates the need for lymph node dissection. The NCCN and ENETS guidelines offer differing treatment recommendations for NF-PanNETs, though both emphasize pancreatectomy with regional lymphadenectomy for tumors >2 cm. Additionally, the NCCN guidelines recommend lymphadenectomy for tumors between 1 and 2 cm when there is potential risk of LNM.8,9,45 However, the guidelines do not elaborate on what is a high-risk factor. Due to the absence of explicit guidelines in clinical practice, the decision to perform lymph node dissection for nodes under the 2 cm threshold currently relies on the clinical judgment and experience of the physician. Therefore, the potential for an inappropriate lymphadenectomy to occur still exists. To address this issue, we have also conducted predictions on LNM at this 2 cm threshold. Our findings indicate that the RDPs exhibits robust performance in predicting LNM for both ≤2 cm and >2 cm NF-PanNETs. The results were validated in the internal/external cohorts and demonstrated that the patients with tumors ≤2 cm and having high RDPs with an elevated likelihood of LNM should undergo surgery and lymph node resection. In contrast, patients with low RDPs have a reduced probability of LNM, follow-up or minimally invasive enucleation can be more assured. For tumors >2 cm, minimally invasive enucleation may be performed if the risk of LNM (low RDPs) is predicted to be low. Instead, standard oncologic resection is performed (high RDPs). Of note, in Center 1, among the 80 patients with NF-PanNETs having tumors ≤2 cm, 25 (31.5%) exhibited LNM. Similarly, in Center 2, 9/37 (33.3%) patients with NF-PanNETs having tumors ≤2 cm displayed LNM. Comparable findings have been documented in previous studies.11,12,46 Given this evidence, it is essential to determine whether tumors ≤2 cm in size have LNM. Because the surgeons acquire the accurate LNM status before operation is very important to choosing the individualized surgical method for patients. Our RDPs signature provides guidance on the presence of preoperative LNM and addresses the limitations in clinical guidelines regarding performing lymph node dissection for tumors at the 2-cm threshold.

In our study, one of the primary inclusion criteria was that patients had undergone lymph node resection, specifically those who had more than 12 lymph nodes resected. As for surgical procedure, we adhere to a consistent surgical approach, with lymph node resection being carried out based on tumor location. Additionally, all LNM was confirmed through pathological examination. Therefore, all the patients underwent the systematic treatment process. Moreover, patients with liver metastases were included. Unlike PDAC metastatic disease, surgical resection did not bring any survival benefit for patients. However, radical surgery for NF-PanNETs, even cytoreductive surgery of >90% of metastatic disease may provide long survival benefits. Some studies suggest that neoadjuvant chemotherapy or radioimmunotherapy is beneficial for patients with NF-PanNETs who have a high potential for recurrence.47, 48, 49 Notably, our RDPs is also significantly associated with DFS. Hence, the RDPs can be instrumental in identifying patients with a high potential for recurrence, who may benefit from neoadjuvant therapy before undergoing surgical resection.

Our study is subject to certain limitations. First, due to the rarity of NF-PanNETs, the patient sample was restricted to two centers, which may limit the external validity and generalizability of our findings to diverse populations, particularly as the external validation cohort displayed a higher AUC. Moreover, overfitting is a critical issue that can arise during model construction. In our approach, we carefully selected the features that demonstrated weak inter-correlations. All cohorts exhibited consistent predictive capabilities. As such, the improved discrimination performance of RDPs in the external validation cohort likely stems from the limited sample size and selection bias, rather than model overfitting. Hence, future larger-scale, multicenter, prospective studies are essential to validate our findings. Second, it is also important to note that incorporating and comparing additional imaging modalities, such as 68Ga-DOTA or 18F-FDG PET imaging, with CT imaging could have enhanced the robustness of our analysis. Third, we were unable to precisely identify the affected lymph nodes owing to the RDPs was derived from the primary tumor. Fourth, we demonstrated the novel RDPs as a final product based radiomics and deep learning score to predict the LNM in NF-PanNETs. However, as outlined in the methods section, our current RDPs require several steps for construction. Therefore, for future clinical practice, in-house software is needed to facilitate easier access to RDPs for every clinician (Supplementary Fig. S5).

In summary, our RDPs may have the potential to preoperatively predict LNM in NF-PanNETs, address the insufficiency of clinical guidelines concerning the 2-cm threshold for tumor lymph node dissection, and provide precise therapeutic strategies. In particular, for NF-PanNETs ≤ 2 cm, a low RDPs of LNM allows for confident follow-up or minimally invasive enucleation, whereas a higher RDPs necessitates standard oncologic resection. For tumors >2 cm, a low RDPs suggests minimally invasive enucleation, but a higher RDPs again calls for standard oncologic resection.

Contributors

WG, WT, YG, SJ, and JC designed the study. WG, YC, and HZ integrated and analyzed the data. YC and HZ collected the clinical data and analysis. WG, WT, and JC obtained funding. WG, ZY, SM, and HZ prepared the figures. WG, YC, HZ, HC, LC, WP, XY, and TN revised the manuscript. WT and SJ supervised the study. All authors had full access to all the data. All authors were involved in drafting the manuscript. All authors read and approved the final manuscript.

Data sharing statement

All the data are available for the reasonable request from the corresponding author which the data approved by the institutional review board of all enrolled centers request.

Declaration of interests

All other authors declare no competing interests.

Acknowledgements

We gratefully thank Prof. Yoshito Tsushima, Prof. Wei Jun Peng and Dr. Li Xie for their invaluable insights and critical scientific discussions regarding this project. The authors would like to thank Shoko Miyazaki, Tomoko Saito for the assisting with laboratory work. We appreciate Chang Liu for consulting on the graphic abstract. This work was supported by Clinical Research Special Project of Shanghai Municipal Health Commission (202340123); JSPS KAKENHI Grant Number JP22K20814 and the Rare Tumor Research Special Project of the National Natural Science Foundation of China (82141104).

Footnotes

Appendix A

Supplementary data related to this article can be found at https://doi.org/10.1016/j.eclinm.2023.102269.

Contributor Information

Shunrong Ji, Email: jishunrong@fudanpci.org.

YaJia Gu, Email: cjr.guyajia@vip.163.com.

Jie Chen, Email: chen0jie@hotmail.com.

Wei Tang, Email: tangwei105@163.com.

Appendix ASupplementary data

Supplementary Material, Figs. S1–S5 and Tables S1–S4
mmc1.docx (588.4KB, docx)

References

  • 1.Dasari A., Shen C., Halperin D., et al. Trends in the incidence, prevalence, and survival outcomes in patients with neuroendocrine tumors in the United States. JAMA Oncol. 2017;3(10):1335–1342. doi: 10.1001/jamaoncol.2017.0589. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Oronsky B., Ma P.C., Morgensztern D., Carter C.A. Nothing but NET: a review of neuroendocrine tumors and carcinomas. Neoplasia. 2017;19(12):991–1002. doi: 10.1016/j.neo.2017.09.002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Lee M.R., Harris C., Baeg K.J., Aronson A., Wisnivesky J.P., Kim M.K. Incidence trends of gastroenteropancreatic neuroendocrine tumors in the United States. Clin Gastroenterol Hepatol. 2019;17(11):2212–2217.e1. doi: 10.1016/j.cgh.2018.12.017. [DOI] [PubMed] [Google Scholar]
  • 4.Egal E.S.A., Jacenik D., Soares H.P., Beswick E.J. Translational challenges in pancreatic neuroendocrine tumor immunotherapy. Biochim Biophys Acta Rev Cancer. 2021;1876(2) doi: 10.1016/j.bbcan.2021.188640. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Bettini R., Boninsegna L., Mantovani W., et al. Prognostic factors at diagnosis and value of WHO classification in a mono-institutional series of 180 non-functioning pancreatic endocrine tumours. Ann Oncol. 2008;19(5):903–908. doi: 10.1093/annonc/mdm552. [DOI] [PubMed] [Google Scholar]
  • 6.Krampitz G.W., Norton J.A., Poultsides G.A., Visser B.C., Sun L., Jensen R.T. Lymph nodes and survival in pancreatic neuroendocrine tumors. Arch Surg. 2012;147(9):820–827. doi: 10.1001/archsurg.2012.1261. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Tomassetti P., Campana D., Piscitelli L., et al. Endocrine pancreatic tumors: factors correlated with survival. Ann Oncol. 2005;16(11):1806–1810. doi: 10.1093/annonc/mdi358. [DOI] [PubMed] [Google Scholar]
  • 8.Shah M.H., Goldner W.S., Benson A.B., et al. Neuroendocrine and adrenal tumors, version 2.2021, NCCN clinical practice guidelines in oncology. J Natl Compr Canc Netw. 2021;19(7):839–868. doi: 10.6004/jnccn.2021.0032. [DOI] [PubMed] [Google Scholar]
  • 9.Falconi M., Eriksson B., Kaltsas G., et al. ENETS consensus guidelines update for the management of patients with functional pancreatic neuroendocrine tumors and non-functional pancreatic neuroendocrine tumors. Neuroendocrinology. 2016;103(2):153–171. doi: 10.1159/000443171. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Jutric Z., Grendar J., Hoen H.M., et al. Regional metastatic behavior of nonfunctional pancreatic neuroendocrine tumors: impact of lymph node positivity on survival. Pancreas. 2017;46(7):898–903. doi: 10.1097/MPA.0000000000000861. [DOI] [PubMed] [Google Scholar]
  • 11.Kuo E.J., Salem R.R. Population-level analysis of pancreatic neuroendocrine tumors 2 cm or less in size. Ann Surg Oncol. 2013;20(9):2815–2821. doi: 10.1245/s10434-013-3005-7. [DOI] [PubMed] [Google Scholar]
  • 12.Ricci C., Casadei R., Taffurelli G., et al. Sporadic small (≤20 mm) nonfunctioning pancreatic neuroendocrine neoplasm: is the risk of malignancy negligible when adopting a more conservative strategy? A systematic review and meta-analysis. Ann Surg Oncol. 2017;24(9):2603–2610. doi: 10.1245/s10434-017-5946-8. [DOI] [PubMed] [Google Scholar]
  • 13.Lim T.Y., Leitman I.M. Risk factors for early morbidity and mortality following pancreatoduodenectomy with concomitant vascular reconstruction. Ann Med Surg (Lond) 2021;68 doi: 10.1016/j.amsu.2021.102587. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Bar-Moshe Y., Mazeh H., Grozinsky-Glasberg S. Non-functioning pancreatic neuroendocrine tumors: surgery or observation? World J Gastrointest Endosc. 2017;9(4):153. doi: 10.4253/wjge.v9.i4.153. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Barenboim A., Lahat G., Nachmany I., et al. Resection versus observation of small asymptomatic nonfunctioning pancreatic neuroendocrine tumors. J Gastrointest Surg. 2020;24:1366–1374. doi: 10.1007/s11605-019-04285-y. [DOI] [PubMed] [Google Scholar]
  • 16.Gillies R.J., Kinahan P.E., Hricak H. Radiomics: images are more than pictures, they are data. Radiology. 2016;278(2):563–577. doi: 10.1148/radiol.2015151169. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Zwanenburg A., Vallières M., Abdalah M.A., et al. The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295(2):328–338. doi: 10.1148/radiol.2020191145. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 18.Gu Q., He M., He Y., et al. CT-measured body composition radiomics predict lymph node metastasis in localized pancreatic ductal adenocarcinoma. Discover Oncology. 2023;14(1):16. doi: 10.1007/s12672-023-00624-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Zhang Z., Wan X., Lei X., et al. Intra- and peri-tumoral MRI radiomics features for preoperative lymph node metastasis prediction in early-stage cervical cancer. Insights Imaging. 2023;14(1):65. doi: 10.1186/s13244-023-01405-w. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 20.Cui Y., Zhang J., Li Z., et al. A CT-based deep learning radiomics nomogram for predicting the response to neoadjuvant chemotherapy in patients with locally advanced gastric cancer: a multicenter cohort study. eClinicalMedicine. 2022;46 doi: 10.1016/j.eclinm.2022.101348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Gu J., Tong T., Xu D., et al. Deep learning radiomics of ultrasonography for comprehensively predicting tumor and axillary lymph node status after neoadjuvant chemotherapy in breast cancer patients: a multicenter study. Cancer. 2023;129(3):356–366. doi: 10.1002/cncr.34540. [DOI] [PubMed] [Google Scholar]
  • 22.Li J., Dong D., Fang M., et al. Dual-energy CT-based deep learning radiomics can improve lymph node metastasis risk prediction for gastric cancer. Eur Radiol. 2020;30(4):2324–2333. doi: 10.1007/s00330-019-06621-x. [DOI] [PubMed] [Google Scholar]
  • 23.Dong D., Fang M.J., Tang L., et al. Deep learning radiomic nomogram can predict the number of lymph node metastasis in locally advanced gastric cancer: an international multicenter study. Ann Oncol. 2020;31(7):912–920. doi: 10.1016/j.annonc.2020.04.003. [DOI] [PubMed] [Google Scholar]
  • 24.Yushkevich P.A., Piven J., Hazlett H.C., et al. User-guided 3D active contour segmentation of anatomical structures: significantly improved efficiency and reliability. Neuroimage. 2006;31(3):1116–1128. doi: 10.1016/j.neuroimage.2006.01.015. [DOI] [PubMed] [Google Scholar]
  • 25.van Griethuysen JJM, Fedorov A., Parmar C., et al. Computational radiomics system to decode the radiographic phenotype. Cancer Res. 2017;77(21):e104–e107. doi: 10.1158/0008-5472.CAN-17-0339. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Gratian L., Pura J., Dinan M., Roman S., Reed S., Sosa J.A. Impact of extent of surgery on survival in patients with small nonfunctional pancreatic neuroendocrine tumors in the United States. Ann Surg Oncol. 2014;21(11):3515–3521. doi: 10.1245/s10434-014-3769-4. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Lee L.C., Grant C.S., Salomao D.R., et al. Small, nonfunctioning, asymptomatic pancreatic neuroendocrine tumors (PNETs): role for nonoperative management. Surgery. 2012;152(6):965–974. doi: 10.1016/j.surg.2012.08.038. [DOI] [PubMed] [Google Scholar]
  • 28.Tsutsumi K., Ohtsuka T., Mori Y., et al. Analysis of lymph node metastasis in pancreatic neuroendocrine tumors (PNETs) based on the tumor size and hormonal production. J Gastroenterol. 2012;47(6):678–685. doi: 10.1007/s00535-012-0540-0. [DOI] [PubMed] [Google Scholar]
  • 29.Harimoto N., Hoshino K., Muranushi R., et al. Significance of lymph node metastasis in resectable well-differentiated pancreatic neuroendocrine tumor. Pancreas. 2019;48(7):943–947. doi: 10.1097/MPA.0000000000001355. [DOI] [PubMed] [Google Scholar]
  • 30.Hashim Y.M., Trinkaus K.M., Linehan D.C., et al. Regional lymphadenectomy is indicated in the surgical treatment of pancreatic neuroendocrine tumors (PNETs) Ann Surg. 2014;259(2):197–203. doi: 10.1097/SLA.0000000000000348. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 31.Kwon W., Jang J.Y., Song K.B., et al. Risk factors for recurrence in pancreatic neuroendocrine tumor and size as a surrogate in determining the treatment strategy: a Korean nationwide study. Neuroendocrinology. 2021;111(8):794–804. doi: 10.1159/000511875. [DOI] [PubMed] [Google Scholar]
  • 32.Makris E.A., Cannon J.G.D., Norton J.A., et al. Calcifications and cystic morphology on preoperative imaging predict survival after resection of pancreatic neuroendocrine tumors. Ann Surg Oncol. 2023;30(4):2424–2430. doi: 10.1245/s10434-022-12783-8. [DOI] [PubMed] [Google Scholar]
  • 33.Tan Q., Wang X., Li Y., Liu Y., Liu X., Ke N. Prognostic factors of small non-functional pancreatic neuroendocrine tumors and the risk of lymph node metastasis: a population-level study. Front Endocrinol. 2022;13 doi: 10.3389/fendo.2022.907415. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Tatsumoto S., Kodama Y., Sakurai Y., Shinohara T., Katanuma A., Maguchi H. Pancreatic neuroendocrine neoplasm: correlation between computed tomography enhancement patterns and prognostic factors of surgical and endoscopic ultrasound-guided fine-needle aspiration biopsy specimens. Abdom Imaging. 2013;38(2):358–366. doi: 10.1007/s00261-012-9953-8. [DOI] [PubMed] [Google Scholar]
  • 35.Zhu H.-B., Nie P., Jiang L., et al. Preoperative prediction of lymph node metastasis in nonfunctioning pancreatic neuroendocrine tumors from clinical and MRI features: a multicenter study. Insights Imaging. 2022;13(1):162. doi: 10.1186/s13244-022-01301-9. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 36.Chen H., Li Z., Hu Y., et al. Maximum value on arterial phase computed tomography predicts prognosis and treatment efficacy of sunitinib for pancreatic neuroendocrine tumours. Ann Surg Oncol. 2022;30(5):2988–2998. doi: 10.1245/s10434-022-12693-9. [DOI] [PubMed] [Google Scholar]
  • 37.Franko J., Feng W., Yip L., Genovese E., Moser A.J. Non-functional neuroendocrine carcinoma of the pancreas: incidence, tumor biology, and outcomes in 2,158 patients. J Gastrointest Surg. 2010;14(3):541–548. doi: 10.1007/s11605-009-1115-0. [DOI] [PubMed] [Google Scholar]
  • 38.Ren S.J., Tan Q.Q., Cao D., Ke N.W., Liu X.B., Wang X. Prognostic role and predictors of lymph node involvement in pancreatic neuroendocrine tumors. Eur J Radiol. 2023;162 doi: 10.1016/j.ejrad.2023.110772. [DOI] [PubMed] [Google Scholar]
  • 39.Partelli S., Gaujoux S., Boninsegna L., et al. Pattern and clinical predictors of lymph node involvement in nonfunctioning pancreatic neuroendocrine tumors (NF-PanNETs) JAMA Surgery. 2013;148(10):932–939. doi: 10.1001/jamasurg.2013.3376. [DOI] [PubMed] [Google Scholar]
  • 40.Jiang Y., Jin J.B., Zhan Q., Deng X.X., Shen B.Y. Impact and clinical predictors of lymph node metastases in nonfunctional pancreatic neuroendocrine tumors. Chin Med J (Engl) 2015;128(24):3335–3344. doi: 10.4103/0366-6999.171427. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 41.Toste P.A., Kadera B.E., Tatishchev S.F., et al. Nonfunctional pancreatic neuroendocrine tumors <2 cm on preoperative imaging are associated with a low incidence of nodal metastasis and an excellent overall survival. J Gastrointest Surg. 2013;17(12):2105–2113. doi: 10.1007/s11605-013-2360-9. [DOI] [PubMed] [Google Scholar]
  • 42.Parekh J.R., Wang S.C., Bergsland E.K., et al. Lymph node sampling rates and predictors of nodal metastasis in pancreatic neuroendocrine tumor resections: the UCSF experience with 149 patients. Pancreas. 2012;41(6):840–844. doi: 10.1097/MPA.0b013e31823cdaa0. [DOI] [PubMed] [Google Scholar]
  • 43.Kim D.W., Kim H.J., Kim K.W., et al. Prognostic value of CT findings to predict survival outcomes in patients with pancreatic neuroendocrine neoplasms: a single institutional study of 161 patients. Eur Radiol. 2016;26(5):1320–1329. doi: 10.1007/s00330-015-3943-5. [DOI] [PubMed] [Google Scholar]
  • 44.Partelli S., Muffatti F., Andreasi V., et al. A single-center prospective observational study investigating the accuracy of preoperative diagnostic procedures in the assessment of lymph node metastases in nonfunctioning pancreatic neuroendocrine tumors. Ann Surg. 2022;276(5):921–928. doi: 10.1097/SLA.0000000000005615. [DOI] [PubMed] [Google Scholar]
  • 45.Howe J.R., Merchant N.B., Conrad C., et al. The North American neuroendocrine tumor society consensus paper on the surgical management of pancreatic neuroendocrine tumors. Pancreas. 2020;49(1):1–33. doi: 10.1097/MPA.0000000000001454. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 46.Haynes A.B., Deshpande V., Ingkakul T., et al. Implications of incidentally discovered, nonfunctioning pancreatic endocrine tumors: short-term and long-term patient outcomes. Arch Surg. 2011;146(5):534–538. doi: 10.1001/archsurg.2011.102. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 47.Partelli S., Bertani E., Bartolomei M., et al. Peptide receptor radionuclide therapy as neoadjuvant therapy for resectable or potentially resectable pancreatic neuroendocrine neoplasms. Surgery. 2018;163(4):761–767. doi: 10.1016/j.surg.2017.11.007. [DOI] [PubMed] [Google Scholar]
  • 48.Squires M.H., Worth P.J., Konda B., et al. Neoadjuvant capecitabine/temozolomide for locally advanced or metastatic pancreatic neuroendocrine tumors. Pancreas. 2020;49(3):355–360. doi: 10.1097/MPA.0000000000001500. [DOI] [PubMed] [Google Scholar]
  • 49.Parghane R.V., Ostwal V., Ramaswamy A., et al. Long-term outcome of “Sandwich” chemo-PRRT: a novel treatment strategy for metastatic neuroendocrine tumors with both FDG- and SSTR-avid aggressive disease. Eur J Nucl Med Mol Imaging. 2021;48(3):913–923. doi: 10.1007/s00259-020-05004-5. [DOI] [PubMed] [Google Scholar]

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Material, Figs. S1–S5 and Tables S1–S4
mmc1.docx (588.4KB, docx)

Articles from eClinicalMedicine are provided here courtesy of Elsevier

RESOURCES