Nomogram Predicting the Risk of Recurrence after Curative-intent Resection of Primary Non-Metastatic Gastrointestinal Neuroendocrine Tumors: An Analysis of the U.S. Neuroendocrine Tumor Study Group

Abstract

Background

The risk of recurrence for patients undergoing curative-intent surgery for non-metastatic gastro-entero-pancreatic neuroendocrine tumors (GEP-NET) is currently poorly defined. We sought to develop and validate a nomogram to predict the risk of recurrence after curative-intent resection.

Methods

Using a pseudo-randomization technique, a training set to develop a predictive nomogram and a test set to validate the nomogram were identified. The predictive ability of the nomogram to predict recurrence was assessed using the c-index.

Results

Among 1,477 patients, 673 (46%) patients were included in the training set and 804 (54%) patients were included in the test set to validate the nomogram. On multivariable analysis, Ki-67, tumor size, nodal status, and invasion of adjacent organs were independent predictors of DFS. The risk of death increased by 8% for each percentage increase in the Ki-67 index (HR 1.08, 95% CI, 1.05–1.10; p<0.001). Patients with GEP-NET invading adjacent organs had a HR of 1.65 (95% CI, 1.03–2.65; p=0.038), which was similar to the increase in the hazard of death associated with tumor ≥3 cm in size (HR 1.67, 95% CI, 1.11–2.51; p=0.014). Furthermore, compared with patients who did not have nodal metastasis, patients who had 1–3 positive nodes had 1.8-fold increased risk of death (HR 1.81, 95% CI, 1.12–2.87; p=0.014), while patients with >3 lymph node metastasis had a 2.5-fold increased risk of death (HR 2.51, 95% CI, 1.50–4.24; p<0.001). The beta-coefficients from the final multivariable model were utilized to develop a nomogram. The nomogram demonstrated a good ability to predict risk of recurrence (training set, c-index 0.739; test set, c-index of 0.718). Patients were stratified by predicted recurrence risk (PRisk) groupings: PRisk <20%, PRisk 20–40%, PRisk 40–80%, PRisk >80%, which correlated with disease free-survival at 10-year of 87%, 68%, 37%, and 0%, respectively.

Conclusion

A nomogram based on four variables (i.e. Ki-67, tumor size, invasion of adjacent organs and lymph node status) was able to predict the risk of recurrence after surgery for GEP-NET. The nomogram demonstrated a good ability to identify patients at risk of recurrence and can be easily applied in the clinical setting.

Introduction

Gastro-entero-pancreatic neuroendocrine tumors (GEP-NET) are a heterogeneous group of tumors that originate from the diffuse neuroendocrine cell system of the gastrointestinal tract and pancreas [1]. Even though GEP-NET have classically been considered rare neoplasia, recent data have suggested an exponential increase in the incidence of GEP-NET over the last several decades [2–5]. Specifically, the annual age-adjusted incidence of GEP-NET was 1.09 per 100,000 persons in 1973 compared with 6.98 per 100,000 persons in 2012 – a six-fold increase [4]. The prevalence of GEP-NET is now higher than all other gastrointestinal cancers except for colorectal neoplasia [3]. GEP-NET tumors can present with a wide range of histologic features and clinical presentations, as well as variable biological behavior. Depending on the primary site and grade, GEP-NET can have an indolent or aggressive course that can be resistant to many types of treatment [6]. For patients with loco-regional disease, curative-intent surgical management of the primary tumor is first-line treatment. In contrast, for patients with more advanced metastatic disease, the goals of resection may include both symptom control, debulking to limit tumor progression, as well as attempts at complete surgical extirpation [7].

While data on overall survival has more commonly been reported, the risk of recurrence following curative resection of primary NET has been less defined. Specifically, data on recurrence after resection, including incidence of recurrence, recurrence pattern and prognostic factors associated with recurrence, remain relatively scarce [4,8–11]. Data on recurrence is, however, important to inform patients about the likelihood of treatment success and the risk of recurrence following surgical intervention. In particular, accurate individual patient-based estimates of risk of recurrence may not only assist with patient counseling and inform decision-making, but also help guide follow-up. While nomograms have been reported as useful tools to discriminate the prognosis of patients for a number of different cancers, nomograms have not been widely applied to patients with GEP-NET [12]. Therefore, the objective of the current study was to identify factors associated with recurrence among patients who underwent curative intent resection of GEP-NET using a large multi-institutional database. Using these factors, we sought to develop and validate a nomogram to predict individual patient-specific risk of recurrence after curative-intent surgical resection of non-metastatic primary GEP-NET.

Patients and Methods

Study population and data collection

In this retrospective cohort study, a multi-institutional database was queried to identify patients who underwent surgery for histologically confirmed GEP-NET between 2000 and 2014 at one of the eight centers participating of the US Neuroendocrine Tumor Study Group (US-NETSG). Patients who underwent surgery for histologically confirmed GEP-NET between 2000 and 2014 at one of the eight centers participating of the US Neuroendocrine Tumor Study Group (US-NETSG) were identified. US-NETSG included The Ohio State University Wexner Medical Center and James Comprehensive Cancer Center, Columbus, OH; Winship Cancer Institute, Emory University, Atlanta, GA; Stanford University, Palo Alto, CA; Virginia Mason Medical Center, Seattle, WA; University of Wisconsin, School of Medicine and Public Health, Madison, WI; Washington University, School of Medicine, St. Louis, MO; Vanderbilt University, Nashville, TN; University of Michigan, Ann Arbor, MI. The Institutional Review Board of the participating institutions approved the study. Only patients who underwent curative-intent surgery for primary GEP-NET without metastasis (M0) were included. Patients who underwent only non-surgical treatments (percutaneous ablation or intra-arterial therapy), debulking, and palliation were excluded. Standard patient demographic and clinico-pathological characteristics were collected, including age, gender, American Society of Anesthesiologist (ASA) class of risk, number of GEP-NET lesions, tumor size, invasion of adjacent organs, margin status, tumor grade, Ki-67 index, and lymph node status.

Statistical analysis

Discrete variables were described as medians with interquartile range (IQR) and categorical variables were recorded as totals and frequencies. Imputation for missing data was performed using fully conditional specification (FCS) implemented by the multivariate imputation by chained equations (MICE) algorithm [13]. Univariable comparisons were assessed using the chi-squared test or fisher’s exact test as appropriate. The outcome for survival analyses was disease-free survival (DFS), defined as the time interval between the date of surgery and the date of recurrence. Time was censored at the date of the last follow-up assessment for patients without recurrence. Survival curves were estimated using the Kaplan-Meier method and differences between the curves were compared using the log-rank test. Variables that had a p-value <0.1 on univariate analyses were included in the final multivariable model.

A pseudo-randomization based on the day of birth was used to identify a training set (patients born within the first 16 days of the month of birth) to develop the model and a test set (patients born within the last 15 days of the month of birth) to validate the model [14]. Multivariable Cox proportional hazards models were used to evaluate associations between variables and DFS. A sensitivity analysis was performed on the un-imputed data to confirm the results of the multivariable analysis. Continuous variables were included in the nomogram using their original format or as categorical variables based on clinical considerations and comparison of the performance of the models with the continuous and categorical forms of the variables. The final multivariable model was used to develop the nomogram. Coefficients from the Cox models were reported as hazard ratios (HR) with corresponding 95% confidence intervals (CI). A P-value of <0.05 (two-tailed) was considered statistically significant. All analyses were performed using STATA version 12.0 (StataCorp LP, College Station, TX, USA) and R software for statistical computing, v. 3.0.2 34, with the additional packages: survival, mice, and Hmisc [13].

Results

Baseline characteristics

Table 1 lists the baseline characteristics of the 1,477 patients in the cohort. There were 723 (48.9%) men and 754 (51.1%) women with the majority of patients being under the age of 65 years (n=1,039, 70.4%). Most patients were ASA class 2 (n=568, 41.3%) or 3 (n=714, 52%), while a smaller subset were class 1 (n=65, 4.7%) or 4 (n=27, 2%). While GEP-NET was not associated with a genetic syndrome in the overwhelming majority of patients (n=1,358, 91.9%), 72 (5%) and 11(0.8%) patients were diagnosed with Multiple Endocrine Neoplasia type 1 (MEN 1) and Von Hippel-Lindau (VHL) syndromes, respectively. Overall, 880 (60.4%) patients had a functional GEP-NET. The primary tumor site was pancreas in over one-half of cases (n=948, 64.2%), followed by jejunum-ileum (n=198, 13.4%) or duodenum (n=101, 6.8%). Other sites such as the stomach, appendix, rectum, colon, ampulla, liver and gallbladder were less common (n=230, 15.6%).

Table 1.

Baseline characteristics N= 1,477

Variables	N (%)	Training Set	Test Set	p-value
All patients	1,477 (100%)	754 (51.1%)	723 (48.9%)	-
Age				0.98
< 65 years	1,039 (70.4%)	528 (70.4%)	508 (70.5%)
≥ 65 years	438 (29.6%)	226 (29.6%)	215 (29.5)
Gender				0.54
Male	723 (48.9%)	375 (49.7%)	348 (48.1%)
Female	754 (51.1%)	379 (50.3%)	375 (51.9%)
ASA Class				0.89
1	65 (4.7%)	35 (4.9%)	30 (4.6%)
2	568 (41.3%)	296 (41.2%)	272 (41.5%)
3	714 (52.0%)	372 (51.7%)	342 (52.2%)
4	27 (2.0%)	16 (2.2%)	11 (1.7%)
NA	103	35	68
Functional Status				0.33
Non-functional	576 (39.6%)	305 (40.8%)	271 (38.3%)
Functional	880 (60.4%)	443 (59.2%)	437 (61.7%)
NA*	21	6	15
Genetic Syndrome				0.91
Not Syndromic	1,358 (91.9%)	695 (93.6%)	663 (94.1%)
MEN 1	72 (5.0%)	38 (5.1%)	34 (4.8%)
VHL	11 (0.7%)	7 (0.9%)	4 (0.6%)
Neurofibromatosis	5 (0.3%)	2 (0.3%)	3 (0.4%)
Tuberous Sclerosis	2 (0.1%)	1 (0.1%)	1 (0.1%)
NA*	29	11	18
Multifocal				0.95
No	1,222 (93.6%)	615 (93.6%)	607 (93.5%)
Yes	84 (6.4%)	42 (6.4%)	42 (6.5%)
NA*	171	97	74
Primary				0.75
Ampulla	28 (1.9%)	16 (2.1%)	12 (1.7%)
Appendix	48 (3.3%)	19 (2.5%)	29 (4.0%)
Colon	34 (2.3%)	14 (1.9%)	20 (2.8%)
Duodenum	101 (6.8%)	53 (7.0%)	48 (6.6%)
Gallbladder	4 (0.3%)	1 (0.1%)	3 (0.4%)
Liver	12 (0.8%)	8 (1.1%)	4 (0.6%)
Pancreas	948 (64.2%)	489 (64.9%)	459 (63.5%)
Rectum	37 (2.5%)	23 (3.1%)	14 (1.9%)
Small Bowel	198 (13.4%)	101 (13.4%)	97 (13.4%)
Stomach	67 (4.5%)	29 (3.9%)	38 (5.1%)
Tumor Differentiation				0.79
Well differentiated	1,101 (88.3%)	562 (88.2%)	539 (88.4%)
Moderately differentiated	118 (9.5%)	59 (9.3%)	59 (9.7%)
Poorly differentiated	28 (2.2%)	16 (2.5%)	12 (1.9%)
NA*	230	117	113
Ki-67 Index, median (IQR)	2.0% (1–5)	2.6% (1–5)	2.0% (1–5)	0.87
Lymphnodes Status				0.79
Negative	704 (60.7%)	368 (61.7%)	336 (59.6%)
1–3 Positives	317 (27.3%)	156 (26.2%)	161 (28.5%)
>3 Positives	139 (12.0%)	72 (12.0%)	67 (11.9%)
NA*	317	158	159
Tumor Size				0.55
< 3 cm	1,031 (69.8%)	521 (69.1%)	510 (70.5%)
≥ 3 cm	446 (30.2%)	233 (30.9%)	213 (29.5%)
Final Margin Status				0.23
Negative	1,246 (84.9%)	644 (85.9%)	602 (83.7%)
Positive	222 (15.1%)	105 (14.1%)	117 (16.3%)
NA*	9	5	4
Perivascular Invasion				0.09
Absent	683 (63.8%)	369 (66.3%)	314 (61.2%)
Present	387 (36.2%)	188 (33.7%)	199 (38.8%)
NA*	407	197	210
Perineural Invasion				0.90
Absent	706 (74.7%)	366 (74.5%)	340 (74.9%)
Present	239 (25.3%)	125 (25.5%)	114 (25.1%)
NA*	532	263	269

NA=not available;

^*multiple imputation was used to address missing data

On final pathology, primary tumor size was < 3 cm in 1,031 (69.8%) patients. The vast majority of tumors were well differentiated (n=1,101; 88.3%); 118 (9.5%) tumors were moderately differentiated tumor, while 2.2% (n=28) were poorly differentiated. Median Ki-67 index was 2% (IQR 1–5). Most patients had no lymph node metastasis (n=704, 60.7%); 317 (27.3%) patients had 1–3 lymph node metastasis and 139 (12.0%) had >3 nodal metastasis. Perivascular and perineural invasion were present in 36.2% (n=387) and 25.3% (n=25.3) of tumors, respectively. Margin status was microscopically negative in 1,246 (84.9%) patients.

Factors associated with disease-free survival

The analytic cohort was divided into a training set (n= 754, 51.1%) used to identify factors associated with DFS to develop the nomogram; subsequently, 723 (48.9%) patients were included in the test set to validate the nomogram. The baseline characteristics of patients included in the training and test sets are reported in Table 1. Of note, there were no differences in the clinico-pathological variables in the training versus test cohorts.

In assessing the 754 patients in the training cohort, several factors were associated with DFS on univariable analysis (Table 2). Specifically, patients with a <3% Ki-67 index had a 5-year DFS of 82.6% (95% CI, 71.05–89.7), while patients with tumors that had a 3–20% and >20% Ki-67 index had 5-year DFS of 69.7% (95% CI, 57.8–78.9) and 26.6% (95% CI, 7.3–51.1), respectively (p<0.001). In addition, 5-year DFS was 38.6% (95% CI, 13.4–63.6) among patients with poorly differentiated GEP-NET, versus 65.2% (95% CI, 44.2–79.9) and 83.4% (95% CI, 78.6–87.2) for patients with moderately and well-differentiated tumors, respectively (p<0.001). Tumor size was also associated with long-term outcomes. Specifically, patients with GEP-NET <3 cm had a 5-year DFS of 87.3% (95% CI, 82.5–90.8) versus 66.3% (95% CI, 57.8–73.5) for tumors ≥ 3 cm (p<0.001). In addition, 5-year DFS for patients with no lymph node metastasis was 84.6% (95% CI, 79.2–87.1) compared with 71.8% (95% CI, 64.7–78.2) for patients with 1–3 metastatic lymph nodes and 56.9% (95% CI, 46.1–69.5) for patients with >3 metastatic lymph nodes (p<0.001). On multivariable analysis, Ki-67, tumor size, nodal status, and invasion of adjacent organs remained independent predictors associated with DFS. Of note, the risk of death increased by 8% for each percentage increase in the Ki-67 index (HR 1.08, 95% CI, 1.05–1.10; p<0.001). Patients with GEP-NET invading adjacent organs had a HR of 1.65 (95% CI, 1.03–2.65; p=0.038), which was similar to the increased hazard of death associated with tumor ≥3 cm in size (HR 1.67, 95% CI, 1.11–2.51; p=0.014). Furthermore, compared with patients who did not have nodal metastasis, patients who had 1–3 positive nodes had 1.8-fold increased risk of death (HR 1.81, 95% CI, 1.12–2.87; p=0.014), while patients with >3 lymph node metastasis had a 2.5-fold increased risk of death (HR 2.51, 95% CI, 1.50–4.24; p<0.001). A sensitivity analysis was performed on the un-imputed data, which confirmed the results of the initial multivariable analysis. Subsequently, the beta-coefficients from the final multivariable model were utilized to develop a nomogram (Figure 1).

Table 2.

Univariate analysis, 5-year Disease-Free Survival

Variables	N (%)	5-year DFS	95% CI	p-value
All patients	754 (51.1%)			-
Age				0.88
< 65 years	528 (70.4%)	79.6%	74.4–83.9
≥ 65 years	222 (29.6%)	80.8%	71.9–87.1
Gender				0.64
Male	375 (49.7%)	79.9%	73.8–84.8
Female	379 (50.3%)	78.9%	73.4–84.9
ASA Class				0.98
1–2	331 (46.0%)	80.2%	73.7–85.3
3–4	388 (54.0%)	79.2%	72.5–84.4
Functional Status				0.007
Non-functional	305 (40.8%)	87.6%	81.9–91.6
Functional	443 (59.2%)	73.7%	67.2–79.2
Genetic Syndrome				0.24
Not Associated	695 (93.0%)	61.1%	35.9–78.9
Associated	52 (7.0%)	67.2%	58.6–74.4
Multifocal				0.68
No	615 (93.6%)	79.9%	75.3–83.8
Yes	42 (6.4%)	76.6%	45.8–91.3
Primary				0.93
Gastriontestinal	265 (35.2%)	71.9%	61.5–80.0
Pancreas	489 (64.8%)	67.7%	44.8–74.8
Tumor Differentiation				<0.001
Well differentiated	562 (88.2%)	83.4%	78.6–87.2
Moderately differentiated	59 (9.3%)	65.2%	44.2–79.9
Poorly differentiated	16 (2.5%)	38.6%	13.4–63.6
Ki-67 Index Label				<0.001
<3%	282 (60.3%)	82.6%	71.5–89.7
3–20%	162 (34.6%)	69.7%	57.8–78.9
>20%	24 (5.1%)	26.6%	7.3–51.1
Lymph node Status				<0.001
Negative	368 (48.8%)	84.6%	79.2–87.1
1–3 Positives	156 (20.7%)	71.8%	64.7–78.2
>3 Positives	72 (9.5%)	56.9%	46.1–69.5
Tumor Size				<0.001
< 3 cm	521 (69.1%)	87.3%	82.5–90–8
≥ 3 cm	233 (30.9%)	66.3%	57.8–73.5
Final Margin Status				0.08
Negative	644 (85.9%)	80.8%	76.2–84.7
Positive	105 (14.0%)	73.9%	59.4–83.9
Lymph-vascular Invasion				<0.001
Absent	369 (66.3%)	89.4%	84.0–93.1
Present	188 (33.8%)	61.5%	50.9–70.6
Perineural Invasion				<0.001
Absent	366 (74.5%)	85.9%	80.1–90.2
Present	125 (25.5%)	62.1%	48.4–73.1

Figure 1.

Nomogram Predicting the Risk of Recurrence at 3-, 5-, and 10-year after Surgery

Prediction Ability of the Nomogram and External Validation

The nomogram demonstrated a good ability to predict risk of recurrence in both the training set (c-index 0.739; standard error [se], 0.02) and the test set (c-index of 0.718; se, 0.03). When the performance of the nomogram was tested in the subgroups of patients characterized by different embryological origins of primary GEP-NET (foregut, midgut, and hindgut), the ability of the nomogram to predict the risk of recurrence was the same in all three groups (p>0.1). Particularly, the c-index was 0.767 (se, 0.03), 0.741 (se, 0.06), and 0.70 (se, 0.12) in the foregut, midgut, and hindgut groups, respectively. Based on the risk of recurrence predicted by the nomogram (PRisk), patient overall risk of recurrence categorized into four distinct groups: group 1 (PRisk <20%), group 2 (PRisk 20–40%), group 3 (PRisk 40–80%), and group 4 (PRisk >80%). Among the 1,477 patients analyzed, more than two third of patients (n=1,052, 71.2%) were in group 1 (PRisk <20%), while 295 (20.0%), 112 (7.6%), and 18 (1.2%) patients were in groups 2 (PRisk 20–40%), 3 (PRisk 40–80%), and 4 (PRisk >80%), respectively. Five-year DFS was 88.7% (95% IC, 85.7–91.2), 68.4% (95% IC, 60.8–74.9), and 45.3% (95% IC, 32.9–56.9) in group 1 (PRisk <20%), group 2 (PRisk 20–40%), and group 3 (PRisk 40–80%), respectively. Of note, all patients in group 4 (PRisk >80%) had recurrence within 3 years following surgery (Figure 2).

Figure 2.

Kaplan Meier Curves for Disease-Free Survival Stratified on the base of the Risk of Recurrence predicted by our nomogram (PRisk): group 1 (PRisk <20%), group 2 (PRisk 20–40%), group 3 (PRisk 40–80%), and group 4 (PRisk >80%).

Among the 207 (14%) patients who relapsed, 98 (47.6%) patients had recurrence < 18 months from surgery, 42 (20.4%) recurred within 18–36 months, while 66 (32.0%) had recurrence >36 months from surgery. Patients in group 1 (PRisk <20%) and group 2 (PRisk 20–40%) had a higher incidence of recurrence >36 months from surgery (group 1, n=33, 41.3%; group 2, n=25, 34.7%), compared with recurrences within 18–36 months from surgery (group 1, n=15, 18.8%; group 2, n=10, 13.9%). Interestingly, patients in groups 3 (PRisk 40–80%), and 4 (PRisk >80%) demonstrated a higher likelihood or recurrence < 18 months from the time of surgery (Figure 3a). Particularly, the incidence of recurrence < 18 months from the time of surgery was 48.9% (n=22) for patients in nomogram group 1 and 77.8% for patients in group 4.

Figure 3.

Comparison of the Incidence of Recurrence among the four PRisk groups identified: a) recurrence <18 months from surgery, recurrence within 18–36 months from surgery, and recurrence >36 months from surgery; b) local and distal recurrence; c) local plus distal recurrence and distant recurrence.

Among the 207 (14%) patients who relapsed, 63 (31.0%) patients had local recurrence, while 140 (69.0%) patients had a distant recurrence. Of note, the nomogram groupings were also associated with the pattern of recurrence. Specifically, patients in group 1 (n=37, 46.8%) and group 2 (n=17, 23.6%) were more likely to recur locally compared with patients in group 3 (n=7, 16.3%) and group 4 (n=2, 22.2%)(p=0.0001)(Figure 3b). In contrast, the incidence of distant recurrence was only 11.9% (n=5) among patients in group 1 (PRisk <20%) versus 60.0% among patients in group 4 (PRisk >80%)(p=0.008; Figure 3c).

DISCUSSION

Current recommendations for peri-operative management (i.e. analogues of somatostatin, neo-adjuvant and adjuvant chemotherapy) and post-operative surveillance of patients who undergo curative-intent surgery of primary GEP-NET are largely based on expert opinion and small retrospective studies focused on specific subtypes of neuroendocrine tumors [7,15–19]. Moreover, the optimal prognostic classification (i.e. nomogram and staging system) to stratify risk of recurrence for GEP-NET patients has not been defined. The American Joint Committee on Cancer (AJCC) TNM classification is one of the most common staging systems used in the clinical setting, which is based on classic pathological variables including tumor characteristics (T), lymph node status (N), and presence of distant metastases (M)[20]. In 2010, the World Health Organization (WHO) described the mitotic count and Ki-67 index as important parameters to effectively grade GEP-NET and included these two parameters in the WHO staging system [21]. Subsequently, the European Neuroendocrine Tumor Society (ENETS) combined the TNM classification with a grading system based on both mitotic count and Ki-67 index [15,22,23]. Even though the ENETS staging system has demonstrated to be superior to the AJCC TNM classification, several other clinico-pathological characteristics have been proposed as additional variables impacting the prognosis of GEP-NET patients, including gender, age at diagnosis, site of primary tumor, grade of differentiation and site of metastatic disease [24–26]. There is also no consensus on the optimal cut-off value for Ki-67 index that should be used to stratify patients’ prognosis [27,28]. Furthermore, these staging systems have exclusively focused on stratifying patients with regards to overall survival, rather than risk of recurrence. The aim of our study was to develop and validate a nomogram to accurately predict individual risk of recurrence after curative resection of GEP-NET. Nomograms can be more accurate and clinically applicable tools to predict outcomes in the context of cancer than conventional staging systems [12]. Even though several previous nomograms have been proposed to predict prognosis, risk of lymph node metastasis, distant metastasis, and overall survival after resection of primary GEP-NET, to the best of our knowledge, no nomograms or staging system predicting the risk of recurrence, independently of the anatomical site of primary GEP-NET, have been proposed [11,29–35]. Particularly, the impact of the organ site of primary GEP-NET on patients prognosis has not been completely clarified.[36] In the current analysis, site of primary GEP-NET was not associated with the risk of recurrence and the predictive ability of the nomogram was no different among subgroups of patients categorized by different embryological origins of primary GEP-NET (foregut, midgut, and hindgut). The current nomogram is important, therefore, as we developed and validated it using a large multi-institutional international database, including only patients with primary, non-metastatic, GEP-NET.

Four variables (i.e. Ki-67 index as a continue variable, node status, tumor size, and invasion of adjacent organs) were strongly associated with risk of recurrence on multivariable analysis and were included in the proposed nomogram. Ki-67 index has been identified as an important biological variable for patients with GEP-NET [22,29,30,36–38]. Even though ENETs and WHO 2010 classification defined three prognostic groups based on Ki-67 index cut-off values of 3% and 20%, we chose to include Ki-67 as a continuous variable in the nomogram, since Ellison et al. had previously noted that Ki-67 index had a linear relationship with survival [29]. While the updated WHO 2017 classification for neuroendocrine tumors proposed new cut-off values, several authors have suggested a modification of the cut-off values for the Ki-67 index to better stratify patient prognosis [28,39–41]. Roughly 60% of patients in our cohort had a Ki-67 index <3%, 35% a Ki-67 index 3–20%, and 5% of patients a Ki-67 index >20%. The 5-year DFS was 82.6 % for patients in the <3% Ki-67 group, 69.7 % for patients with Ki-67 3–20%, and only 26.6% in the >20% Ki-67 group. In the proposed nomogram, Ki-67 index was, however, a stronger predictor of recurrence as a continuous variable versus a categorical variable. Interestingly, when Ki-67 index was considered a continue variable, the risk of recurrence increased by 8% for each additional percentage point of Ki-67.

Tumor size was also a strong predictor of recurrence and was included in the nomogram as a categorical variable. Specifically, patients with tumors <3 cm had a 5-year DFS of 87% versus 66% for patients with tumors ≥3 cm. In fact, tumor size ≥ 3 cm conferred over a 1.5-fold increased risk of recurrence. Previous studies have similarly demonstrated that tumor size can impact prognosis of patients with GEP-NET. For example, Mosquera et al. reported that the risk of metastasis increased by 39% for each 1 cm increase in tumor size [35]. In a separate study, Fang et al. reported that patients with tumors 2–4 cm and >4cm had an increased hazard of death compared with patients with tumors smaller than 2 cm [11]. In addition, data from other studies have correlated tumor size with the risk of lymph node metastasis [42–44]. To this point, Sohn et al. reported that tumor grade combined with tumor size is an important predictive factor for lymph node metastasis and could serve as a prognostic factor for survival outcomes [45]. Collectively, the data strongly suggest that tumor size impacts prognosis of patients with GEP-NET tumors.

Even though lymph node status has been reported as an important predictor of survival among patients with GEP-NET, there are less data on the impact of metastatic nodal disease on the risk of recurrence. Analyzing clinico-pathological factors associated with recurrence in 188 patients who underwent surgery for GEP-NET, Slatger et al. reported that lymph node involvement was an important independent risk factor for recurrence (HR 2.61; 95%CI, 1.17–5.83)[38]. In a separate study, Dieckhoff et al. reported that lymph node ratio (LNR) was a more precise method to predict outcome rather than simple lymph node status [46]. In this analysis, patients with a LNR >0.2 had a 5-year DFS of 46% versus 76% for patients with a LNR ≤0.2 [46]. Similarly, Martin et al. reported an increased risk of death of 1.5-, 2-, and 3-fold for patients with LNR ≤0.2 (HR, 1.5), LNR 0.2–0.5 (HR, 2.0), and LNR >0.5 (HR, 3.1), respectively, compared with patients without nodal metastasis [10]. In the current analysis, the number of metastatic nodes was also strongly associated with the risk of recurrence on both univariable and multivariable analyses. Particularly, patients with node negative disease had a 5-year DFS of 85% versus 72% for patients with 1–3 nodal metastases and a DFS of 57% for patients with >3 nodal metastases. In fact, on multivariable analysis, compared with individuals who were node negative, patients with 1–3 and >3 nodal metastases had a 1.8- and 2.5-fold increased risk of recurrence.

While prognostic factors associated with recurrence after resection of GEP-NET have been reported in the literature, data regarding patterns of recurrence are scarce [36,38,46–48]. Using the Surveillance, Epidemiology, and End Results (SEER) and Medicare databases, Shen et al. analyzed patterns of recurrence after surgical resection among 2,366 patients with non-metastatic NETs. While 16% of patients developed metastatic disease within 5 years, only 1% of patients developed metastases between 5 and 10 years after surgery. Moreover, only 10% of patients with local disease had recurrence compared with 31% of patients with GEP-NET that invaded adjacent organs or had lymph node metastasis [49]. In a separate study, Strosberg et al. reported that the incidence of recurrence peaked approximately 2 years after surgery [50]. In the current study, the proposed nomogram demonstrated good ability to predict 3-, 5-, and 10-year recurrence in both the training and test set (c-index: training set 0.739 test set 0.718). In addition, the nomogram groupings were associated with timing and patterns of recurrence. For example, patients in nomogram groups 1 and 2 were more likely to experience recurrence >36 months from surgery. In contrast, patients in nomogram groups 3 and 4 had a much higher high incidence of recurrence <18 months from surgery, with recurrence >36 months from surgery being less common (p<0.001). In addition to time of recurrence, the nomogram groupings were associated with the pattern of recurrence. Specifically, while the majority of patients in nomogram group 1 had a similar incidence of local (47%) versus distal recurrence (53%), patients in nomogram groups 3 and 4 were much more likely to experience a distant recurrence rather than a local recurrence. These data strongly suggest that a nomogram based on biological (Ki67, nodal status), as well as anatomical (tumor size, invasion of adjacent organs) factors can help identify which individual patients are at highest risk of recurrence.

Several limitations should be considered when interpreting the results. Given the retrospective design and the multicenter nature of the study, selection bias was possible. While a pseudo-randomization was used to create two cohorts of patients for the development and validation of the nomogram, the nomogram will require further external validation in a separate cohort of patients. In addition, the study cohort included only patients who underwent surgery at one of eight specific academic centers in the United States. Therefore, the results from the current study may not be generalizable to non-academic, community centers or lower volume centers. Finally, the Ki-67 index was included in the model as a continue variable while lymph node status and tumor size were included as categorical variables. Even though, this approach was based on clinical considerations and previously published data, this approach may be apotential source of bias. To examine this possibility, an analysis comparing the performances of the models including either continuous or categorical forms of the variables was performed to assess the risk of bias. These analyses demonstrated no difference in the ability to predict the risk of recurrence among the different models (all p>0.1).

CONCLUSION

A nomogram based on four variables (i.e. Ki-67, tumor size, invasion of adjacent organs and lymph node status) was able to predict the risk of recurrence after surgery for GEP-NET. The nomogram groupings were also associated with different timing and patterns of recurrence. The nomogram demonstrated a good ability to identify patients at risk of recurrence (c-index: training set 0.739, test set 0.718) and can be easily applied in the clinical setting. While this nomogram will need to be further validated, nomograms such as the one proposed herein may be helpful clinical tools for providers to inform patients about the risk of recurrence, as well as personalize adjuvant treatment and surveillance strategies following surgery for patients with primary GEP-NET.

Supplementary Material

Supp. Fig. 1. Figure S1.

Calibration plot comparing predicted and actuarial disease free survival at 5 years

Table 3.

Multivariable survival analysis

Variables	HR	95% CI	p-value
Tumor Size			0.014
< 3 cm	-	-
≥ 3 cm	1.67	1.11–2.51
Lymph node Status
Negative	-	-	-
1–3 positive nodes	1.81	1.12–2.87	0.014
>3 positive nodes	2.51	1.50–4.24	0.001
Ki-67	1.08	1.05–1.10	<0.001
Invasion of Adjacent Organs
Absent	-	-
Present	1.65	1.03–2.65	0.038