A Novel Validated Recurrence Risk Score to Guide a Pragmatic Surveillance Strategy After Resection of Pancreatic Neuroendocrine Tumors

Abstract

Objective:

Despite heterogeneous biology, similar surveillance schemas are utilized after resection of all pancreatic neuroendocrine tumors (PanNETs). Given concerns regarding excess radiation exposure and financial burden, our aim was to develop a prognostic score for disease recurrence to guide individually tailored surveillance strategies.

Methods:

All patients with primary nonfunctioning, nonmetastatic well/moderately differentiated PanNETs who underwent curative-intent resection at 9-institutions from 2000 to 2016 were included (n = 1006). A Recurrence Risk Score (RRS) was developed from a randomly selected derivation cohort comprised of 67% of patients and verified on the validation-cohort comprised of the remaining 33%.

Results:

On multivariable analysis, patients within the derivation cohort (n = 681) with symptomatic tumors (jaundice, pain, bleeding), tumors >2cm, Ki67 >3%, and lymph node (LN) (+) disease had increased recurrence. Each factor was assigned a score based on their weighted odds ratio that formed a RRS of 0 to 10: symptomatic = 1, tumor >2cm = 2, Ki67 3% to 20% = 1, Ki67 >20% = 6, LN (+) = 1. Patients were grouped into low- (RRS = 0–2; n = 247), intermediate-(RRS = 3–5; n = 204), or high (RRS = 6–10; n = 9)-risk groups. At 24 months, 33% of high RRS recurred, whereas only 2% of low and 14% of intermediate RRS recurred. This persisted in the validation cohort (n = 325).

Conclusions:

This international, novel, internally validated RRS accurately stratifies recurrence-free survival for patients with resected PanNETs. Given their unique recurrence patterns, surveillance intervals of 12, 6, and 3 months are proposed for low, intermediate, and high RRS patients, respectively, to minimize radiation exposure and optimize cost/resource utilization.

Pancreatic neuroendocrine tumors (PanNETs) compromise approximately 3% of all pancreatic malignancies, although the ubiquity of high-resolution cross-sectional imaging has contributed to increased incidence in recent years.^1–4 The majority of PanNETs lack the characteristic hormone-related syndromes seen in their “functional” counterparts (ie, insulinomas, gastrinomas, glucagonomas) and are thus described as “non-functional” (NF-Pan-NETs).^5–8 Although NF-PanNETs do not have a secretory hormonal phenotype, there is complex biological and molecular heterogeneity among these tumors, with a recent effort among researchers and clinicians to define distinct clinical and pathologic signatures which may predict worse outcomes.^9–11 Macroscopic features such as tumor size and lymph node (LN) metastases as well as microscopic features such as lymphovascular invasion, the presence of necrosis, and the Ki-67 proliferative index have all been shown to predict more aggressive tumor biology in PanNETs.^12–16 In fact, the Ki-67 index is considered the most important molecular marker for PanNETs, and has been incorporated as a factor in the World Health Organization (WHO) prognostic staging system.^17–21

As the therapeutic armamentarium grows with a deeper understanding of the molecular diversity of PanNETs, surgical resection still remains the primary treatment modality to achieve cure.^22–25 Even among well-selected patients with advanced disease, surgery is associated with a significantly improved survival compared to those patients who do not undergo resection.²⁶ Although some patients with small (<2cm) PanNETs may be treated with surveillance instead of resection, surgical options in general range from enucleation to formal oncologic resection with lymphadenectomy.^27–32 After complete resection, up to 27% of patients with primary nonmetastatic PanNETs are at risk for disease recurrence within the first 3 years.³³ Given the expanding therapeutic options of peptide receptor radionucleotide therapy (PRRT), chemotherapy, and targeted therapy alongside repeat resection, surveying these patients with cross-sectional imaging after primary resection is paramount, as patients with recurrent disease may be candidates for salvage therapy.^22,34,35

There is a lack of consensus, however, on the optimal guidelines for surveillance after resection for PanNETs. Current guidelines from the North American Neuroendocrine Cancer Tumor Society (NANETS) recommend cross-sectional imaging 3 to 6 months after resection and then every 6 to 12 months for the subsequent years. The National Comprehensive Cancer Network (NCCN) recommend cross-sectional imaging once within the first year postoperatively, and then anywhere from every 6 to 12 months afterwards.^36,37 The European Neuroendocrine Tumor Society (ENETS), European Society for Medical Oncology (ESMO), and Commonwealth Neuroendocrine Tumor Collaboration dictate surveillance regimens based on tumor grade, with patients recommended to undergo radiographic surveillance as frequently as every 3 to 6 months, even for low-grade tumors.^38,39 Other groups recommend no surveillance for low-risk patients (grade 1, node-negative <2 cm tumors), and higher frequency imaging for patients with higher-risk phenotypes (Ki-67 index >5% and LN-positive disease).⁴⁰

When considering the biological heterogeneity among NF-PanNETs, a standardized “one-size-fits-all” surveillance schema for all patients introduces some potential pit-falls: excessive imaging of patients increases radiation exposure and healthcare costs without any added survival benefit. Alternatively, too little surveillance may delay the diagnosis of potentially treatable recurrent disease. Thus, the primary aim of this study was to use a multi-institutional, international database to develop a prognostic risk score for disease recurrence after resection of primary, nonmetastatic NF-PanNETs to guide individually tailored surveillance strategies. Our secondary aim was to determine potential cost-savings to the US healthcare system when using a risk-stratified recurrence surveillance schedule compared to a standard regimen for all patients.

METHODS

Patients were included from the US Neuroendocrine Tumor Study Group (US-NETSG) database and an international collaborator to form a cohort of 9 academic tertiary and quaternary referral centers (Emory University, San Raffaele Scientific Institute- “Vita-Salute” University, The Ohio State University, Stanford University, Virginia Mason University, Vanderbilt University, University of Michigan, University of Wisconsin, and Washington University in St. Louis). Institutional Review Board (IRB) approval was obtained at each respective study site before data collection. All patients who underwent curative-intent resection of a primary nonfunctioning, nonmetastatic well- or moderately differentiated PanNET from 2000 to 2016 were included.

Demographic, histopathologic, and perioperative data were collected from the electronic medical record. Surgical pathology was reviewed by expert pathologists at each institution. Staging was based on the American Joint Committee on Cancer (AJCC) 7th edition guidelines. Symptomatic tumors were defined by the presence of any preoperative symptom related to the tumor including pain, jaundice, pancreatitis, and bleeding. Proximal tumors were defined as being located within the pancreatic head, whereas distal tumors were located in the neck, body, or tail. Patients who underwent transfusion intraoperatively or during the immediate postoperative period were defined as having any transfusion. The primary outcome was recurrence-free survival (RFS). Disease recurrence was defined strictly as evidence of any suspicious lesion found within the body on cross-sectional imaging which suggests a recurrence of disease, with or without tissue-biopsy pathologic confirmation. Patients were surveilled for disease recurrence per individual protocols at each respective institution. The primary aim of the study was to assess the association between clinicopathologic variables and disease recurrence and to devise a prognostic score to accurately risk-stratify RFS.

To create a Recurrence Risk Score (RRS), patients were randomized 2:1 into a derivation cohort (DC) or validation cohort (VC). Chi-squared analysis was used to compare categorical variables, and Student t test was used for continuous variables. Univariate binary logistic regression analysis was used to determine the association of clinicopathologic factors with disease recurrence in the DC. A multivariable model was constructed using sequential regression entry with variables statistically associated (P < 0.05) with disease recurrence on univariate analysis. Factors in the multivariable model were included based on data availability and excluded if data were redundant between variables (ie, location of tumor and type of resection) or represented in another, more general variable [ie, symptomatic tumors and the presence of preoperative pain, gastrointestinal (GI) bleeding, and biliary obstruction]. The weighted odds ratios [odds artio (OR) minus 1] for variables statistically significantly associated with disease recurrence (P < 0.05) on multivariable analysis were used to create the RRS. Patients were then categorized into score groups based on the proportion of patients with disease recurrence within each score as well as differences in RFS. The VC was subsequently scored using the RRS and patients were placed within their respective score groups. Kaplan-Meier (KM) survival plots were constructed and Cox-regression analysis was done to determine the association of RRS score group with RFS. Given potential differences in disease recurrence for patients with sporadic PanNETs versus those with a familial genetic cancer syndrome, KM plots stratified by RRS score group were also constructed excluding those patients who have a known genetic syndrome. Statistical analysis was conducted using SPSS 22.0 software (IBM Inc., Armonk, NY).

Cost analysis was conducted to estimate total savings (USD) to the US healthcare system during a 2-year period using a risk-stratified surveillance protocol compared to a standard regimen for all patients. The 2018 incidence of resected NF-PanNETs was estimated based on published population data.^36,41–43 Patients were simulated to either undergo a standard, “one-size-fits-all” surveillance regimen at every 3, 4, or 6 months versus a surveillance protocol based on their estimated risk of recurrence using the RRS (low risk: every 12 months, intermediate risk: every 6 months, high risk: every 3 months). Simulated patients no longer received further surveillance in the model once a recurrence was “detected.” The cost model was simulated using a computed tomography (CT)-based protocol (non-contrast CT scan of the chest with a contrast CT of the abdomen/pelvis) and a magnetic resonance imaging (MRI)-based protocol (non-contrast CT scan of the chest with an MRI of the abdomen/pelvis). The prevalence of patients within each risk-score group and the probability of recurrence within each group were estimated using our large database and KM survival estimates, respectively. The cost of surveillance imaging was estimated using the 2018 Medicare Physician Fee Schedule. The cost model was simulated 1000 times and average cost at 2 years is reported. The variation in cost estimates across simulations is because of the random chance at each time point that the patient either recurs and stops surveillance, or does not recur and continues surveillance.

RESULTS

Of the 2421 patients within the US-NETSG and international collaboration database, 1006 met inclusion criteria. Patients were randomized into the DC (n = 681) and VC (n = 325). Demographic, histopathologic, and perioperative data are listed in Table 1. The median age of the entire study population was 59 years [interquartile range (IQR) 49–66] which was similar in the DC and VC (DC: 59, IQR 49–66; VC: 58, IQR 48–67; P = 0.319). Fifty-three percent (n = 529) of the study cohort was male (DC: 53%; VC: 52%; P = 0.957), and the median body mass index (BMI) was 27 (IQR 24–32), which was similar between the DC and VC (DC: 27, IQR 24–32; VC: 27, IQR 24–33; P = 0.439). Forty-two percent (n = 423) of patients presented with a symptomatic tumor (DC: 42; VC: 43; P = 0.923), the most common symptom being pain for 37% of patients (n = 375; DC: 38%; VC: 35%; P = 0.421). The majority of tumors were located in the distal pancreas (n = 720; DC: 72%; VC: 72%; P = 0.463), and the type of resection patients underwent was similar between groups (distal pancreatectomy: n = 651; DC: 65%, VC: 66%; P = 0.704). The median tumor size for the entire study cohort was 2.3 cm (IQR 1.5–3.8), which was similar between DC and VC (DC: 2.2, IQR 1.4–3.8; VC: 2.5, IQR 1.5–3.8; P = 0.481). Eighty-five percent (n = 859) of patients underwent an R0 resection (DC: 85%; VC: 87%; P = 0.307). On final pathologic analysis, the majority of patients had a Ki-67 index <3% (62%, n = 473; DC: 61%; VC: 62%; P = 0.871). Postoperatively, 49% (n = 489) of patients experienced any complication (DC: 49%; VC:49%; P = 0.870). Overall, 13% (n=130) experienced disease recurrence within the study period (DC: 13%; VC: 13%; P = 1.00). Median follow-up was 41.0 months.

TABLE 1.

Clinicopathologic Factors in the Study Cohort, Derivation Cohort, and Validation Cohort

Baseline Variable	US-NETSG+ Italian Cohort (n = 1006, %)	Derivation Cohort (n = 681, %)	Validation Cohort (n = 325, %)	P (Derivation vs Validation)
Age, y, median (IQR)	59 (49–66)	59 (49–66)	58 (48–67)	0.319
Male	529 (53)	359 (53)	170 (52)	0.957
BMI, median (IQR)	27 (24–32)	27 (24–32)	27 (24–33)	0.439
Race				0.707
White	822 (82)	561 (84)	261 (83)
Black	73 (7)	53 (8)	20 (6)
Latino	19 (2)	12 (2)	7 (2)
Performance Status				0.670
Independent	893 (89)	599 (98)	294 (98)
Partially Dependent	21 (2)	14 (2)	7 (2)
Genetic syndrome				0.286
MEN1	54 (5)	42 (6)	12 (4)
von Hippel-Lindau	13 (1)	8 (1)	5 (2)
Neurofibromatosis	1 (0.1)	1 (0.1)	0 (0)
Tuberous sclerosis	2 (0.2)	2 (0.3)	0 (0)
Symptomatic	423 (42)	285 (42)	138 (43)	0.923
Gastrointestinal bleed	22 (2)	19 (3)	3 (1)	0.096
Jaundice	33 (3)	18 (3)	15 (5)	0.147
Gastrointestinal obstruction	4 (0.4)	2 (0.3)	2 (0.6)	0.825
Pancreatitis	64 (6)	42 (6)	22 (7)	0.824
Pain	375 (37)	260 (38)	115 (35)	0.421
Chromogranin A, ng/L, median (IQR)	54 (10–168)	58 (9–157)	50 (12–197)	0.257
Surgical technique				0.688
Open	761 (76)	510 (75)	251 (77)
Laparoscopic	216 (21)	152 (22)	64 (20)
Robotic	27 (3)	18 (3)	9 (3)
Location of tumor				0.463
Proximal	283 (28)	193 (28)	90 (28)
Distal	720 (72)	486 (72)	234 (72)
Type of resection				0.704
Enucleation	73 (7)	55 (8)	18 (6)
Pancreatoduodenectomy	279 (28)	185 (27)	94 (29)
Distal Pancreatectomy	651 (65)	439 (65)	212 (66)
Arterial resection	5 (1)	1 (0.1)	4 (1.2)	0.071
Major venous resection	41 (4)	27 (4)	14 (4)	0.941
Method of Transection				0.450
Sutured	102 (22)	75 (23)	27 (18)
Stapled	346 (74)	230 (71)	116 (78)
Splenectomy	523 (52)	357 (53)	166 (51)	0.741
Lymphadenectomy	697 (69)	466 (69)	231 (72)	0.407
Estimated Blood loss, median (IQR)	200 (100–400)	200 (100–450)	200 (100–400)	0.811
Any Transfusion	185 (18)	92 (14)	93 (36)	<0.001
Tumor size, cm, median (IQR)	2.3 (1.5–3.8)	2.2 (1.4–3.8)	2.5 (1.5–3.8)	0.481
Tumor size >2cm	559 (56)	359 (53)	200 (62)	0.012
Number of tumors, median (IQR)	1 (1–1)	1 (1–1)	1 (1–1)	0.531
Final resection status				0.307
R0	859 (85)	579 (85)	280 (87)
R1	124 (12)	84 (12)	40 (12)
Tumor differentiation				0.571
Well differentiated	849 (84)	571 (92)	278 (93)
Moderately differentiated	73 (7)	52 (8)	21 (7)
Tumor grade				0.697
Low grade	547 (54)	376 (65)	171 (66)
Intermediate grade	283 (28)	198 (34)	85 (33)
High grade	10 (1)	8 (1)	2(1)
Ki-67%				0.871
<3%	473 (62)	322 (61)	151 (62)
3%−20%	289 (37)	198 (37)	91 (37)
>20%	12 (2)	9 (2)	3 (1)
Mitotic rate				0.677
<2	560 (56)	383 (82)	177 (80)
2–20	126 (13)	82 (18)	44 (20)
>20	2 (1)	1 (0.2)	1 (0.5)
Necrosis	57 (6)	35 (7)	22 (9)	0.311
Lymphovascular invasion	263 (26)	177 (29)	86 (30)	0.875
Perineural invasion	178 (18)	116 (21)	62 (23)	0.494
Lymph node positive	232 (23)	156 (26)	76 (27)	0.952
Depth of tumor invasion				0.457
Limited to pancreas	720 (72)	487 (74)	233 (74)
Beyond the pancreas	252 (25)	170 (26)	82 (26)
Into celiac axis or SMA	2 (0.2)	2 (0.3)	0 (0)
T Stage				0.483
T1	416 (41)	292 (43)	124 (38)
T2	315 (31)	202 (30)	113 (35)
T3	266 (26)	182 (27)	84 (26)
T4	2 (0.2)	1 (0.1)	1 (0.3)
Any Postoperative complication	489 (49)	329 (49)	160 (49)	0.870
Highest Clavien-Dindo				0.240
I	1 (0.1)	1 (0.3)	0 (0)
II	137 (14)	89 (23)	48 (24)
III	363 (36)	239 (62)	124 (60)
IV	62 (6)	41 (11)	21 (11)
V	8 (1)	5 (1)	3 (2)
Superficial surgical site infection	39 (4)	29 (4)	10 (3)	0.450
Deep surgical site infection	39 (4)	20 (3)	19 (6)	0.043
Deep vein thrombosis	16 (2)	11 (2)	5 (2)	1.000
Pancreatic fistula				0.412
Grade A	111 (11)	70 (37)	41 (41)
Grade B	11 (1)	9 (5)	2 (2)
Grade C	166 (17)	110 (58)	56 (57)
Reoperation	37 (4)	24 (4)	13 (4)	0.870
Readmission	201 (20)	122 (18)	79 (24)	0.026
Recurrence	130 (13)	88 (13)	42 (13)	1.000

DC

In the DC (n = 681), patients had a 92% 2-year RFS (Fig. 1A). Factors significantly associated with disease recurrence are listed in Table 2 and include: BMI, symptomatic tumors (including GI bleed, biliary obstruction, and pain), open surgical technique, proximal tumor location, pancreatoduodenectomy, major venous resection at the time of operation, lymphadenectomy during operation, receiving any transfusion, tumor size >2 cm, tumor grade, Ki-67 index, mitotic rate, presence of necrosis, lymphovascular invasion, perineural invasion, LN positivity, T stage, deep surgical site infection, deep vein thrombosis, and readmission. On multivariable analysis, when selecting variables which were not redundant (ie, the presence of preoperative symptoms and preoperative pain, bleeding, jaundice) and variables with adequate patient data available, 4 factors persisted as being associated with disease recurrence: symptomatic tumors [OR 1.9, 95% confidence interval (CI) 1.1–3.5, P = 0.041), tumor size >2 cm (OR 2.6, 95% CI 1.2–5.4, P = 0.011), Ki-67 index (<3%: reference; 3%−20%: OR 1.8, 95% CI 1.0–3.3, P = 0.064; >20%: OR 6.7, 95% CI 1.6–28.5, P = 0.010), and LN positivity (OR 1.9, 95% CI 1.1–3.6, P = 0.039).

FIGURE 1.

Recurrence-free survival in (A) all patients in the derivation cohort (n = 681) and stratified by individual recurrence risk score (B) and recurrence risk score group (C).

TABLE 2.

Binary Logistic Regression: Clinicopathologic Factors Associated With Recurrence

	Univariate			Multivariable
Variable	OR	95% CI	P	OR	95% CI	P
BMI	0.94	0.90–0.98	0.008
Symptomatic	2.4	1.5–3.8	<0.001	1.9	1.1–3.5	0.041
Pain	1.8	1.1–2.8	0.013
GI Bleed	6.3	2.3–16.9	<0.001
Biliary Obstruction	3.1	1.3–7.4	0.010
Distal tumor location	0.6	0.4–0.9	0.033	1.2	0.6–2.3	0.556
Type of resection
Enucleation	Ref	—	—
Pancreatoduodenectomy	3.9	1.2–12.6	0.023
Distal pancreatectomy	1.5	0.5–4.4	0.437
Lymphadenectomy	2.5	1.4–4.6	0.002
Surgical technique
Open	Ref	—	—
Laparoscopic	0.13	0.04–0.4	0.001
Robotic	0.28	0.03–2.2	0.223
Major Venous resection	3.6	1.6–8.3	0.003	2.4	0.8–6.9	0.118
Tumor Size >2 cm	4.4	2.5–7.7	<0.001	2.6	1.2–5.4	0.011
Ki 67
<3%	Ref	—	—	Ref	—	—
3%–20%	2.4	1.4–4.1	0.002	1.8	1.0–3.3	0.064
>20%	14.1	3.6–55.9	<0.001	6.7	1.6–28.5	0.010
Tumor grade
Low grade	Ref	—	—
Intermediate grade	2.6	1.5 −4.4	0.001
High grade	20.5	4.7–90.4	<0.001
Mitotic rate
<2	Ref	—	—
2–20	3.6	1.9–6.8	<0.001
>20	0	0–0	1.000
Necrosis	3.9	1.8–8.4	0.001
Lymphovascular invasion	4.7	2.8–7.8	<0.001
Perineural invasion	2.2	1.2–3.7	0.006
Lymph node-positive	3.6	2.2–5.8	<0.001	1.9	1.1–3.6	0.039
T Stage
T0	Ref	—	—
T1	2.8	1.4–5.4	0.002
T2	6.3	3.4–11.8	<0.001
T3	—	—	—
T4	—	—	—
Deep Surgical site infection	3.0	1.1–7.9	0.031
Deep vein thrombosis	5.5	1.4–20.8	0.013
Any transfusion	1.8	1.0–3.3	0.045
Readmission	2.2	1.3–3.6	0.003

Preoperative factors not associated with recurrence were age, sex, race, ASA class, functional status, presence of a genetic syndrome, GI obstruction, pancreatitis, chromogranin A, arterial resection, method of transection, splenectomy, estimated blood loss, number of tumors, final resection status, tumor differentiation, any complication, highest Clavien-Dindo, superficial surgical site infection, pancreatic fistula, and reoperation. On multivariable analysis, factors were included based on data availability and excluded if data were represented in another, more general variable (ie, symptomatic tumors and the presence of preoperative pain, GI bleeding, and biliary obstruction).

RRS

When creating the RRS, the OR of the 4 factors associated with an increased odds of disease recurrence on MV analysis in our DC was considered, specifically: symptomatic tumors, tumor size >2 cm, Ki67 index, and LN positivity. Each factor was assigned a score based on their respective weighted OR (OR minus 1): symptomatic tumor = 1, tumor size >2 cm = 2, Ki-67 <3% = 0, Ki-67 3% to 20% = 1, Ki-67 >20% = 6, LN positivity = 1. Asymptomatic tumors, size <2 cm and LN negative were all assigned a value of zero (0). This created a RRS ranging from 0 to 10. When scoring the DC, only patients who had all 4 factors available for analysis were included (n = 460). The percentage of patients who experienced disease recurrence increased with increasing score: 0 points, 4% (n = 4/89); 1 point, 5% (n = 4/81); 2 points, 4% (n = 3/77); 3 points, 16% (n = 16/99); 4 points, 25% (n = 19/77); 5 points, 35% (n = 9/28); 6 and 7 points, no patients; 8 points, 0% (n = 0/1); 9 points, 75% (n = 3/4); 10 points, 50% (n = 2/4). Two-year RFS also decreased with increasing score (Fig. 1B): 0 points, 99%; 1 point, 99%; 2 points, 99%; 3 points, 89%; 4 points, 85%; 5 points, 78%; 6 and 7 points, no patients; 8 points, RFS data not available; 9 points, 75%; 10 points, 50%. Based on these percentages, 3 score groups were formed: low (0–2 points), intermediate (3–5 points), and high (6–10 points). When applying these score groups to the DC, there was a statistically significant increase in the percentage of patients who had disease recurrence with increasing score group (low: 5%, intermediate: 22%, high: 56%, P < 0.001, Table 3). There was also an association with decreased RFS with increasing score group [low: reference; intermediate: hazard ratio (HR) 5.4, 95% CI 2.8–10.5, P < 0.001; high: HR 33.5,95% CI 11.4–98.6, P < 0.001]. On Kaplan-Meier analysis, there was a decreased 2-year RFS with increasing g score group (low: 98.5%, intermediate: 86.4%, high: 66.7%, P < 0.001, Fig. 1C). When excluding patients with a known genetic cancer syndrome from the DC (n = 413), there continued to be a decreased 2-year RFS with increasing score group (low: 99.5%, intermediate: 86.6%, high: 66.7%; P < 0.001, Fig. 2A).

TABLE 3.

Association of Recurrence Risk Score Group With the Prevalence of Disease Recurrence and Decreased Recurrence-Free Survival in the Derivation and Validation Cohorts

	Derivation Cohort (n = 455)				Validation Cohort (n = 217)
Risk Score	Percent Recurrence	P	HR RFS (95% CI)	P	Percent Recurrence	P	HR RFS (95% CI)	P
Low	5% (n = 11/245)	<0.001	Ref		6% (n = 7/119)	0.001	Ref
Intermediate	22% (n = 44/201)		5.4 (2.8–10.5)	<0.001	18% (n = 17/95)		3.7 (1.5–8.8)	0.004
High	56% (n = 5/9)		33.5 (11.4–98.6)	<0.001	100% (n = 3/3)		22.5 (5.7–88.9)	<0.001

FIGURE 2.

Recurrence-free survival excluding patients with a known genetic syndrome in the (A) derivation cohort (n = 413) and (B) validation cohort (n = 206).

VC

In our VC, patients had a 91% 2-year RFS (Fig. 3A). When applying the RRS to the VC (n = 325), 217 patients were included based on data availability. In the VC, increasing score group was similarly associated with an increasing percentage of patients who experienced disease recurrence (low: 6%, intermediate: 18%, high: 100%; P = 0.001, Table 3). There was also an association with decreased RFS with increasing score group (low: reference; intermediate: HR 3.7,95% CI 1.5–8.8, P = 0.004; high: HR 22.5,95% CI 5.7–88.9, P < 0.001). On Kaplan-Meier analysis, there was a decreased 2-year RFS with increasing score group (low: 98.1%, intermediate: 82.3%, high: 66.7%, P < 0.001, Fig. 3B). When excluding patients with a known genetic cancer syndrome from the VC (n = 206), there continued to be a decreased 2-year RFS with increasing score group (low: 98.0%, intermediate: 82.0%, high: 66.7%; P < 0.001, Fig. 2B).

FIGURE 3.

Recurrence-free survival in (A) all patients in the validation cohort (n = 325) and stratified by recurrence risk score group (B).

Cost Analysis

Using an estimated incidence of resectable, well/moderately differentiated NF-PanNETs in the United States in 2018 as 531 cases per year, the estimated cost-savings for the utilization of a risk-stratified surveillance schedule are listed in Table 4. The estimated cost for cross-sectional imaging based on Medicare reimbursements was $686 for a CT-based protocol and $929 for an MRI-based protocol. If patients were to undergo a risk-stratified surveillance schedule based on the RRS (low risk: q12 months; intermediate risk: q6 months; high risk: q3 months), compared to a standard surveillance schedule of either q3, q4, or q6 months for all patients, the estimated cost savings to the US healthcare system is between $377,600 and $2,394,590 over 2 years (Table 4).

TABLE 4.

Cost Model Results for a 2-Year Surveillance Period

Surveillance Schedule	Mean cost ± SD: CT Protocol	Mean Cost ± SD: MRI Protocol
Standard surveillance, q3 mo	$2,800,404 ± $18,761	$3,790,188 ± $25,915
Risk-stratified surveillance	$1,030,922 ± $9934	$1,395,598 ± $12,890
Cost savings	$1,769,482	$2,394,590
Standard surveillance, q4 mo	$2,101,795 ± $13,751	$2,846,638 ± $18,386
Risk-stratified surveillance	$1,030,922 ± $9934	$1,395,598 ± $12,890
Cost Savings	$1,070,873	$1,451,040
Standard surveillance, q6 mo	$1,408,552 ± $8319	$1,905,812 ± $11,450
Risk-stratified surveillance	$1,030,922 ± $9934	$1,395,598 ± $12,890
Cost savings	$377,600	$510,214

Standard surveillance regimen defined as either a q3-, q4-, or q6-month schedule versus a risk-stratified surveillance regimen defined as q12-, q6, or q3-month schedule for low-, intermediate-, or high-risk groups, respectively. An estimated incidence of 531 cases per year of resectable, well/moderately differentiated NF-PanNETs in the US was used.

DISCUSSION

This study presents a novel, internally validated risk score to guide pragmatic surveillance strategies for patients with resected well/moderately differentiated NF-PanNETs. Using preoperative factors (presence of tumor-related symptoms) and pathologic features (tumor size, LN positivity, and Ki-67 index) associated with disease recurrence, patients were scored on a scale from 0 to 10. After grouping patients into low-, intermediate-, or high-risk groups, the RRS accurately stratified patients by increasing risk of recurrence. Compared to patients in the low-risk group (0–2 points), patients in the intermediate risk group (3–5 points) had between a 4 to 5 times higher likelihood of recurrence, and patients in the high-risk group had between a 23 and 34 times higher likelihood of recurrence. Thus, we recommend that patients in the low-, intermediate-, and high-risk group be surveyed for disease recurrence at every 12-, 6-, or 3-month intervals, respectively, as this minimizes costs to the US healthcare system based on cost-predictive modeling.

In recent years, NF-PanNETs have been increasingly recognized as a heterogeneous group of tumors with diverse biologic behavior. Although some tumors are indolent, other aggressive types are more likely to metastasize and become incurable in spite of recent therapeutic advances. The RRS incorporates known adverse pathologic characteristics associated with NF-PanNETs to stratify the risk of recurrence in these patients. For example, tumor size >2cm renders patients with a T2 lesion, and thus classifies their disease as at least Stage II per American Joint Committee on Cancer (AJCC) and ENETS staging systems.22 LN positivity is another known adverse prognostic factor among PanNETs, with LN positive disease automatically categorizing patients as having AJCC Stage 3 disease.^28,30 It is important to note, however, that in our scoring system, Ki-67 >20% carries the highest prognostic weight for recurrence and places patients into the high-risk group, regardless of other clinic-pathologic features. The prognostic importance of Ki-67 index has been well described in the literature previously, and is reflected in our score.⁴⁴ The only preoperatively assessed clinical feature in our score is the presence of symptoms, specifically pain, gastrointestinal bleed, jaundice, or pancreatitis. Preoperative symptoms may serve as a marker for advanced disease, especially as the incidence of asymptomatic NF-PanNETs increases in the era of ubiquitous cross-sectional imaging.

Of the factors that comprise the RRS, the Ki-67 index is likely the strongest prognostic factor for patients in predicting recurrence. Having a high Ki-67 index >20% generally is associated with the other components in our risk score (larger tumors >2 cm which may cause clinical symptoms and have a higher propensity to metastasize to LN s). Conversely, tumors with a Ki-67 >20% are rarely, if ever, sub-2 cm, asymptomatic, or LN-negative. However, we believe that the strongest application of our score is to discern and accurately stratify those patients in the low- and intermediate-risk groups. It is important to note that patients who have tumors with a Ki-67 index of <3% may be incorrectly considered “low-risk” if only utilizing the Ki-67 index. As we demonstrated, if these patients have a combination of tumors >2 cm and either LN positivity or clinical symptoms, they would be considered “intermediate-risk” as reflected by their increased recurrence rates and would merit more frequent surveillance compared to a low-risk tumor. Thus, even with a Ki-67 index of <3%, they would potentially benefit from a q6 month surveillance schedule. Similarly, having a Ki-67 index of 3% to 20% alone without the other risk factors in our score allows patients to remain within the low-risk score group as this factor receives only 1 point. These patients would require a combination of symptoms, tumor size >2 cm, or LN positivity to be placed into the intermediate-risk category. Thus, the other 3 factors in the risk score do help accurately stratify a tumor with a Ki-67 index of 3% to 20%, which otherwise would be considered an “intermediate-risk” tumor if solely based on the Ki-67 index. We, therefore, feel that compared to using only Ki-67 index, the RRS allows providers to more accurately stratify a patient’s risk of recurrence based on several clinical and pathologic factors.

Although surgery remains the cornerstone of curative treatment for PanNETs, recommendations for surveillance after resection lack uniformity. Leading international groups, such as NANETs and the NCCN, dictate that cross-sectional imaging be performed once 3 to 12 months after resection, then every 6 to 12 months afterwards.^36,37 Given the prognostic importance of tumor grade in NETs, ENETs, and ESMO stratified their recommended surveillance regimens by grade, although the groupings by tumor grade differ: per ENETS, grade 1 tumors should undergo annual surveillance and grade 2 to 3 tumors should undergo q3 month surveillance, whereas ESMO recommends that grade 1 to 2 tumors undergo q3 to 6 month surveillance and grade 3 tumors undergo q2 to 3 month surveillance.^38,39 Not only do these conflicting surveillance recommendations create confusion among physicians, but in the era of increasing scrutiny on the utilization of healthcare resources, over-surveillance leads to excess costs without adding survival benefit. Furthermore, irradiation with cross-sectional imaging places patients at higher risk of developing secondary neoplasms, and compliance with overly-aggressive surveillance regimens is challenging.⁴⁵ The utility of our RRS is in its application of known adverse prognostic features in PanNETs to stratify patients to a more pragmatic surveillance approach, which we demonstrated to save up to $2million during a 2-year period to the US-healthcare system. The cost savings are largely derived from the fact that most patients who undergo resection fall into the low-risk category, and thus applying a more individually tailored surveillance strategy of annual imaging to this group of patients with a very low risk of recurrence has potentially the greatest impact on the economy.

Our study was limited primarily by its retrospective design. When constructing our risk score, many patients were excluded from final analysis because of the absence of data available for all 4 factors in the RRS. This absence limited the number of patients in both the DC and VC, which may have decreased the power of the RRS. Detailed analysis of patients with a score of 6, 7, or 8 was also not possible. In the RRS, however, a Ki-67 >20% is necessary to obtain a score of ≥6. This adverse pathologic factor is usually associated with other clinicopathologic factors including symptoms, enlarged tumor size, and LN positivity, thus making a score of 6 to 8 difficult to generate and almost clinically irrelevant, as the Ki-67 >20% is the primary driving factor of biologic behavior. Furthermore, surgical conduct and pathologic examination may differ between institutions, although all institutions included are high-volume academic tertiary or quaternary referral centers. Lastly, in our cost model, we considered only the costs of the cross-sectional imaging itself to the US Healthcare system based on Medicare Physician Fee Schedule, excluding other third-party payer considerations; the impact of false positive results, and further testing/procedures a positive finding on surveill ance may lead to were not accounted for in the cost model. If applied to standard US insurance payers, the potential savings are considerably higher given the charge and reimbursement levels for testing in the private sector. This is emphasized by the fact that the median patient age was 59 years in our study, which is less than Medicare conventional eligibility. This may actually lead to a greater cost savings with a risk-stratified surveillance strategy.

CONCLUSION

The RRS proposed in the present study is a novel, internally validated tool which incorporates known adverse clinicopathologic factors for patients with nonfunctioning, well/moderately differentiated PanNETs to accurately stratify the risk of disease recurrence after surgical resection. The utility of this score to guide a pragmatic surveillance approach after resection can generate up to a $2million cost-savings to the US-healthcare system during a 2-year period.

DISCUSSANTS

J. Howe (Iowa City, IA):

Dr. Maithel, this is another important article from your group combining data from 8 American and 1 Italian center. This allowed you to bring together a large number of patients with this relatively rare tumor. I applaud your group for the quality of this work and the other studies you have done to date.

The important issue which you addressed is to determine which factors predispose recurrence in patients with resected nonfunctional and non-metastatic PNETs, and from this, this suggests a rational schedule for surveillance of these patients. Current recommendations for major North American and European societies vary from recommending follow-up imaging anywhere from 3 to 12 months, and justifications of these intervals are lacking. This leads to overuse of imaging and laboratory tests for patients at low risk for early recurrence or underuse in those patients at high risk for recurrence that might benefit from earlier recognition or recurrence in medical therapy.

In this article, can you clarify what you defined as recurrence by following these patients radiographically? Did this include arterial enhancing or otherwise suspicious lesions in the pancreas, regional nodes, liver, or elsewhere and did not require biopsy?

Next, your strategy for determining recurrence risk using factors identified by multivariable analysis in using the observed risks to weight these factors seems sound. You held up well in your VC.

One factor which is commonly accepted as a best prognostic factor other than metastasis is the Ki67 level. In this study, high Ki67 accounted for a disproportionate share of the 6 points on your 10-point scale, but only about 2% of patients were in this high-risk group. More than half fell into the low-risk group, and getting to the intermediate-risk group required a combination of a tumor >2 cm, symptoms, positive nodes, or a Ki67 of 3% to 20%.

Did you evaluate how Ki67 performed alone, or consider further separation of Ki67 into, say, 3% to 10% ranges and 11% to 20%?

My final question is an issue which surgeons have been struggling with for some time, is whether nonfunctional PNETs >2 cm should be observed or resected. You had 447 patients with tumors <2 cm, and there was a 4-fold less chance of recurrence in these patients. What percentage of these patients had nodal metastases at presentation? And how many subsequently developed recurrence or metastasis?

Can you comment on whether you think tumors <2 cm should be resected or observed?

Response From S.K. Maithel:

Thank you, Dr. Howe, for those comments and thoughtful questions. In terms of what constituted recurrence, the data field was defined as either a biopsy-proven or a clinically/radiographically suspicious nodule. As the institutions were reviewing the patient charts, they would simply record recurrence if it was regarded as so clinically at that time, regardless if it was biopsy-proven or clinically/radiographically suspicious.

Regarding your question about just using the Ki67, we completely acknowledge that of the factors that constitute our risk score, the Ki67 is most likely the strongest prognostic factor. However, as I pointed out, the majority of patients who undergo resection fall into the <3% category, so we feel that by incorporating these other 3 factors of symptoms, tumor size, and LN status, we are actually better able to discriminate the sub 3% category. For example, a patient who had a tumor with Ki67 <3%, but was >2 cm with either symptoms or LN positivity, would actually fall into the intermediate category and thus, would represent a different biologic behavior than a patient who had a sub 3% tumor and did not have any of those other factors. I think if we just used Ki67 alone, the tendency would be to group all <3% Ki67 tumors into a low-risk category, which would likely be inaccurate. That being said, I agree that once the Ki67 starts increasing, it is probably the main driving factor for recurrence. For example, on the other end of the spectrum, a tumor with Ki67 >20% represents an aggressive tumor, and although we did not have many patients with a risk score of 6, 7, or 8, it is clinically a rare event to have such a tumor that is asymptomatic, <2 cm, and LN-negative.

Further dividing the Ki67 into discrete categories of 3% to 10% or 10% to 20% I think is a fantastic idea, and is something that we tried to do, but we were unable to do because of missing granularity of information in the database. As this database spans 17 years, oftentimes the Ki67 was only recorded as a category, that is less than 3%, 3% to 20%, or >20% so we did not have the granularity of the data to further divide into discrete categories. I think that would be a great idea for future prospective studies. Furthermore, in speaking with Dr. Falconi who leads the pancreatic section of the ENETS Society, he has informed us that the society is thinking of moving toward reclassifying the Ki67 to <5%, 5% to 10%, 10 to 20%, and >20%, thus completely in line with your suggestion.

In terms of the question of the sub 2-cm tumors, I think that is a continued source of debate as we just saw at the recent SSO annual meeting last month. Our group, the US-NETSG, just completed a study that will be published soon where we looked at 309 patients who underwent resection of sub 2-cm tumors, and we found that overall the LN-positive rate was 9%. However, the LN-positive rate increased to >20% in patients who had a sub 2-cm tumor with a Ki67 >3% and was in a proximal pancreas location. When we looked at the RFS of LN-positive sub-2 cm, the 5-year RFS was 80%, which is similar to our intermediate risk category. So I think as we approach sub 2-centimeter tumors, we cannot approach it as one category. We have to treat them on an individual basis. I think patients who have sub 2-cm tumors that are potentially LN positive will probably benefit from resection.

C. Vollmer (Philadelphia, PA):

Might we be a little more bold and effective if you came out and said that the low-risk group, which is the majority of patients, needs no surveillance at all thereafter?

Response From S.K. Maithel:

Thank you, Dr. Vollmer. We were actually thinking about that as well, and I am sure that Dr. Brennan is thinking a very similar thing. The problem is, I think, when approaching patients with a surveillance plan, one has to balance patient reassurance and patient anxiety. I think there is a component of patient anxiety that would be uncomfortable with no surveillance whatsoever even if after resection, you explain to the patient that it has an indolent biologic behavior. Furthermore, although not always involved in surveillance for the low-risk NETs, our medical oncology colleagues may also not be comfortable with a plan of no surveillance. We have to work together as a team to care for these patients, particularly as the medical armamentarium continues to grow. That being said, I think it is very reasonable. There is only about a 1% to 2% risk of recurrence in our low-risk category, so there would definitely be cost savings if you did not do any surveillance at all with very little harm to patients.

A. Lowy (La Jolla, CA):

Shishir, a great job. This kind of study reminds us that in the molecular era, clinical pathological data are still really powerful to help us make decisions.

Do you think using this dataset you can define a group of patients in whom enucleation alone is completely sufficient to further reduce the morbidity of pancreatectomy?

Two, is chest CT scan really necessary for this group of patients, especially if you are talking about surveying a low-risk group? The incidence of chest recurrence has got to be vanishingly small in this group. I am curious about your thoughts.

Response From S.K. Maithel:

Thank you, Dr. Lowy, for those comments and questions. In terms of enucleation, I think a lot of it depends on the risk of LN metastases. Our RRS is not intended to predict LN metastases and thus cannot really select patients that would be reasonable candidates for enucleation, mainly because LN status was utilized in the risk score. That being said, our group has looked into the risk factors for LN positivity in the sub 2-centimeter population, which is probably the population that is most amenable to enucleation, and we found that if they do have a low Ki67 <3% or if they were distally located in the pancreas, the risk of LN metastases was extremely low — <5%. In those patients, I think enucleation would be reasonable, as long as the pancreatic duct proximity was not prohibitive.

In terms of chest CT, you are absolutely right, I personally do not use chest CT a lot for surveillance, particularly for tumors that fall into the low-risk category. We did include chest CT in the cost modeling, however, just for completion sake, as I do believe many practitioners, including some medical oncologists, do utilize CT of the chest as part of their cross-sectional surveillance imaging.

W. Inane (New York, NY):

I have no disclosures. Congratulations on a well-presented study. Two questions.

Number 1, if I looked at your data correctly, it seems like you have a higher percentage of tumors in the tail of the pancreas, but islet cells are evenly distributed throughout the head, body, and tail. Do you think that this observation in any way introduced some sort of selection bias and that perhaps patients who had tumors in the head of the pancreas were more likely to undergo surveillance?

Number 2, I would value your group’s opinion on the role of gallium 68 dotatate PET scanning which has become a very useful study for surveillance. Have you applied your methodology to that effective imaging modality?

Response From S.K. Maithel:

Thank you for those questions, Dr. Inane. Yes, you are absolutely right, the predominant location was in the tail. It is hard to know, given that this is a database of patients who underwent resection, the exact denominator of patients including those who were being observed. From my own personal recollection of my own patients, I don ot think there is a higher proportion of patients being observed with tumors in the head, unless of course one is specifically looking at that sub 2 cm category. When you look at the sub 2 cm category, I think the threshold to do a Whipple may be a little higher than it is to perform a distal pancreatectomy. I cannot really comment exactly on whether the distribution of tumors is introducing bias or just was the characteristics of this cohort.

As far as the dotatate scan, I think it is an expensive test, it is a very valuable test, but it needs to be applied strategically and selectively. I think doing a dotatate scan for a patient with a low-risk tumor such as a 2-cm neuroendocrine tumor that has no other risk factors and has a low Ki67 index of <3% would be a suboptimal utilization of resources and in that situation, it is probably not going to be helpful at all and very unlikely to change the treatment or natural history of that patient. For high-risk tumors, however, that are high-risk for having or developing metastatic disease, I think they could be used very well in that setting. Now with the introduction of PRRT in this country, dotatate scan has become the standard to determine whether or not a patient is a candidate for PRRT

C. Wolfgang (Baltimore, MD):

When you look at your dataset, what really drives the boss, as has been shown in other studies, is the Ki67 which is the minority of patients. When you think about how this model or how this score will be used, at the end of the day, high Ki67, independent of this model, is usually who is followed closely, and that sort of correlates with what your model is showing. I still think it gives further refinement. But the real sort of bang for the buck would be in those with a Ki67 of <2%. You have a large dataset obviously with tissue available because you have the Ki67s. Is there any data that you have on the pathways which are known to correlate with tumor biology so the ATTRX pathway or the ALT pathway, and further substratifying the 2% which is the majority of your patients?

Then the second question, I am surprised that in your model for follow-up that age is not a factor in determining, so thank we know there is disease-specific mortality with neuroendocrine tumor but it is relatively indolent and kills during the period of 10 to 20 years as opposed to ducal adeno which is in months. So a resected PNET in an 80-year-old who has less of a life expectancy is different than the same resected PNET in a 50-year-old. I wonder whether you thought of addressing that in your model.

Response From S.K. Maithel:

Thank you, Dr. Wolfgang, for those insightful questions. Our group has further examined the sub 3% Ki67category, and we found that patients with tumors with a Ki67 of 2% to 3% versus <2%, had distinct biologic behavior. Thus, although we tend to group all <3% tumors in the same low-risk category, there did seem to be a different biologic behavior in the 2% to 3% subgroup. That being said, all of them did relatively well, so the clinical significance of further subcategorizing such low Ki67 values is debatable.

We have not looked into a lot of different pathways but we just published a study where we did assess different vascular proliferation pathways, and we did find that patients with tumors that had STAT-3 overexpression did actually have worse RFS after resection. We need to test and validate that finding in a larger cohort of patients.

In terms of considering age, that is a great question. I think it is very pertinent to this population as well, and also to Dr. Vollmer’s question about potentially doing no surveillance, particularly as it applies to an older patient. The median age in this cohort was 59 years, thus we did not really factor age into our RRS. However, it is something we will definitely look into to see if we can further titrate and improve upon our current model.

N. Perrier (Houston, TX):

Thank you, Shishir, for a nice, large study that is very informative.

If I read your early slides correctly, you did include the MEN population in your dataset. First, I am wondering whether you are able to discern an outcome difference between those with germline MEN mutations versus those with sporadic menin mutations. These are really different populations.

Second, can you segregate by type of mutation? There are studies suggesting that some mutations are more aggressive than others. This is really relevant in that the new MEN guidelines with attempts to determine screening intervals—particularly with regard to prevention and prediction of PNETs for young patients genetically screened and found to be positive.

Does your group has any ability moving forward to look at endoscopic ultrasound use and to incorporate that in the model? I know we have been talking about oncologic therapies, but those with germline mutations are followed by medical endocrinologists, and it would be interesting to know whether in the MEN population whether endoscopic ultrasound is a more common modality and more sensitive? Your opinion would be welcomed.

S.K. Maithel:

Thank you, Dr. Perrier, for the very thoughtful questions. We did address this issue in the manuscript, but I did not present that data today because of time constraints. We actually did exclude the familial patients, including MEN and Von Hippel-Lindau, and did a separate analysis where we applied the risk score to the population with and without those patients, and found that still the risk score had identical results in both groups. So it does seem even when including those familial patients, the risk factors that comprise the risk score are still driving the risk of recurrence.

I think this risk score is predominantly designed for a distant recurrence, as that was the predominance of recurrence in our patient cohort. On the contrary, when assessing patients with MEN and Von Hippel-Lindau, often times we are talking about local regional recurrence, hence the use of endoscopic ultrasound for surveillance. It is not something we are able to look at in this dataset. The granularity of the data is not clear enough across these 9 institutions during a 17-year period to do such an analysis. I think it would be a great idea for future studies.

In terms of discriminating between types of familial conditions, specifically between VHL or MEN or specific genetic mutations within, that is something we have not done. Again, we did not have the granularity of specific mutation analysis. We treated them as a single group and found that the risk score quite nicely discriminated patients whether or not we included patients with familial NETs.

N. Perrier (Houston, TX):

You have got such a great study that we will wait for that answer because for the MEN1 population it would really be helpful.