Comparison of the diagnostic accuracy of three current guidelines for the evaluation of asymptomatic pancreatic cystic neoplasms

Supplemental Digital Content is available in the text

Abstract

Asymptomatic pancreatic cysts are a common clinical problem but only a minority of these cases progress to cancer. Our aim was to compare the accuracy to detect malignancy of the 2015 American Gastroenterological Association (AGA), the 2012 International Consensus/Fukuoka (Fukuoka guidelines [FG]), and the 2010 American College of Radiology (ACR) guidelines.

We conducted a retrospective study at 3 referral centers for all patients who underwent resection for an asymptomatic pancreatic cyst between January 2008 and December 2013. We compared the accuracy of 3 guidelines in predicting high-grade dysplasia (HGD) or cancer in resected cysts. We performed logistic regression analyses to examine the association between cyst features and risk of HGD or cancer.

A total of 269 patients met inclusion criteria. A total of 228 (84.8%) had a benign diagnosis or low-grade dysplasia on surgical pathology, and 41 patients (15.2%) had either HGD (n = 14) or invasive cancer (n = 27). Of the 41 patients with HGD or cancer on resection, only 3 patients would have met the AGA guideline's indications for resection based on the preoperative cyst characteristics, whereas 30/41 patients would have met the FG criteria for resection and 22/41 patients met the ACR criteria. The sensitivity, specificity, positive predictive value, negative predictive value of HGD, and/or cancer of the AGA guidelines were 7.3%, 88.2%, 10%, and 84.1%, compared to 73.2%, 45.6%, 19.5%, and 90.4% for the FG and 53.7%, 61%, 19.8%, and 88% for the ACR guidelines. In multivariable analysis, cyst size >3 cm, compared to ≤3 cm, (odds ratio [OR] = 2.08, 95% confidence interval [CI] = 1.11, 4.2) and each year increase in age (OR = 1.07, 95% CI = 1.03, 1.11) were positively associated with risk of HGD or cancer on resection.

In patients with asymptomatic branch duct-intraductal papillary mucinous neoplasms or mucinous cystic neoplasms who underwent resection, the prevalence rate of HGD or cancer was 15.2%. Using the 2015 AGA criteria for resection would have missed 92.6% of patients with HGD or cancer. The more “inclusive” FG and ACR had a higher sensitivity for HGD or cancer but lower specificity. Given the current deficiencies of these guidelines, it will be important to determine the acceptable rate of false-positives in order to prevent a single true-positive.

1. Introduction

Asymptomatic pancreatic cysts are a common clinical problem but only a minority of these cases progress to cancer. Mucinous cysts, which include intraductal papillary mucinous neoplasm (IPMN) and mucinous cystic neoplasms (MCNs), are considered premalignant lesions with the potential to progress to cancer. However, natural history studies and surgical series of IPMNs estimate their rate of malignant transformation range from 6% to 40%.^[1–4] A recent pooled analysis of 12 surgical series involving over 600 patients with surgically resected MCNs found a 15% rate of malignancy.^[5–7] The major challenge in the surveillance of asymptomatic IPMNs and MCNs is their slow rate of cancer progression and the inability to distinguish those that progress. Known and suspected risk factors for cyst malignancy include size of cyst ≥3 cm, presence of a solid component or mural nodule within the cyst, main-duct IPMN (MD-IPMN), and dilation of the pancreatic duct (PD).^[5] Based on these cyst features multiple management guidelines have been proposed to refine the indications for surgical resection and improve our ability to identify cysts at higher risk of malignant transformation. There are currently 3 main guidelines for the management for pancreatic cystic lesions: The 2012 International Consensus Guideline, also known as the Fukuoka Guidelines (FG),^[7] the 2015 American Gastroenterological Association (AGA) guidelines,^[8] and the American College of Radiology (ACR).^[9] The latter 2 guidelines address only asymptomatic, incidentally discovered pancreatic cysts whereas the 2012 FG guidelines also include symptomatic patients in their treatment algorithm. The AGA guidelines further exclude all MD-IPMN which are addressed by the other guidelines. Even in the cohort of asymptomatic patients with incidentally discovered cysts there are significant differences in the resection thresholds suggested by these 3 guidelines. There is no study to date that has directly compared the diagnostic accuracy of these disparate guidelines in a low-risk population, namely those with incidentally discovered, asymptomatic non-MD cysts. Our aim was to compare the accuracy of the 2015 AGA guidelines, the 2012 FG, and the 2010 ACR guidelines to detect malignancy in cysts by retrospectively applying these criteria to patients at our centers who have already undergone surgical resection for an asymptomatic, incidentally discovered pancreatic cyst and thus have final pathology available to affirm the presence or absence of suspected high-grade dysplasia (HGD) or invasive cancer.

2. Methods

2.1. Patient selection

We conducted a retrospective study at 3 academic referral centers for patients who underwent surgical resection for an asymptomatic pancreatic cyst from January 2008 to December 2013. Only patients with MCN and branch duct (BD)-IPMNs (confirmed on surgical pathology) were included in the study. MD-IPMNs (defined as cysts with MD dilation >1 cm) were excluded, as there is general agreement that these cysts should be resected in fit patients because of their higher risk of malignant transformation. Because we excluded all cysts with an associated PD dilation of >1 cm some BD-IPMNs with secondary MD dilation were also excluded so that our study cohort would exactly match the population addressed by the AGA guidelines. The subjects were identified from prospectively maintained pancreatic cyst databases. The databases capture all patients who underwent surgical resection for any type of pancreatic cystic neoplasm. Patients with nonspecific abdominal pain not attributable to pancreatitis with an incidentally discovered cyst on imaging were included in our study after individual chart review by the investigators. We excluded patients with any preoperative symptoms of chronic pancreatitis, unexplained weight loss, jaundice, or findings consistent with MD-IPMN. In total, 269 patients (center 1 = 119, center 2 = 106, and center 3 = 44) met inclusion criteria (Fig. 1). The Institutional Review Board of each center approved the study.

Figure 1.

Flowchart of patients from 3 centers with asymptomatic pancreatic cysts after excluding patients with main-duct intraductal papillary mucinous neoplasm (IPMN), symptomatic cysts, and solid pancreatic masses.

2.2. Predictors used in comparing guidelines

Based on the 2012 FG guidelines, in an asymptomatic patient with an incidental pancreatic cyst, the presence of any one of the following cyst features seen on imaging and confirmed by endoscopic ultrasound (EUS) were considered indications for surgical resection: cyst size ≥3 cm, presence of mural nodule or solid component, dilation of the PD > 5 mm but not meeting criteria of MD-IPMN, or suspicious or positive cytology for malignant cells (Tanaka 2012). The AGA guidelines recognize 3 “high risk features”: cyst size ≥3 cm, mural nodule, and dilation of the PD >5 mm. The AGA recommends EUS with fine needle aspiration (EUS-FNA) when at least 2 of these features are present on cross-sectional imaging and proceed to surgical resection after EUS if at least 2 of these “high risk” cyst features are confirmed on EUS or there is positive cytology on FNA. Last the ACRs recommendation for surgical resection is based heavily on cyst size alone with a cutoff size of >3 cm for resection in surgically fit patients. We excluded all symptomatic patients or those with obstructive jaundice as all 3 guidelines agree these patients should be offered surgical resection if medically fit. We then retrospectively reviewed and applied each of the 3 different guideline criteria for operative management to each patient's preoperative imaging and EUS-FNA data to determine whether they would have met the respective guideline criteria for resection (Table 1).

Table 1.

Major guideline recommendations for resection of asymptomatic mucinous cyst.

2.3. Pathologic diagnosis

All pathologic diagnoses were based on the resected pancreatic surgical specimen. Malignant cysts were defined as IPMN with HGD or pancreatic adenocarcinoma, mucinous cystadenocarcinoma, and pancreatic neuroendocrine cystic tumors. Benign cysts were defined as IPMN or MCN with low grade or moderate dysplasia, lymphoepithelial cyst, serous cyst, or other benign cystic lesions.

2.4. Statistical analysis

The baseline characteristics of the study cohort were compared by 2-tailed Student t test for continuous variables and chi-squared tests for categorical variables. The performance characteristics of the guidelines were evaluated by calculating the sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and accuracy of the 3 guidelines in predicting HGD or invasive cancer. Fisher exact test was used in the univariate analysis to identify predictors of HGD or cancer on resection. All covariates with P-value <.2 in the univariate analysis are included in the multivariable model. Multivariable logistic regression analysis was used to calculate odds ratios (ORs) and 95% confidence intervals (CIs) to identify preoperative cyst features that were associated with risk of HGD and/or cancer on resection. All statistical analyses were performed with STATA software.

3. Results

3.1. Patient characteristics

The baseline characteristics of the study cohort, overall, and by surgical pathology classification of benign versus malignant are shown in Table 2. Of the 269 patients included, 228 (84.8%) had a benign diagnosis or low-grade cystic neoplasm based on surgical pathology (defined as low grade or moderate grade IPMN or MCN), and 41 patients (15.2%) were classified as malignant as they were diagnosed as either HGD (n = 14) or invasive cancer (n = 27) on surgical pathology. The majority of patients were female (71.1%). Patients who had HGD or cancer were significantly older than those who had a benign or low-grade cyst (74 vs 66.4 years, respectively, P < .01). The most commonly seen “high risk” features in all patients were a cyst size greater than 3 cm (41%), presence of mural nodule (20%), and PD dilation (7%). Among those patients with benign or low-grade cysts, compared to those with HGD or invasive cancer, 39% versus 51% had a cyst size ≥3 cm, 19.3% versus 21.9% had a visible mural nodule on imaging, and 7% versus 7.3% had PD dilation of 5 to 9 mm; no statistically significant differences were observed. Of those who ultimately had a cyst with HGD or adenocarcinoma on surgical resection, 73.1% would have met the FG criteria for resection, 51% met the ACR criteria for resection based on cyst size >3 cm, and only 7.3% met the AGA resection criteria.

Table 2.

Characteristics of the study cohort stratified by final pathology.

There are some inherent differences in the clinical practices of the 3 centers represented in this study, most notably in the frequency of EUS used in the evaluation of pancreatic cysts (supplemental Table 1). In center 1, EUS was selectively performed in only a subset of patients (the other centers performed EUS in all patients). In center 1, EUS was performed only in cases when imaging did not met FG criteria (3 additional patients underwent EUS with FG criteria). Our study was not powered to evaluate the added benefit of EUS in this cohort. However, in center 1, the additional yield of identifying a mural nodule or cytology with malignancy in these cases was seen in 13% of cases (6 additional mural nodules and 1 positive cytology). For those patients who did not undergo EUS at center 1 the AGA guideline algorithm for the use of EUS were extrapolated from the cross-sectional imaging features found in those patients. The AGA guideline criteria for surgical resection are similar to the AGA criteria for the use of EUS-FNA.

Despite the overall high percentage of patients who underwent EUS in the entire cohort (75.8%), FNA was positive, defined by suspicious or frankly malignant cells, in only 3 patients. These were all confirmed on resection to be invasive cancer. There was no false-positive cytology in our cohort.

3.2. Comparison of the diagnostic value of the 2015 AGA, 2012 international guidelines, and 2010 ACR recommendations for cyst resection

Of the 41 patients with HGD or cancer on resection, only 3 patients met the AGA guideline's indications for resection based on having at least 2 high-risk cyst features preoperatively. In contrast, 30 out of the 41 patients met the FG criteria for resection and 22 out of 41 patients met the ACR recommendation to undergo resection (Table 3). Hence, the “miss rate” for HGD or cancer is 92.7% if the AGA guidelines for resection were applied versus a “miss rate” of 48.8% and 26.8% using the ACR and FG guidelines, respectively. Conversely, of the 228 patients who ultimately had a benign diagnosis, “unnecessary” surgery would have been recommended for a low grade or benign cystic lesion in 11.8% of patients if the AGA guidelines were applied, compared to 39% and 54% if the ACR and FG were used (Table 2). The sensitivity, specificity, PPV, and NPV of the AGA guidelines were 7.3%, 88.2%, 10%, and 84.1% compared to 73.2%, 45.6%, 19.4%, and 90.4% for the FG (Table 4). Using the lone criteria of size >3 cm the ACR guidelines had a sensitivity of 53.7%, specificity of 61%, PPV 19.8%, and NPV 88% for HGD or cancer on resection. The overall diagnostic accuracy of the AGA guidelines was highest at 75.8% versus 49.8% for the FG guidelines and 59.8% for the ACR guidelines.

Table 3.

Proportion of patients with HGD or cancer by guideline criteria for resection.

Table 4.

Comparison of the performance characteristics of the major guidelines in predicting HGD or cancer on resection of asymptomatic pancreatic cysts.

3.3. Univariable and multivariable analysis for the prediction of a malignant cyst

All covariates associated with HGD or cancer with a P value <.2 in the univariable analysis were included in the multivariable model. In the multivariable analysis, older age (OR 1.07, 95%CI 1.03–1.11 for each 1 year increment, P < .01) and cyst size >3 cm compared to <=3 cm (OR 2.08, 95%CI 1.02–4.27, P = .04) were associated with an increased odds of HGD or cancer on resection.

4. Discussion

Incidentally discovered, asymptomatic pancreatic cysts from imaging studies are a common clinical problem with an estimated prevalence of 3% to 15%. The incidence of these cysts increases with age.^[10] Although the majority of these asymptomatic cysts pose no significant clinical threat, a small proportion of IPMNs and MCNs, can progress to cancer. Once invasive IPMN- or MCN-based carcinoma is identified and there is locoregional lymph node involvement, outcomes are similar to very poor survival of stage-matched outcomes with pancreatic ductal adenocarcinoma.^[11] Prior to the publication of the 2015 AGA guidelines the most widely used treatment algorithm in the management of these asymptomatic pancreatic cysts was based on the 2012 International Guidelines which recommended resection for all MD-IPMNs, BD-IPMNs that are causing symptoms (specifically jaundice, weight loss, or pancreatitis), those with high-risk stigmata confirmed on EUS such as mural nodule, dilation of the PD, positive cytology for malignancy, and size greater than 3 cm in those who are surgically fit.^[7] Multiple cohort studies have shown that the 2012 FG guidelines decrease the number of patients who underwent unnecessary surgery for low risk cysts, improve diagnostic accuracy, and have a higher specificity compared to the previous guidelines.^[12–15] However, the relative predictive accuracy and importance of any individual high-risk cyst feature and the cumulative risk conferred when multiple features are present remains unclear. A large meta-analysis involving 5788 patients found that cyst size >3 cm was the most highly predictive feature of malignancy with OR of 62, compared to mural nodule with OR 9 and MD involvement, with OR 7.^[16] In line with this meta-analysis, the ACRs recommendation for management of incidentally found, undifferentiated pancreatic cysts on imaging suggests EUS for characterization, cytology and then resection for nonserous cysts based solely on a size cutoff of greater than 3 cm.^[9] In contrast to both of those guidelines the 2015 AGA guidelines and the accompanying technical review offered the most stringent criteria for the resection of asymptomatic cysts to date. The AGA guidelines first require the presence of at least 2 high-risk features seen on cross-sectional imaging for referral for EUS to further verify the presence of these features. If EUS confirms at least 2 high-risk features are present, then the recommendation is made for resection.

In our cohort of low risk patients with asymptomatic BD-IPMN or MCNs who underwent resection, the overall rate of HGD or cancer was 15.2%. This is within range of the reported rate of malignancy in similar low-risk populations in the literature.^[16,17] Only 11% of all patients in our study would have met the AGA criteria to undergo resection and 7.3% of those who ultimately had HGD or cancer on resection met the AGA surgical threshold. We found that using the 2015 AGA guidelines improved the specificity of finding a malignant cyst on resection to 88.2% compared to 45.6% and 61% using the 2012 FG and 2010 ACR guidelines. However, the sensitivity of the AGA guidelines for detecting HGD or cancer is significantly lower at only 7.3% compared to 73.2% (2012 FG) and 53.7% (2010 ACR). With the higher threshold for resection in the AGA guidelines fewer patients (11.8%) would have undergone “premature” surgery based on a benign or low-grade cyst on resection.

The frequency of surgery for benign pathology in our cohort may be due to 2 factors. First, our database range includes patients from as early as 2008 prior to the publication of the ACR or FG guidelines when less restrictive criteria could have been used to prompt resection. Second, some of these patients were referred to surgery based on a preoperative suspicion for MCN from an elevated cyst fluid carcinoembryonic antigen or amylase level obtained during EUS-FNA. There is still some controversy regarding the best management of MCN with the FG guidelines recommending resection of all suspected MCNs regardless of size or presence of HGD.^[7]

We found that a preoperative cyst size ≥3 cm and advanced age were independently predictive of malignancy. This is similar to the findings of the meta-analysis by Anand et al^[16] which identified a cyst size ≥3 cm as the highest risk feature predictive of malignant transformation. Other studies have suggested the presence of mural nodule to be highly specific for malignancy but this cyst feature was not found to be significantly predictive of cancer in our study.^[14] One of the key limitations in the development of accurate guidelines for the management of pancreatic cysts has been that no single cyst feature has been consistently found across the literature to be both highly sensitive and specific for malignant potential. The ongoing investigation of molecular markers may be helpful in further refinement of our treatment algorithms and prognostication for asymptomatic cysts. Whole genome sequencing has identified some promising markers of dysplastic and malignant cysts including gene mutations in TP53, PIK3CA, and other mutations leading to loss of PTEN expression.^[18–20] One recently published study looking at a small subgroup of patients with IPMNs who had molecular testing in addition to the existing algorithms for risk-stratifying mucinous cysts found the addition of molecular testing increased the sensitivity for malignancy to 100%, specificity of 90% in identifying high-grade or malignant IPMNs.^[21] Advances in molecular testing of pancreatic cysts to both categorize and prognostic for discrimination and accuracy requires larger scale, prospective validation but is a promising addition to the imaging based risk-stratification algorithm we currently use.

One of the main limitations of our study is its retrospective nature. This is a major limitation in almost all studies of the natural history of these pancreatic cysts as the absolute rate of malignant transformation is low and the time to malignancy tends to be long. In addition all patients in our study had been determined to need surgical resection for their asymptomatic pancreatic cyst and thus may reflect an enriched population with higher prevalence of concerning cyst features and thus a higher rate of malignancy compared to a surveillance-only population. However, we chose a surgical population for this study to ensure that the histopathology of the cysts is confirmed and the presence of HGD or malignancy is unambiguous. An important additional limitation is inherent to the binomial categorization of size measurements for PD or cysts. For example, in a clinical setting a 9 mm PD would be just as concerning as a 10 mm, but in using this criteria, this minimal difference shifts patients from one group to the other.

In summary, we evaluated a large combined cohort of asymptomatic patients with MCNs and applied 3 leading guidelines to compare the relative accuracy of each guideline in predicting advanced neoplasia on resection. The stringent resection criteria of the AGA guidelines led to a high rate of missed cancers and suffer from poor sensitivity compared to the 2012 FG and 2010 ACR guidelines. The 2012 FG guidelines had the highest sensitivity in detecting neoplasia at 73.2% but this is offset by low specificity of only 45.6%, thereby increasing the number of unnecessary surgeries if these guidelines were followed. There was no single cyst feature which was independently predictive of malignancy. Given the grave prognosis of pancreatic cancer and the poor relative performance of all these guidelines, there is still an unresolved clinical dilemma of determining the acceptable rate of false-positives in order to prevent a single cancer. Based on our findings, we believe the strict use of the AGA guidelines will lead to missing too many cancers or high risk cyst lesions and therefore the routine use of these guidelines in practice should not be adopted. However, our study also shows the limitations associated with the earlier guidelines and emphasizes the need for further research in this area to identify better clinical predictors.