Development of an artificial intelligence-based prediction platform for early recurrence of resectable pancreatic cancer after curative surgery–toward future use as an indication for neoadjuvant treatment: a retrospective multicenter cohort study

Abstract

Purpose

Neoadjuvant treatment (NAT) is now the standard for borderline resectable pancreatic cancer (RPC) and is being considered for RPC. Early recurrence after curative surgery in RPC is often seen as a treatment failure, prompting considerations for NAT. Our goal was to develop an artificial intelligence (AI)-based predictive model utilizing preoperatively available factors to forecast early recurrences of resected RPC.

Methods

This study included 469 patients who underwent surgery for RPC between 2011 and 2019. Clinicopathologic and oncologic data were retrospectively reviewed. Preoperative variables, including laboratory data and imaging findings, were collected. Early recurrence was defined as recurrence occurring within a year after surgery. Deep neural networks were then used to select variables by assessing their importance. A new model predicting early recurrence of RPC was subsequently developed.

Results

Of the patients evaluated, 199 (42.4%) experienced early recurrence. The predictive model included 14 preoperative variables: CA 19-9, preoperative pancreatitis, serum albumin, platelet count, lymphocyte count, the American Society of Anesthesiologists physical status classification, tumor size, monocyte count, age, body mass index, CRP, hemoglobin, WBC count, and CEA. The area under the curve for the model was 0.786 in the training set and 0.734 in the test set.

Conclusion

We developed an AI-based model to predict the early recurrence of RPC using preoperative parameters. By identifying patients at risk of early recurrence, optimal individualized treatments such as NAT can be considered. Future prospective studies are crucial to establish clear indications for NAT in RPC.

INTRODUCTION

Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal malignancies. According to recent data from the American Cancer Society, the 5-year relative survival rate for pancreatic cancer across all stages is 13% [1]. Surgical resection is the only curative treatment so far, yet recurrence is prevalent. Earlier studies have documented that up to 85% of patients undergoing curative surgery for PDAC would experience recurrence [2,3], and in 35%–50%, recurrence would occur within a year after surgery [4,5].

Neoadjuvant treatment (NAT) is now widely accepted, especially for borderline resectable pancreatic cancer (BRPC), to attain resectability and enhance survival after resection [6]. NAT is also considered for resectable pancreatic cancer (RPC), along with the recognition that early recurrence of RPC should be seen as treatment failure [7]. The recent guidelines from the National Comprehensive Cancer Network (NCCN) recommend NAT for RPC with high-risk features, such as markedly elevated serum CA 19-9 or large primary tumors [8]. However, these guidelines present only subjective and ambiguous clinical features, lacking specific criteria or cutoff values. Consequently, numerous ongoing studies aim to identify potential candidates for NAT among patients with RPC [9,10].

Early recurrence, typically defined as recurrence within 12 months after curative resection, has been recognized as a marker of biologically aggressive pancreatic cancer. Several studies have reported that patients who experience early recurrence have significantly poorer survival outcomes compared to those with late recurrence or no recurrence [4,5]. While the role of NAT in RPC remains under investigation, identifying patients at high risk for early recurrence may help inform treatment planning and facilitate risk-adapted approaches. Therefore, predicting early recurrence using preoperative variables can offer clinically relevant insights, even in patients who are deemed technically resectable.

Meanwhile, there has been growing interest in applying artificial intelligence (AI) for predicting prognosis in patients with malignancy. It is expected that a number of limitations of conventional statistics could be compensated for by AI-driven analytics. Several previous reports showed promising results in applying AI for early detection and survival prediction in pancreatic cancer [11,12,13].

In this study, our objective was to develop an AI-based calculator that predicts early recurrence of RPC following curative resection. The newly developed platform leverages preoperative factors to estimate the likelihood of recurrence within a year after surgery, enabling the selection of appropriate treatment (surgery or NAT) for patients with RPC.

METHODS

Ethics statement

The Institutional Review Boards (IRBs) of Samsung Medical Center (SMC 2022-08-168) and Severance Hospital, Yonsei University College of Medicine (YUHS 4-2024-0900) approved this study. The IRBs also waived the need for written informed consent since the research presented no more than minimal risk to subjects and there was no anticipated objection.

Patient database

Data from patients who underwent surgery with curative intent for PDAC at Samsung Medical Center and Severance Hospital between 2011 and 2019 were retrospectively reviewed. Patients with RPC were included in this analysis, and resectability was evaluated using the recent NCCN guidelines, which involved preoperative CT scans, MRI, and PET scans [6]. Patients who underwent any form of NAT or lacked data on recurrence were excluded from the analysis.

Preoperative clinical variables

Demographic information and preoperative clinical data, including laboratory and imaging findings, were collected. Underlying medical conditions were documented from medical histories or physician’s prescriptions. Cardiovascular diseases include hypertension, angina, myocardial infarction, heart failure, and cerebral vascular disease. Chronic liver diseases included chronic viral hepatitis and liver cirrhosis. Blood tests captured complete blood counts (CBC) with differentials and tumor markers; CA 19-9 and CEA. Preoperative tumor sizes were measured using the longest diameter from CT or MRI scans, selecting the larger of any multiple measurements if available. All preoperative assessments were performed within a month before surgery, and the most recent results were utilized in the analysis.

Oncologic outcomes and recurrence

Pancreaticoduodenectomy, left-sided pancreatectomy, or total pancreatectomy was conducted by hepatobiliary specialists at each institution. The final pathology reports stated tumor stages based on the 8th edition of the American Joint Committee on Cancer system [14].

After discharge, patients underwent regular follow-ups every two to three months, which included blood tests and imaging studies. Recurrence was suspected when a patient exhibited elevated tumor markers and suspicious lesions on imaging scans. A PET scan or biopsy was then performed to confirm recurrence.

Early recurrence was defined as recurrence within one year following curative surgery (recurrence-free survival <12 months).

Statistical analysis

The Student t-test and chi-square test were used to compare clinicopathological characteristics between the early recurrence group and the no/late recurrence group. Binary logistic regression analysis was conducted to identify risk factors associated with early recurrence. Odds ratios (ORs) and 95% confidence intervals (CIs) were reported. For the machine learning analysis, the random forest model from ‘h2o’ was employed. The predictive power of the newly developed model was evaluated by plotting the receiver operating characteristic (ROC) curve and measuring the area under the curve (AUC).

All statistical analyses were performed using SAS software ver. 9.4 (SAS Institute Inc.) and R software ver. 4.0.5 (R Foundation for Statistical Computing).

RESULTS

A total of 469 patients were included in the analysis. Among them, 199 (42.4%) experienced early recurrence within 12 months, 130 (27.7%) experienced late recurrence, and 140 (29.9%) remained recurrence-free during the follow-up period. Table 1 presents the clinicopathological characteristics of the cohort and comparisons between the early recurrence group and the no/late recurrence group. The prevalence of chronic pancreatitis was significantly higher in the early recurrence group than in the no/late recurrence group (16.6% vs. 6.7%, P = 0.001). Patients with early recurrence exhibited higher mean preoperative CA 19-9 levels compared to those with no or late recurrence (771.7 U/mL vs. 419.5 U/mL, P = 0.001). The mean preoperative platelet counts also differed significantly between the groups (251,000/µL vs. 238,000/µL, P = 0.012). The estimated preoperative tumor size was larger in the early recurrence group than in the no/late recurrence group (2.6 cm vs. 2.4 cm, P = 0.024). Regarding outcomes, the early recurrence group had a higher proportion of patients with stage III disease compared to the no/late recurrence group (28.7% vs. 15.5%, P < 0.001). The median overall survival was 16 months in the early recurrence group and 42 months in the no/late recurrence group (P < 0.001).

Table 1. Comparisons of clinicopathological characteristics between patients with early recurrence and those with no or late recurrence.

Values are presented as number only, mean ± standard deviation, number (%), or median only.

ASA, American Society of Anesthesiology; PS, physical status; PV/SMV, portal vein/superior mesenteric vein; AJCC, American Joint Committee on Cancer.

^a)The group labeled ‘No or late recurrence’ includes 140 patients with no recurrence and 130 patients with late recurrence.

Table 2 shows the results of risk factor analysis using binary logistic regression and variable selection using machine learning, which involved 20 preoperative variables. In the multivariable analysis, underlying chronic pancreatitis (OR, 2.918; 95% CI, 1.565–5.442; P < 0.001) and preoperative CA 19-9 (OR, 2.257; 95% CI, 1.465–3.478; P < 0.001) were identified as factors increasing the risk of early recurrence.

Table 2. Binary logistic regression analysis and variable selection using machine learning for early recurrence (with 20 preoperative variables).

OR, odds ratio; CI, confidence interval; ASA, American Society of Anesthesiology; PS, physical status.

^a)Presented as mean ± standard deviation.

Variable selection was conducted using the recursive feature elimination with cross-validation (RFECV) method, and ultimately, 14 input variables were selected for the analysis. The accuracy, demonstrating the predictive capability of the variables, exceeded 0.8 for all selected variables. The Kappa statistic validated this accuracy by adjusting for chance agreement. Each input variable was ranked based on its relative importance, and their scaled importance was illustrated in Fig. 1; CA 19-9 (1.000), underlying chronic pancreatitis (0.670), serum albumin level (0.506), platelet count (0.374), lymphocyte count (0.331), American Society of Anesthesiology physical status classification (0.327), preoperatively estimated tumor size (0.227), monocyte count (0.161), age (0.084), body mass index (0.081), CRP (0.075), hemoglobin level (0.064), WBC count (0.053), and CEA (0.023).

Fig. 1. A scale of variable importance from a machine learningbased variable selection (n = 14). ASA, American Society of Anesthesiology; PS, physical status.

Utilizing the selected 14 variables, an AI risk calculator was developed (Fig. 2). The ROC curves for both the training and test sets were presented (Fig. 3), with the AUCs of 0.786 and 0.734, respectively. The calculator is accessible online at “http://panc-recur.mdbcdss.com.”

Fig. 2. The newly developed artificial intelligence-based risk calculator (http://panc-recur.mdbcdss.com).

Fig. 3. The receiver operating characteristic curves of the newly developed platform from (A) the training and (B) the test set data. AUC, area under the curve.

DISCUSSION

Since surgical resection remains the only curative treatment for PDAC to date, surgery is generally considered first for resectable tumors. However, the criteria for ‘resectability’ vary among different study groups, and some aspects, such as vascular involvement and tumor markers, remain highly equivocal [15]. Furthermore, even patients with RPC experience cancer recurrence after curative surgery. In the present study, we limited our participants to those with RPC only, and 42.4% presented with early recurrence. In these instances, it is plausible to consider that the tumor might have been more advanced or a systemic disease, rather than a true RPC. Given that NAT has emerged as the new treatment standard for BRPC, it may also be an option for some RPC cases where early recurrence and treatment failure are concerns.

In a meta-analysis reviewing primarily retrospective studies that compared NAT and upfront surgery for RPC, the author investigated pathologic outcomes and survival rates between these patient groups [16]. The results indicated that NAT increased the R0 resection rate while reducing the pathological T and N stages. There were some recent randomized controlled studies including 38 to 93 RPC patients, all of whom received neoadjuvant gemcitabine [17]. The authors argued that NAT constitutes an acceptable treatment option for patients with clearly resectable cancer and suggested that the outcomes of ongoing trials with neoadjuvant FOLFIRINOX warrant attention. However, a significant concern remains: the potential for local or distant cancer progression during NAT, which could compromise patients’ survival by forgoing timely surgery [7]. Above all, no study has yet identified proper criteria for selecting RPC patients who may benefit from NAT. Therefore, this study aimed to predict the likelihood of failure following upfront surgery in the preoperative phase and to propose potential indications for NAT in RPC.

The variables included in our newly developed model are primarily derived from preoperative laboratory data. In addition to CA 19-9 and CEA, which are established prognostic factors for pancreato-biliary malignancies [18,19], the model incorporates CBC with differentials such as hemoglobin, WBC, lymphocyte, monocyte, and platelet counts. This aligns with several prior studies that explored the role of inflammatory markers as prognosticators in resected pancreatic cancer. A recent meta-analysis evaluated prognostic factors linked to early recurrence following curative surgery of pancreatic cancer, identifying preoperative hemoglobin, lymphocyte, neutrophilto-lymphocyte ratio, and platelet-to-lymphocyte ratio (PLR) as statistically significant risk factors for recurrence within 12 months [20]. Notably, platelets and lymphocytes, which constitute the PLR, ranked fourth and fifth respectively in terms of variable importance. The exact mechanisms by which PLR influences the prognosis of pancreatic cancer remain unknown. Some authors have proposed that several growth factors released by platelets may promote tumorigenesis and angiogenesis [21]. Others believe that platelet aggregation around tumor cells may facilitate immune evasion of the tumor [22]. On the other hand, lymphocytes are essential for anticancer activity via induction of cytotoxic cell death [23]. Consequently, an increase in platelets coupled with a decrease in lymphocyte count could indicate poor prognosis. Further research is necessary to explore how these biomarkers might inform treatment strategies and reveal potential changes in the tumor microenvironment in RPC.

In terms of serum albumin, it is recognized as a prominent nutritional marker and known for its anti-inflammatory and antioxidant effects [24]. Not only in pancreatic cancer, but low serum albumin levels have also been associated with cancer-related mortality in breast, lung, and colorectal cancers [25]. In addition to albumin, CRP was ranked as the eleventh most important variable in our model. Numerous studies have reported that albumin, CRP, and the CRP-to-albumin ratio can all serve as independent prognostic factors for the overall survival of pancreatic cancer patients [26,27,28]. However, further research is required, given the limited data on the optimal cutoff levels of each variable and the clinical implications of albumin replacement or nutritional support in pancreatic cancer patients. Preoperative nutritional status may be the only modifiable factor before surgery. Current and ongoing studies are investigating the clinical and oncological effects of preoperative nutritional support and rehabilitation in pancreatic cancer and other gastrointestinal malignancies [29,30]. If extensive physical and nutritional preparation during NAT improves patient outcomes, NAT could be more widely recommended for those at high risk of early recurrence after upfront surgery. Also, our institution is planning a future study using other markers like prealbumin and transferrin, which are potential prognosticators of immune and nutritional status in patients with pancreatic cancer.

Several previous studies have introduced risk prediction models for the early recurrence of resected pancreatic cancer using preoperative factors. Shimagaki et al. [4] enrolled 153 PDAC patients who underwent surgery and developed a predictive scoring system for recurrence within a year. Their model incorporates three preoperative parameters: PLR, CA 19-9, and tumor size, achieving an AUC of 0.757. In another recent study, the authors developed a nomogram, and it is notable that they integrated deep learning models using preoperative radiological findings [13]. This study highlighted the potential of utilizing entire preoperative images, rather than just measurable features such as tumor size and its relations with vessels, to predict the prognosis of PDAC. All authors from the aforementioned studies posited that predicting early recurrence could guide treatment strategies, particularly the use of NAT in PDAC patients. The major difference between the prior studies and the present study is that the aforementioned studies included not only RPCs, but also BRPCs with vascular involvement and even metastatic PDAC. Given that NAT is now established as the standard for BRPC, our study holds a greater clinical relevance by aiming to refine the indication for NAT in RPC patients.

The present study has several limitations. First, as a multicenter retrospective study, it introduced several biases. Heterogeneity in preoperative evaluation, patient selection, surgical timing, and postoperative surveillance existed due to the involvement of numerous physicians in the treatment process of the patients. Second, patients who were lost to follow-up after surgery were excluded, leading to potential selection bias, as data on recurrence were unavailable. Third, regarding methodology, we did not use direct regularization or full cross-validation across model training to reduce overfitting risk. Instead, we performed RFECV to limit the number of features and reduce model complexity. This approach served to mitigate the risk of overfitting by selecting only the most predictive variables. Also, the Kappa statistic was used to assess performance reliability beyond chance-level agreement, offering a more robust evaluation metric than accuracy alone. Lastly, external validation should be performed in the future to verify the predictability and applicability of this platform. Although the current platform demonstrates potential to stratify patients by the likelihood of early recurrence, we acknowledge that its direct application as a criterion for NAT remains premature. The role of NAT in RPC is still being actively explored in ongoing retrospective and prospective studies. Nevertheless, early recurrence may be considered a proxy for biologically aggressive disease, and this model may serve as an adjunctive tool to help guide multidisciplinary treatment planning. Future validation studies, including a prospective cohort designed to assess the clinical impact of NAT in patients predicted to be at high risk of early recurrence, are currently being planned.

Despite these limitations, this study remains highly inspiring as it developed a new platform with fair predictive value utilizing preoperative parameters easily accessible during routine evaluation for PDAC. AI-based analysis was employed to address the potential limitations of conventional statistics and to improve the model’s predictive capabilities. Additionally, this platform can express the risk of early recurrence numerically, providing a more tangible measure compared to other guidelines. Moreover, a user-friendly webpage for the risk calculator was established. In conclusion, our newly developed AI-based risk calculator for predicting early recurrence of RPC could be valuable in estimating prognosis and selecting patients who might benefit from NAT in the preoperative phase. Further research is required to confirm the accuracy and clinical practicality of the model.