Prediction of Pathological Complete Response after Chemoradiation for Locally Advanced Rectal Cancer Using Machine Learning with Clinical Features, the INTERCEPTOR Study Preliminary Model

Abstract

Background:

Organ-preserving strategies are increasingly used in rectal cancer for those achieving a pathological complete response (pCR) following chemoradiotherapy (CRT). In locally advanced rectal cancer (LARC), CRT may not significantly impact local recurrence or overall survival when an R0 resection is feasible. Accurate pretreatment prediction of pCR is critical to assist patients in deciding on CRT when it may not be oncologically required. The initial phase of the ongoing INTERCEPTOR study evaluated whether machine learning (ML) models using pretreatment clinical variables could improve pCR prediction beyond the baseline probability of 15-20%.

Methods:

Patients with LARC who received CRT followed by total mesorectal excision between 2004 and 2018 at a tertiary referral center were analyzed. Eligible patients received ≥25 fractions of CRT (50.0-50.4 Gy with concomitant capecitabine) and surgery ≥6 weeks after CRT. Extreme gradient boosting (XGBoost) models were trained with 5-fold cross-validation. Model performance was assessed using AUROC, sensitivity, and specificity. Feature importance was assessed with Shapley additive explanations (SHAP) and XGBoost feature importance.

Results:

Among 238 included patients, 30 (12.6%) achieved pCR. The number of radiologically positive lymph nodes was the strongest single predictor, but with limited discriminative power (AUROC 0.65 ± 0.04, sensitivity 0.83 ± 0.15 and specificity 0.45 ± 0.1). Combining positive node count with additional clinical variables led to a modest improvement in performance. SHAP analysis confirmed positive lymph node count was the most influential predictor.

Conclusion:

Pretreatment clinical variables alone provide poor-to-fair accuracy for predicting pCR after CRT in LARC.

Introduction

In rectal cancer patients, organ-preserving strategies are increasingly adopted for those achieving a clinical complete response (cCR), following neoadjuvant treatment[1-5]. In patients with locally advanced rectal cancer (LARC), chemoradiotherapy (CRT) is widely used as neoadjuvant approach[6,7]. However, recent evidence suggests that when a R0 resection is feasible, CRT does not significantly reduce local recurrence rates or improve overall survival for this patient cohort[8]. Consequently, in resectable LARC patients where CRT is not oncologically required, the rationale for administering CRT is the potential to achieve pathological complete response (pCR), thereby enabling organ-preserving management such as watch-and-wait. Despite the growing interest in organ-preserving approaches, only 15-20% of LARC patients achieve pCR after CRT[3,5]. This modest baseline likelihood highlights the need for a reliable pretreatment prediction of pCR, where patients can weigh the potential benefit of pCR and organ-preservation against morbidity of CRT. The ability to predict pCR prior to treatment start would therefore represent a shift in clinical management. Accurate pretreatment risk stratification would help identify resectable LARC patients who are most likely to benefit from neoadjuvant CRT and facilitate personalized treatment strategies. To address this challenge we apply machine learning (ML), a predictive analytic approach that can outperform traditional modeling techniques in predicting clinical outcomes. Compared with linear models (e.g. logistic regression) ML offers enhanced modeling capacity and can incorporate non-linear relationships between multiple variables[9-11]. ML has been successfully applied to predict pCR using post-treatment clinical features, such as radiomics-based space-time models that integrate post-treatment MRI and clinical features. However, research applying ML to pretreatment factors is still at an early stage and literature on this topic is scarce[12-14].

The initial phase of the INTERCEPTOR study aims to explore whether a ML model based on routinely available pretreatment clinical variables can improve pCR prediction in LARC patients undergoing CRT, thereby identifying those most likely to benefit beyond the baseline probability of 15-20%.

Methods

Patient selection

Patients with LARC who underwent neoadjuvant CRT followed by total mesorectal excision (TME) with curative intent were identified from a prospectively maintained institutional registry. All TME resections were performed at Erasmus MC Cancer Institute, a tertiary referral center, between January 2004 and December 2018.

Diagnosis of LARC and clinical staging was based on thoracoabdominal (PET-)CT and/or pelvic MRI, confirmed by diagnostic biopsy and staged according to the TNM classification available at time of inclusion. LARC was defined according to the Dutch national guideline as cT4 tumors, or cT3 tumors with <1 mm mesorectal fascia margin, cN2 disease, extramural vascular invasion, or extra-mesorectal lymph node involvement[15]. Eligible patients received long-course CRT (50.0-50.4 Gy in 25-28 fractions with concomitant capecitabine as radiosensitizer) and underwent surgical resection ≥6 weeks after completion of CRT (according to the Dutch guideline for colorectal cancer)[15]. Patient and treatment characteristics were all collected and recorded, as well as pathological outcomes. pCR was defined as ypT0N0M0, corresponding to Mandard tumour regression grade 1[16]. According to the 2003 Dutch colorectal cancer guideline, radiologically suspicious lymph nodes were defined as those with a diameter > 3 mm on MRI. The 2008 guideline increased the threshold to ≥ 5 mm for a positive lymph node. The 2014 guideline introduced additional morphological criteria (round shape, irregular borders, heterogeneous texture) for identifying a positive lymph node. Mesorectal lymph nodes were classified as positive if the diameter was < 5 mm with all three morphological features, 5-9 mm with at least two features, or ≥ 9 mm regardless of morphology.

Patients were excluded if pretreatment biopsies or clinical data were missing, if fewer than 25 fractions of radiotherapy were administered, if capecitabine was omitted, if patient received induction chemotherapy prior to CRT, or if surgery was performed <6 weeks after finishing CRT. Patients were classified into two groups: pCR and non-pCR.

Statistical analysis

Data were presented as count with percentages or median with interquartile ranges (IQR). Categorical data of non-pCR versus pCR patients were compared using the Chi Square test. Continuous data were analyzed using the Mann Whitney U test. A p-value of <0.05 was considered statistically significant. All statistical analyses were performed in SPSS (IBM SPSS Statistics for Windows, Version 31.0., USA).

Modeling

Extreme gradient boosting (XGBoost) is a decision-tree based method, which uses the average output of multiple trees, known as an ensemble, to generate more accurate predictions[17]. Using this model, the top 10 features based on feature importance were extracted. In the feature search stage, handcrafted feature combinations were used to identify the most predictive subset of features. Five-fold-cross-validation was used to evaluate the model's performance. Missing continuous variables were imputed using the median, while cases with missing values in other variables were excluded from the analysis, resulting in the inclusion of 218 out of 238 patients. After identifying the highest-performing multivariate model, its predictive performance was compared with that of the best univariate model, which had demonstrated statistical significance. All analyses were conducted in Python (version 3.12), with XGBoost models implemented using Scikit-learn. Model performance estimates were derived exclusively from internal 5-fold cross-validation within the same dataset, without the use of an independent test cohort or external validation. Consequently, reported performance metrics may overestimate generalizability.

Outcome

The primary outcome was the prediction of pCR (ypT0N0M0) by the machine learning model. Model performance was evaluated using the area under the receiver operating characteristic curve (AUROC) as the principal metric. The AUROC ranges from 0.5 (random classification) to 1 (perfect classification) and summarizes the ability to distinguish positive from negative cases. In addition, sensitivity, specificity, and balanced accuracy were calculated to provide performance measures. Model interpretability was assessed using Shapley additive explanations, which quantify the contribution of individual variables to model predictions[18]. Shapley additive explanation values were used to quantify the contribution of each variable to model predictions.

Results

Included patients and univariate analysis

A total of 238 patients with LARC who underwent neoadjuvant CRT followed by surgery were included, resulting in 208 non-pCR and 30 pCR cases (Table 1). Most patients in both groups had cT3-T4 tumors and node-positive disease. The median number of radiologically positive nodes was higher in the non-pCR group compared with the pCR group (5 [IQR 2-8] vs. 3 [IQR 1-5], p = 0.007). No significant baseline differences were observed between the two groups were observed in clinical T-stage, clinical node-positivity and clinical M-stage, presence of fistula on pretreatment imaging, ASA score, preoperative serum CEA level, or tumour distance to anorectal junction. The cohort of 238 patients was used for the univariate analysis and overall prediction. However, for the multivariate prediction, 20 patients were excluded due to missing values that could not be imputed. Consequently, 218 patients were included in the XGBoost-based multivariate prediction analysis.

Table 1.

Baseline Characteristics Non-pCR and pCR Patients and Univariate Predictors of pCR.

	level	Missing in %	Non-pCR	pCR	p-value
N			208	30
Age (median [IQR])		0	63 (16)	59 (23)	0.348
Sex (%)	Male	0	130 (63)	16 (53)	0.423
cT (%)		0			0.433
	cT1		0	0
	cT2		2 (1)	0
	cT3		100 (48)	18 (60)
	cT4		106 (51)	12 (40)
cN (%)		0			0.248
	cN0		43 (21)	9 (30)
	cN+		165 (79)	21 (70)
cM (%)		0			0.725
	cM0		191 (92)	27 (90)
	cM1		17 (8)	3 (10)
Fistula (%)	Yes	0	21 (10)	5 (17)	0.281
Number positive nodes on radiology(median [IQR])		2	5 (2-8)	3 (1-5)	0.007
Distance to anorectal junction (%)		1			0.580
	0 - 5.0 cm		99 (48)	14 (47)
	>5.0 - 10.0 cm		78 (38)	14 (47)
	>10.0 - 15.0 cm		22 (11)	2 (6)
	>15.0 cm		7 (3)	0
Preoperative CEA-level (median [IQR])		24	6.5 (3.0-17.7)	5.9 (3.0-18.4)	0.785
ASA-score		0			0.258
	ASA 1		64 (33)	8 (28)
	ASA 2		98 (51)	19 (66)
	ASA 3		31 (16)	2 (7)
Weeks interval between end CRT and surgery (median [IQR])		3	11 (10-14)	12 (10-14)	0.620
Type of surgery		0			0.037
	LAR		79 (38)	10 (33)
	APR		80 (39)	16 (53)
	Posterior exenteration		31 (15)	3 (10)
	Total pelvic exenteration		18 (9)	1 (3)
pT (%)		0
	pT1		5 (2)	-
	pT2		31 (15)	-
	pT3		123 (59)	-
	pT4		49 (24)	-
pN (%)		0
	pN0		138 (66)	-
	pN+		70 (34)	-

Because the number of positive lymph nodes was statistically significant in univariate analysis, its predictive performance as a single-variable model was further assessed. In cross-validated XGBoost models, the number of positive lymph nodes alone yielded a mean AUROC of 0.65 ± 0.04 (Figure X). Although sensitivity was relatively high (0.83 ± 0.15), specificity (0.45 ± 0.1) was low, underscoring its limited discriminative capacity when used in isolation.

Figure X.

Cross-validated predictive performance of positive lymph node count (ROC curve and confusion matrices), where class 0 represents non-pCR cases and class 1 represents pCR cases.

Model prediction

All features were ranked according to their importance in the XGBoost model (Figure A). The top-ranked features were manually combined, guided by their expected clinical relevance (Table 2).

Figure A.

Relative importance of top variables for prediction of pathologic complete response (all features) by XGBoost.

Table 2.

Predictive Performance of Handcrafted Clinical Feature Combinations for Predicting pCR.

Case	Selected features	AUC	Sensitivity	Specificity	Balanced ACC
1	Positive nodes number, Weight, cN stage	0.74±0.11	0.500 ± 0.279	0.835 ± 0.032	0.667 ± 0.130
2	Positive nodes number, weight	0.72±0.09	0.400 ± 0.226	0.798 ± 0.051	0.599 ± 0.089
3	Positive nodes number, distance to anorectal junction	0.62±0.06	0.567 ± 0.082	0.676 ± 0.054	0.621 ± 0.030
4	Positive nodes number, Number nodes	0.65±0.08	0.533 ± 0.245	0.634 ± 0.103	0.583 ± 0.114
5	Positive nodes number, cT	0.65±0.05	0.833 ± 0.149	0.474 ± 0.104	0.653 ± 0.057
6	cN stage	0.63±0.11	0.800 ± 0.194	0.549 ± 0.109	0.674 ± 0.096

Combination 1: Positive nodes number with weight and cN stage; Combination 2: Positive nodes number with weight; Combination 3: Positive nodes number with distance to anorectal junction; Combination 4: Positive nodes number number nodes; Combination 5; Positive nodes number with cT; Combination 6: Positive nodes number with cT, and cN stage. Performance metrics are reported as mean ± standard deviation across cross - validated XGBoost models.

Given the statistical significance of the number of positive lymph nodes in the univariate analysis, the performance was evaluated in combination with additional clinically relevant features. Handcrafted feature sets were constructed by combining the number of positive lymph nodes with variables such as cT, clinical node stage, BMI, and weight. As summarized in Table 2, these combinations led to modest performance improvements over the single-variable model. The strongest results were observed when the number of positive lymph nodes was combined with weight and cN stage (Combination 1), achieving an AUROC of 0.74 ± 0.11 and a balanced accuracy of 0.67 ± 0.13. Detailed visualization for Combination 1 is provided in Supplementary Figures Z, respectively.

The ten most influential variables in the Shapley additive explanation values are presented in Supplementary Figure X.

Discussion

In this study, a XGBoost ML model was trained on pretreatment clinical factors to predict pCR after CRT in patients with LARC. The findings add to existing literature demonstrating the potential of ML to extract clinically meaningful patterns from routinely collected clinical factors. In this study, the number of radiological positive lymph nodes was identified as the strongest predictor for pCR, with higher count of radiological positive lymph nodes in non-pCR patients; however, the discriminative ability of this factor was limited (AUROC 0.65). Combining this factor with other additional clinical features slightly improved the model (AUROC 0.74). This study underscores that pretreatment clinical factors are associated with response, but their predictive value remains limited, reaching only poor-to-fair discrimination, insufficient for reliable prediction of pCR for LARC patients.

Although CRT is often applied for LARC, its benefits are unevenly distributed. Only 15-20% of patients achieve a pCR, while 6-11% are exposed to significant, acute as well as late, treatment-related toxicity without oncological gain[19,20]. These disadvantages highlight the critical importance of reliable pretreatment prediction of pCR, as it is essential for patients to know their individual likelihood of achieving a pCR. Such knowledge could empower shared decision-making and enable organ-preserving strategies, such as watch-and-wait, in those most likely to benefit.

The findings in this study are in line with previous studies which reported clinical node negativity, tumour size and well-differentiated tumours as independent predictors for pCR[14]. In another study similarly nodal negativity as well as longer intervals to surgery were identified as predictive factors for pCR. However, the association between longer intervals between CRT and surgery was not observed in our cohort, likely reflecting the uniformity of treatment intervals and the exclusion of patients with surgery <6 weeks post-CRT in our study[21]. Huh et al. reported younger age and lower preoperative serum CEA level as predictors for pCR, but this association could not be validated, partly due to the substantial proportion of missing CEA data in our registry[22].

Although weight and cN stage, did not reach statistical significance as a predictor of pCR, when added to our model with the number of radiological positive lymph nodes, its performance improved slightly (AUROC 0.74). However, the evidence in literature regarding weight is inconsistent. Fisher et al reported that lower weight demonstrated the highest probability of achieving a pCR. In contrast, Bedrikovetski et al described that BMI ≥25 was associated with a increased likelihood of pCR, findings that are line with our results[23,24]. Fisher et al also described lower cN stage was associated with a higher probability of pCR, but from the SHAP values in this study, it could not be concluded if a lower cN stage was favorable for pCR.

A summary of studies describing clinical factors for predicting pCR in rectal cancer is provided in Supplementary Table 1. When comparing our results to prior ML-based studies predicting pCR, important methodological differences occur. Many previous models incorporated heterogeneous rectal cancer populations, variable CRT regimens, or wide ranges of surgery intervals, limiting comparability. In contrast, the INTERCEPTOR study focuses on a uniformly treated, well-defined LARC cohort with standardized CRT only regimen and consistent surgical timing (≥6 weeks), thereby reducing treatment-related variability and providing a cleaner outcome for evaluating an independent contribution of baseline clinical factors.

Several groups have reported encouraging results in rectal cancer using “pathomics”, a field that applies ML and deep learning (DL) techniques to digital pathology slides for predicting pCR in pretreatment rectal biopsies. Lou et al developed a ResNet-18-based model achieving an AUC of 0.72 in predicting CRT response from pretreatment biopsies[25]. Wu et al employed a ResNet-101 architecture and reported high patch- and slide-level performance (AUC 0.98), although patient-level validation was limited[26]. More recently, Zhang et al integrated pathological whole slide features into machine-learning classifiers and demonstrated prediction of both pCR and prognosis in two different cohorts, with external validation AUCs above 0.90[27]. Complementing these pathology-based efforts, Feng et al demonstrated that a multimodal “radiopathomics” model combining MRI and pathology slides achieved superior results (AUC 0.87) compared to either modality alone[28]. Collectively, these studies highlight both the promise of pathology-based ML in rectal cancer and the added value of multimodal integration.

Comparable findings have been reported in other malignancies. In breast cancer, ML-based analyses of biopsy pathology slides have been shown to reliably predict response to neoadjuvant chemotherapy, with tumor-infiltrating lymphocytes emerging as a consistent marker of treatment response[29,30]. In gastric cancer, multimodal AI combining histology with CT and/or MRI has outperformed clinical predictors for predicting response to neoadjuvant therapy[31,32]. These cross-cancer findings strengthen the plausibility that pretreatment tissue morphology, when studied by AI, contains predictive cues of therapy responsiveness.

This study has several limitations. First, external validation was not conducted in this study, limiting the generalizability of the findings. This limitation primarily reflects constraints related to data availability rather than legal or ethical restrictions. Moreover, external validation in the literature remains scarce. Only a minority of rectal cancer studies, including Zhang et al and Feng et al have validated their models across independent cohorts, and performance estimates were consistently lower than in internal testing[27,28]. This underscores the importance of multi-institutional, prospective validation for improving generalizability. Second, there is a persistent issue of class imbalance in most existing studies. Reported pCR rates in LARC cohorts range between 15-20%, with pCR as the minority class[1,3]. Small sample sizes further exacerbate this problem by increasing the risk of model overfitting. The cohort in this study reflects these limitations, and ongoing efforts to expand data through national registries and other available studies will be critical for building balanced and representative training datasets. Additionally, data on lateral lymph nodes and their dissection were not available in this dataset. Since radiologically positive lymph nodes emerged as the most important predictor of pCR in this model, incorporating data on lateral lymph nodes could provide valuable insights. This aspect will be considered for inclusion in the next phase of the study. Finally, while our study focused on clinical features alone, an increasing number of studies suggest that multimodal AI approaches may deliver more accurate prediction models. Studies in rectal, gastric and breast cancer consistently showed that combining radiological with pathological features enhances predictive performance[28,30-32]. Integrating clinical features with histopathology, radiology and perhaps genomic and/or liquid biopsy data represents a plausible next step in precision oncology for the prediction of pCR.

The INTERCEPTOR study is an ongoing project, aimed at developing a multimodal prediction model for organ preservation in LARC patients. This first phase of the INTERCEPTOR study was to develop a baseline ML model using only standard clinical variables to establish a performance benchmark. The next step in the INTERCEPTOR study will be the integration of pathomics through the inclusion of pretreatment biopsy slides to capture morphological and tumour microenvironmental features predictive of treatment response. In parallel, ongoing work focuses on circulating tumor DNA (ctDNA) as a liquid biomarker for predicting response to CRT[33]. Combining clinical, pathological, and molecular features, including ctDNA, represents a promising multimodal strategy that could substantially improve response prediction in CRT.

In conclusion, pretreatment clinical variables alone provide only poor-to-fair discriminatory ability for pretreatment prediction of pCR after CRT in LARC. While our findings highlight the potential of clinical-based ML models to guide personalized treatment strategies, further development of the model with the addition of pretreatment biopsies, radiological features, and prospective external validation remain essential before clinical implementation can be realized for enabling organ preservation in likely pCR patients.

Clinical trial registration

Not applicable.

Ethics approval statement

The study was not subject to the Medical Research Involving Human Subjects Act.

Paper based on previous communication statement

This paper is not based on a previous communication to a society or meeting.

Conflict of interest statement

All the aforementioned authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Patient consent statement

Written informed consent for participation was waived for this study due to the retrospective nature of this study.

Data availability statement

All data related to the research is available upon reasonable request.

Author contributions:

Study concepts: DM, SS, FA, AS, MD, CV, DH, JT

Study design: DM, SS, FA, AS, MD, CV, DH, JT

Data acquisition: DM, CV, DH, JT

Quality control of data and algorithms: DM, SS, FA, AS, MD, CV, DH, JT

Data analysis and interpretation: DM, SS, FA, AS, MD, CV, DH, JT

Statistical analysis: SS, FA, AS

Manuscript preparation: DM, SS

Manuscript editing: DM, SS

Manuscript review: FA, AS, MD, CV, DH, JT

Supplementary Files

Supplementary file 1.

Supplementary file 2.

Supplementary file 3.

Supplementary file 4.

Funding Statement

This research received no specific grant or funding from any public, commercial or not-for-profit institution. The manuscript has not been or will be in a podium or poster meeting presentation.