MRI radiomics-based machine learning model for complete response classification after chemoradiotherapy in advanced rectal cancer

Abstract

Objective

Accurate identification of complete response (CR) after neoadjuvant chemoradiotherapy (NCRT) is essential for selecting candidates for watch-and-wait treatment in advanced rectal cancer. This study aimed to develop and evaluate a magnetic resonance imaging (MRI) radiomics-based machine learning model to classify CR and Non-CR in post-NCRT rectal MRI images.

Methods

Using region-of-interest masks, 107 radiomic features were extracted and normalized to a 0–1 range using min-max scaling. Four feature selection methods (ANOVA, RFE, SBS, and LASSO) were paired with four classifiers (LR, SVM, RF, and XGB), and all combinations were evaluated through 5-fold cross-validation in the training set using mean ROC AUC as the comparison metric.

Results

Among 16 combinations, RFE + SVM achieved the highest cross-validation AUC of 0.89. On an independent test set, the model achieved a sensitivity of 0.84, specificity of 0.80, accuracy of 0.82, and F1-score of 0.82. The most informative features were intensity and local texture features, including TotalEnergy, 10Percentile, Coarseness, Strength, Correlation, ZoneEntropy, and Idmn.

Conclusions

MRI radiomics combined with machine learning shows preliminary promise as an exploratory approach for classifying CR and Non-CR after NCRT in rectal cancer. Given the retrospective, single-center design and the absence of external validation, these findings should be interpreted as preliminary, and prospective multicenter validation is required before clinical implementation. A multicenter external validation integrating endoscopy and digital rectal examination findings is planned to enhance generalizability and clinical utility.

I. Introduction

Radical resection is the standard treatment for rectal cancer. However, postoperative complications such as sexual dysfunction, urinary dysfunction, and low anterior resection syndrome can significantly impair the patients’ quality of life. In patients with rectal cancer who achieve a clinical complete response (cCR) following neoadjuvant chemoradiotherapy (NCRT), a nonoperative management approach with active surveillance has demonstrated survival outcomes comparable to those with a pathological complete response (pCR) confirmed following surgery. Notably, this strategy avoids unnecessary stoma formation and surgical complications, thereby introducing the concept of a “Watch and Wait” approach. ¹

To implement such a “Watch and Wait” strategy, post-treatment assessment in rectal cancer patients is of critical significance. When a cCR is determined through digital rectal examination (DRE), colonoscopy, and magnetic resonance imaging (MRI), nonoperative organ-preserving management has demonstrated favorable efficacy and safety. Therefore, combined use of these diagnostic modalities is essential for appropriate patient selection. ²

However, the accuracy of cCR determination after NCRT remains uncertain. Several studies have reported cases in which patients assessed as achieving cCR did not achieve pCR. When using only T2-weighted imaging (T2WI), the reported sensitivity ranges from 33% to 65%, while the specificity ranges from 74% to 93%.^3–6 Even with the addition of diffusion-weighted imaging (DWI) to T2WI, the reported sensitivity remains at 43–77%, with specificity ranging from 80% to 94%.^7–10

In addition, for pCR prediction based on mrTRG, when TRG1 is considered as pCR, the reported sensitivity is 18–51% and the specificity is 92–95%. When both TRG1 and TRG2 are considered as pCR, the sensitivity increases to 60–78%, while the specificity decreases to 56–68%.^11,12 Therefore, the implementation of a watch-and-wait approach requires precise assessment and careful decision-making to appropriately select cCR patients, and it is of primary importance to improve the accuracy of cCR identification using rectal MRI.

Radiomics is a technique that quantifies subtle texture, shape, and intensity patterns in medical images and features that are often imperceptible to the human eye and converts them into quantitative metrics. When combined with machine learning, it is regarded as a promising tool for predicting treatment response and prognosis.^13–15

Applying radiomics to rectal MRI enables the objective quantification of tumor and peritumoral heterogeneity, microstructural features, and morphological changes. This approach reduces the reliance solely on subjective clinical assessments and may contribute to improving the accuracy of cCR identification.^16,17

Several prior studies have investigated MRI-based radiomics for treatment response prediction and prognosis in rectal cancer. Shin et al. developed and validated a large-scale post-NCRT MRI radiomics model in 898 patients with locally advanced rectal cancer using T2-weighted and ADC images, achieving an AUC of 0.82 with superior sensitivity compared to experienced radiologists, demonstrating that quantitative radiomic features can complement conventional radiological assessment of pathological complete response. ¹⁸ Coppola et al. provided a comprehensive review of radiomics applications in rectal cancer MRI, highlighting its expanding role in staging, treatment response assessment, and outcome prediction beyond conventional imaging, while emphasizing the importance of feature standardization and robust validation frameworks. ¹⁹

Fu et al. further demonstrated in a multicenter cohort of 600 patients that MRI radiomics signatures derived from T2-weighted and diffusion-weighted images could significantly stratify recurrence risk in rectal cancer patients with negative pretreatment CEA, achieving a C-index of up to 0.822 in the external test cohort, underscoring the broad prognostic utility of MRI-based radiomics in rectal cancer management. ²⁰

Beyond rectal adenocarcinoma, radiomics-based approaches have also been explored in the broader anorectal region. Vanzulli et al. recently demonstrated that pretreatment T2-weighted MRI radiomics features, particularly gray-level co-occurrence matrix features, could predict chemoradiotherapy response in anal canal squamous cell carcinoma, achieving an AUC of 0.867. Although the tumor histology and anatomical location differ from rectal adenocarcinoma, these findings suggest that quantitative MRI-based radiomic analysis may have broader applicability across anorectal malignancies treated with chemoradiotherapy, providing additional methodological context for the present study.

This study aimed to develop and validate a classification model that distinguishes complete response (CR) from Non-complete response (Non-CR) by extracting radiomic features from MRI scans of patients with locally advanced rectal cancer who underwent NCRT. Furthermore, by identifying and summarizing the key features utilized by the optimal model, we aim to provide objective evidence to assist in the selection of patients with cCR, thereby enabling a more accurate application of the watch-and-wait strategy and ultimately improving the quality of life through organ-preserving management. In addition, this study has several features that may differentiate it from prior approaches. First, whereas previous studies often included more heterogeneous rectal cancer populations, our study focused on patients with low- and mid-rectal cancer, excluding upper rectal cancer, which may allow evaluation within a more clinically homogeneous subgroup. Second, by applying a machine learning-based radiomics approach to MRI analysis, our study explored the potential of integrating multiple quantitative imaging features in a more objective manner. Therefore, rather than claiming a definitive improvement over previous studies, our findings suggest the potential clinical usefulness of this approach for response classification and for supporting the identification of candidates for organ-preserving strategies such as watch-and-wait.

2. Materials and methods

Figure 1 provides an overview of the workflow used in this study. After labeling the cases as CR or Non-CR, 107 radiomics features were extracted from the ROIs (Figure 1a and b). The extracted features were subsequently used to train the models by combining four feature selection methods, ANOVA, RFE, SBS, and LASSO, with four classifiers, LR, SVM, RF, and XGB (Figure 1c). All combinations were evaluated through cross-validation using the same procedure, and the performance was assessed in terms of sensitivity, specificity, accuracy, F1 score, and AUC. The best-performing combination was identified, and imaging biomarkers were derived based on this model (Figure 1d). This retrospective observational prediction-model study was reported in accordance with the STROBE guideline and the TRIPOD+AI guideline. In addition, the radiomics workflow was described with reference to radiomics-specific reporting principles to improve transparency and reproducibility.

Figure 1.

Study flowchart for radiomics-based classification of rectal MRI images. (a) Data labeling: CR (n=47) and Non-CR (n=176) groups with representative MRI images. (b) Radiomic feature extraction from ROIs, yielding 107 quantitative features across seven categories (First Order, GLCM, GLRLM, NGTDM, GLSZM, GLDM, and Shape). (c) Model training: 16 combinations of four feature selection methods (ANOVA, RFE, SBS, LASSO) and four classifiers (LR, SVM, RF, XGB) evaluated through 5-fold cross-validation. (d) Model validation: performance assessed using sensitivity, specificity, accuracy, and F1-score, with selection of the optimal model.

2.1. Dataset

This was a single-center retrospective observational study conducted at Gachon University Gil Medical Center from July 1, 2025, to December 31, 2025, using MRI images of patients diagnosed with rectal cancer between December 1, 2012, and July 31, 2025. The study was designed to evaluate treatment response at the post-NCRT clinical decision point using MRI-based analysis. Accordingly, the study population was defined in the context of response assessment after neoadjuvant treatment rather than subsequent management alone. The inclusion criteria were patients who underwent radical surgery following preoperative NCRT, or did not receive radical surgery after NCRT but maintained a complete response through radiologic, endoscopic, and physical examination without evidence of recurrence for at least five years (Figure 2). The exclusion criteria were patients who did not undergo preoperative NCRT and those without MRI scans before or after NCRT, and those whose MRI examinations were performed using 1.5T equipment.^21,22 Although patients could subsequently follow different clinical pathways after response assessment, all included cases were evaluated within the same institutional framework for post-treatment response evaluation. This study aimed to reflect the real-world clinical setting in which treatment response after NCRT may be linked to different downstream management strategies. CR was defined by pathological verification as pathologic complete response and no recurrence with five-year long-term follow-up after NCRT without surgery, which was identified in 43 and 4 patients, respectively. Non-CR was defined as the presence of residual tumor or recurrence observed during follow-up. ²³ In this study, complete response was defined in accordance with the available reference standard within each clinical pathway. In surgically treated patients, CR corresponded to pathological complete response, whereas in patients without residual or recurrent disease during long-term follow-up, CR reflected a sustained complete response status based on clinical and radiological assessment. This definition was adopted to capture clinically meaningful complete response across the post-NCRT setting, although potential heterogeneity in outcome definition is acknowledged. A total of 223 3.0T MRI scans were included in the study, comprising 47 CR cases and 176 Non-CR cases. Clinical information was obtained from electronic medical records (EMR), and imaging data were collected from the hospital picture archiving and communication system (PACS). Tumor ROIs were manually delineated on axial T2-weighted MRI images slice-by-slice using the AVIEW software (Coreline Soft, Republic of Korea) by a board-certified radiologist with more than 10 years of clinical experience, and subsequently reviewed and validated by a board-certified colorectal surgeon. Radiomic features were extracted from the annotated tumor masks within predefined ROI slice groups rather than from the entire unpartitioned tumor volume. For the Non-CR group, three representative axial slices corresponding to the superior, middle, and inferior portions of the tumor were selected for each patient and treated as a single predefined ROI group, resulting in one ROI per patient and 176 ROIs from 176 Non-CR patients. For the Non-CR group, one ROI per patient was extracted, yielding 176 ROIs from 176 patients. For the CR group, multiple ROI groups were generated because of the limited number of CR patients. Specifically, three consecutive axial slices at different levels along the tumor extent were grouped together and treated as one ROI, resulting in 241 ROIs from 47 CR patients. Although multiple ROIs were generated for some CR patients, the radiomic feature vectors extracted from all ROIs belonging to the same patient were averaged feature-wise to generate a single patient-level feature vector before five-fold cross-validation. Therefore, each patient contributed only one feature vector to subsequent feature selection, model training, cross-validation, and hyperparameter tuning. The dataset was partitioned at the patient level, and all ROIs originating from the same patient were assigned exclusively to the same fold. Feature selection, feature-importance estimation, feature scaling, class-weight calculation, and hyperparameter tuning were performed only within the training subset of each fold. The corresponding held-out validation fold was used exclusively for performance evaluation and did not contribute to any model-development.

Figure 2.

Rectal MRI examples: (a) Non-CR, (b) CR. Each item shows left: ROI-marked image, right: zoomed crop of the ROI.

All stages of model development and evaluation were conducted at the patient level. ROI-level radiomic features extracted from multiple ROIs belonging to the same patient were first aggregated into a single patient-level feature vector prior to any data splitting. The dataset was then partitioned at the patient level using stratified sampling into a training set and an independent test set. Cross-validation and hyperparameter tuning were performed exclusively within the training set, and feature selection was applied within each cross-validation fold using only the training fold data. The independent test set was used only once after final model selection to evaluate model performance. This procedure ensured that no patient-level data leakage occurred across training, validation, or test partitions at any stage of the analysis.

All data were de-identified by assigning study identification numbers, and the study was conducted after obtaining approval from the Institutional Review Board (IRB No. GDIRB2025-234). To ensure objectivity in ROI definition and to minimize potential bias, tumor regions were initially delineated by a board-certified radiologist with more than 10 years of clinical experience, and subsequently reviewed and validated by a board-certified colorectal surgeon.

2.2. Research environment

All analyses were performed on Windows 11 Pro. The system specifications included an Intel® Core™ i7-6700 CPU (Intel, Santa Clara, CA), 32 GB RAM, and an NVIDIA GeForce GTX 1060 6 GB GPU (NVIDIA, Santa Clara, CA). Radiomic feature extraction was performed using PyRadiomics (version 3.1.1), an open-source Python-based package. Various imaging features were calculated using functions such as RadiomicsGLCM, RadiomicsGLDM, RadiomicsGLRLM, RadiomicsFirstOrder, RadiomicsGLSZM, and RadiomicsNGTDM. ²⁴ For classifier training and feature selection algorithm implementation, the machine learning library scikit-learn (version 1.6.1) was used. ²⁵ OpenCV-Python (version 4.12.0) was used for bitwise operations in image loading, storage, and preprocessing.

2.3. Preprocessing

To preserve texture information as much as possible, the original resolution and pixel spacing of the images were maintained, and no resizing, resampling, or data augmentation was applied. ²⁶

First, to minimize the contrast differences between sequences, the window level and width were standardized to align the image contrast. Subsequently, the 107 extracted radiomic features were normalized to a 0–1 range using min–max scaling, thereby preventing variables with large numerical magnitudes (e.g., TotalEnergy) from dominating the learning process over texture-related features with values close to zero. ²⁷ Intensity normalization was performed in a two-step manner. Specifically, window levels were first standardized to reduce inter-sequence variability. Afterward, following an outlier assessment, min–max scaling was applied to the 107 extracted radiomic features to improve numerical stability and to prevent features with larger scales from exerting excessive influence on model weight optimization during machine learning training, resulting in all feature values being rescaled to the 0–1 range. For all feature selection methods (ANOVA, RFE, SBS, LASSO) and classifiers (LR, SVM, RF, XGB), identical settings were uniformly applied to ensure fair comparison.

2.4. Radiomic feature extraction

In this study, radiomic analysis was performed on the lesion regions of post-NCRT T2-weighted MRI images, resulting in the extraction of 107 quantitative features.

A total of 18 first-order statistics summarized the overall pixel intensity distribution within the lesion, including not only the mean, median, and standard deviation, but also higher-order statistical properties such as skewness, kurtosis, and energy. Sixteen gray-level size zone matrix (GLSZM) features were used to quantify the structural complexity and tissue heterogeneity based on the size and number of zones connected by the same gray level. Five neighborhood gray-tone difference matrix (NGTDM) features characterize coarseness, contrast, and busyness by measuring the intensity differences between a center pixel and its neighbors. Sixteen gray-level run-length matrix (GLRLM) features capture repetitive patterns and linear structures by calculating the run length and frequency of consecutive pixels with the same intensity. Fourteen gray-level dependence matrix (GLDM) features were used to quantify the homogeneity and dependency patterns by assessing the extent to which neighboring pixels shared similar intensities. Fourteen shape features provided three-dimensional geometric descriptors of the tumor, including volume, surface area, sphericity, and diameter. Finally, 24 gray-level co-occurrence matrix (GLCM) features were used to quantify the local texture characteristics, including contrast, homogeneity, and correlation, based on the joint occurrence of adjacent pixel intensities. All features were computed under identical parameter settings to maintain consistency across the dataset.^15,28–30 Radiomics features were extracted using PyRadiomics under identical extraction settings for all cases. The images were analyzed in their native resolution without voxel resampling, and gray-level discretization was performed using a fixed bin width of 25. After feature extraction, all radiomic features were normalized to a range of 0–1 using min–max scaling for subsequent model development. This normalization step was applied only after feature extraction for feature scaling and did not affect the original image intensities used for radiomics calculation. For each patient, radiomic features extracted from all available ROI slices were aggregated to generate a single representative feature vector, and the final prediction unit was the individual patient, consistent across training, cross-validation, and independent test set evaluation. Formal reproducibility or stability testing of radiomic features was not performed in the present study and is acknowledged as a limitation.

2.5. Feature selection and machine learning model

In machine learning models, an excessive number of input variables can increase computational complexity and paradoxically reduce performance owing to redundant or irrelevant information. In radiomics analysis, in which dozens to hundreds of features are extracted simultaneously, it is essential to evaluate the predictive contribution of each feature and identify only the most relevant ones. This process improves model accuracy, enhances computational efficiency, and reduces the risk of overfitting. ³¹

Feature selection can be broadly categorized into three approaches. The first is the filter method, in which features are evaluated individually during preprocessing, independent of the learning process, based on statistical or information-theoretic criteria, and subsequently preliminarily filtered. Representative techniques include ANOVA (analysis of variance), χ² test, mutual information, and correlation coefficient-based elimination. The second is the wrapper method, which directly tests different feature subsets by iteratively training classifiers to identify combinations that improve the performance. ³² Although computationally expensive, this approach better reflects the compatibility with the model. Representative methods include recursive feature elimination (RFE), sequential forward selection (SFS), and sequential backward selection (SBS). ³³ In the present study, RFE was implemented using logistic regression (LogisticRegression ()) as the internal estimator. Feature importance was determined from the absolute values of the logistic regression coefficients. Starting from 107 radiomic features, RFE retained 10 features and eliminated 10 features at each iteration until 10 features remained. The embedded method performs feature selection within the training process itself by utilizing regularization or weights to automatically filter variables. This category includes techniques such as LASSO (L1 regularization), elastic net, and tree-based feature importance methods (e.g., SelectFromModel). ³⁴ In this study, four representative methods were selected from the three categories of feature selection: ANOVA, RFE, SBS, and LASSO. For classification models, we employed logistic regression (LR), support vector machine (SVM), random forest (RF), and XGBoost (XGB).^35–38 LR provides results that can be intuitively interpreted as probabilities. SVM identifies a decision boundary that maximizes the margin between two groups, enabling robust classification even in high-dimensional feature spaces. RF ensembles multiple decision trees, allowing the flexible modeling of variable interactions and nonlinearities while mitigating overfitting. XGB applies a boosting strategy that sequentially corrects residuals with greater weight in misclassified cases, thereby enhancing performance even in complex decision boundaries.^39,40

All machine learning analyses were performed using 5-fold stratified cross-validation at the patient level. In each iteration, the dataset was split into 80% training data and 20% validation data, while preserving the proportion of CR and Non-CR cases. Within each cross-validation fold, feature selection and hyperparameter tuning were performed independently using only the training subset of that fold. Grid-search hyperparameter tuning was performed within the training subset, ⁴¹ and the final model configuration was selected based on the mean validation performance across the five folds. To address the imbalanced class distribution between CR and Non-CR cases, class weighting was applied during model training. ⁴² To prevent information leakage, feature selection and hyperparameter tuning were performed strictly within the training subset of each fold, and the validation subset was never used during these processes. The detailed hyperparameter settings for the machine learning models are summarized in Table 1.

Table 1.

Hyperparameter settings for the machine learning models.

Model	Hyperparameters
LR	C: 0.001, penalty: L2, random state: 50, solver: liblinear
SVM	C: 1, kernel: RBF, gamma: 100, probability: True, random state: 50
RF	max depth: 4, min samples split: 25, n estimators: 600, random state: 50
XGB	Learning rate: 0.01, max depth: 4, n estimators: 200, random state: 50

Because the cohort was imbalanced (47 CR and 176 Non-CR cases), class imbalance was addressed using algorithm-level weighting rather than resampling-based methods. For LR, SVM, and RF, class weights were calculated within the training subset of each fold using the balanced heuristic, defined as n_samples/(n_classes × n_c). Based on the overall cohort distribution, this corresponds approximately to class weights of 2.37 for CR and 0.63 for Non-CR. For XGB, imbalance was handled using a weighting parameter based on the ratio of Non-CR to CR cases; based on the overall cohort, this value corresponds to approximately 3.74. However, to avoid information leakage, these weights were not fixed using the full dataset and were instead recalculated separately from the training subset within each cross-validation fold. The held-out validation subset was never used to determine class weights, scaling parameters, feature selection, or hyperparameter tuning. No over-sampling, under-sampling, or synthetic minority over-sampling methods such as SMOTE were applied, as we aimed to preserve the original patient-level radiomic feature distribution without introducing duplicated or synthetic samples.

2.6. Statistical analysis

Model predictions were compared with patient-level reference labels determined by surgical pathology or long-term clinical and radiological follow-up. Based on this comparison, true positives (TP), false negatives (FN), true negatives (TN), and false positives (FP) were obtained, from which sensitivity, specificity, accuracy, and F1-score were calculated. CR was defined as the positive class and Non-CR as the negative class. Accordingly, sensitivity was calculated for the CR class, and the F1-score was calculated as a binary F1-score with CR as the positive class; macro-averaged and weighted F1-scores were not used. To assess performance across different decision thresholds, receiver operating characteristic (ROC) curves and the area under the curve (AUC) were computed. All analyses were performed at the patient level for each of the 16 combinations of four feature-selection methods ANOVA, RFE, SBS, and LASSO and four classifiers LR, RF, SVM, and XGB. The entire cohort, comprising 47 CR patients and 176 Non-CR patients, was evaluated using patient-level stratified five-fold cross-validation. In each fold, patients were divided into a training fold and a held-out validation fold while preserving the proportion of CR and Non-CR patients. Feature selection, feature-importance estimation, feature scaling, class-weight calculation, hyperparameter tuning, and model training were performed exclusively within the training fold. The corresponding held-out validation fold was used only for performance evaluation. The optimal combination identified through cross-validation was subsequently evaluated on an independent test set, where all performance metrics were reported with 95% confidence intervals.

3. Results

Table 2 presents the characteristics of the data collected in this study. A total of 223 patients were included: 47 in the CR group and 176 in the Non-CR group. The overall mean age was 63.06 ± 10.8 years, with group-specific means of 64.6 ± 11.7 years for Non-CR and 62.7 ± 10.5 years for CR. There were 155 males (69.6%) and 68 females (30.4%), with males predominating both the CR and Non-CR groups. In total, 417 ROIs were analyzed, including 176 Non-CR ROIs and 241 CR ROIs.

Table 2.

Baseline characteristics and group-wise distribution of the study cohort; SD, standard deviation.

Variables	All	Non-CR	CR
Number of patients	223	176	47
Number of ROI	417	176	241
Age, years, mean ± SD	63.06 ± 10.8	64.6 ± 11.72	62.66 ± 10.54
Sex
Male	155(69.6%)	117 (66.7%)	38(80.9%)
Female	68(30.4%)	59 (33.3%)	9 (19.1%)

The results of the mean AUC comparison for each combination obtained through 5-fold cross-validation are shown in Figure 3. Among the 16 combinations of the four feature selection methods and four classifiers, those based on RFE exhibited superior performance, with RFE + SVM achieving the highest AUC of 0.89. In contrast, ANOVA-based combinations tended to perform poorly overall, with ANOVA + RF yielding the lowest AUC (0.81) among all 16 combinations.

Figure 3.

Heatmap of mean AUC values from 5-fold cross-validation across 16 combinations of four feature selection methods (ANOVA, RFE, SBS, LASSO) and four classifiers (LR, SVM, RF, XGB). Darker colors indicate higher AUC values. RFE + SVM achieved the highest mean AUC of 0.89.

Table 3 presents the performance metrics of the RFE method, which demonstrated higher performance than the other feature selection methods in combination with each machine learning model. The sensitivity, which reflects the ability to correctly identify CR cases and avoid false-negative CR classifications, was highest for RF at 0.85, followed by SVM and LR at 0.84. Specificity, an indicator of reducing unnecessary positive classifications, was the highest for SVM at 0.80, followed by LR and XGB at 0.78, and RF at 0.71. Accuracy, representing the proportion of correct predictions across all cases, was 0.82 for both SVM and LR, 0.80 for XGB, and 0.78 for RF. The F1-score, which combines precision (the proportion of correctly predicted CR cases among all cases predicted as CR) and recall (the proportion of actual CR cases correctly identified), was highest for SVM and LR at 0.82, followed by XGB at 0.80 and RF at 0.79.

Table 3.

Sensitivity, specificity, accuracy, and F1-score for LR, RF, SVM, and XGB; values in parentheses indicate the 95% confidence interval (CI).

​	Sensitivity (95% CI)	Specificity (95% CI)	Accuracy (95% CI)	F1-score (95% CI)	AUC (95% CI)
LR	0.84 (0.73-0.95)	0.78 (0.68-0.88)	0.82 (0.75-0.87)	0.82 (0.75-0.88)	0.88 (0.82-0.94)
RF	0.85 (0.71-0.99)	0.71 (0.59-0.83)	0.78 (0.71-0.85)	0.79 (0.72-0.87)	0.89 (0.83-0.95)
SVM	0.84 (0.73-0.94)	0.80 (0.72-0.89)	0.82 (0.76-0.88)	0.82 (0.76-0.88)	0.86 (0.80-0.92)
XGB	0.82 (0.71-0.93)	0.78 (0.72-0.83)	0.80 (0.73-0.87)	0.80 (0.72-0.88)	0.87 (0.81-0.93)

Figure 4 illustrates the mean feature importance values obtained from the RFE feature-selection method in combination with each machine-learning model. TotalEnergy accounted for the highest contribution, followed in order by 10Percentile, Coarseness and Strength from the NGTDM family, Correlation from GLCM, Skewness, ZoneEntropy, Idmn, LargeDependenceHighGrayLevelEmphasis, and Kurtosis. By feature category, four belonged to the first-order statistics: two to NGTDM, two to GLCM, one to GLSZM, and one to GLDM. This distribution indicated that intensity-based features (energy, percentiles, skewness, and kurtosis), neighborhood gray-tone differences (NGTDM), and co-occurrence matrix–based local texture features (GLCM) were predominant. Overall, TotalEnergy had the largest weight, whereas the importance of the other features gradually decreased.

Figure 4.

Top-10 radiomic features ranked by mean importance value, averaged across four feature selection methods (ANOVA, RFE, SBS, LASSO) in combination with four classifiers. TotalEnergy showed the highest mean importance, followed by 10Percentile and ngtdm_Coarseness.

Figure 5 shows the distribution of the feature importance for the RFE feature selection method across each machine learning model. Among the four classifiers, TotalEnergy contributes the most. The intensity-based lower percentile feature (10Percentile) and roughness index (coarseness) also demonstrated consistently high weights. In particular, 10Percentile (0.092) and coarseness (0.067) were ranked among the top features in the SVM. Strength, which reflects texture regularity, was relatively more prominent in the tree-based models. Among the GLCM features, correlation exhibited low but consistent importance across all models, whereas Idmn was meaningful only in the tree-based models. In contrast, Kurtosis and LargeDependenceHighGrayLevelEmphasis were close to zero or negative, indicating a negligible impact. Overall, regardless of the model type, global intensity-based statistics formed the core discriminative axis, with tree-based models additionally emphasizing the NGTDM-derived texture features.

Figure 5.

Heatmap of model-specific feature importance for the top ten radiomic features across four classifiers (LR, SVM, RF, XGB). Darker colors indicate higher importance values. TotalEnergy consistently showed the highest importance across all classifiers.

4. Discussion

In this study, to quantitatively classify CR and Non-CR on post-NCRT rectal MRI, a total of 107 radiomic features were extracted and analyzed in combination with four feature selection methods (ANOVA, RFE, SBS, and LASSO) and four classifiers (LR, SVM, RF, and XGB). The extracted features were normalized to a range of 0–1 for training, and the data were split at the patient level into training and test sets. Model comparisons were conducted using 5-fold cross-validation of the training set, with the mean AUC as the evaluation metric. The highest classification performance was achieved using the RFE + SVM combination with an AUC of 0.89, whereas the ANOVA-based combinations generally performed worse, with ANOVA + RF showing the lowest score. The superior performance of RFE is likely attributable to its ability to eliminate redundant or irrelevant variables and retain only the most informative features, whereas SVM demonstrated robustness in distinguishing cases with ambiguous boundaries. In contrast, LR showed lower performance owing to its limited capacity to capture complex patterns, and RF and XGB yielded moderate results consistent with the ensemble models, although the performance varied depending on the combined feature selection method.

An analysis of the features that were truly informative revealed that radiomic features reflecting intensity distribution and local textural differences consistently ranked among the most important. TotalEnergy represents an aggregate intensity-related measure within the analyzed lesion. However, its value may be influenced not only by the intensity distribution but also by the number of voxels, voxel volume, lesion volume, and native voxel spacing. Therefore, the observed difference in TotalEnergy between CR and non-CR cases should not be interpreted as direct evidence of tumor homogeneity, fibrosis, or any specific pathological state. Rather, it should be regarded as a cohort-specific statistical association that may reflect a composite effect of image intensity and lesion geometry. 10Percentile reflects the distribution of low-signal regions and may capture the persistence of dark components in Non-CR patients, possibly related to residual tumor or necrotic tissue. Texture-based features such as Coarseness and Strength represent the granularity of patterns and the distinctness of textural structures, respectively. The observed texture patterns may be compatible with differences in post-treatment tissue organization between CR and non-CR cases. However, the biological mechanisms underlying these radiomic differences remain uncertain, and the features cannot independently distinguish fibrosis from residual tumor or necrotic tissue. Correlation measures the synchrony of intensity changes between neighboring pixels, while ZoneEntropy reflects the irregularity of uniform-intensity clusters — potentially lower in CR owing to more homogeneous signal distribution and higher in Non-CR possibly due to heterogeneous tumor components. Finally, Idmn quantifies local homogeneity and may serve as a descriptor of the relatively uniform fibrotic tissue potentially observed in CR.

Collectively, these radiomic features may capture differences in post-treatment signal distribution and spatial organization between CR and non-CR cases. However, these associations should not be interpreted as direct evidence of fibrosis, scar tissue organization, residual tumor, or necrosis, because direct histopathological correlation was not performed in the present study. Thus, they numerically complement conventional image interpretation in cases where it is clinically ambiguous whether a finding represents scar tissue or residual tumor.

From a clinical perspective, the present model should be regarded as an exploratory tool for quantitative MRI-based assessment rather than a clinically ready decision-support system. Given that complete response assessment in rectal cancer relies on combined evaluation including digital rectal examination, endoscopy, and MRI, the proposed radiomics-based approach is intended to complement rather than replace existing multimodal assessment strategies, and further prospective validation is required before clinical implementation. In particular, when posttreatment MRI alone provides inconclusive findings incorporating radiomics-derived probabilities or quantitative scores into multidisciplinary discussions may potentially contribute to decision-making on whether to pursue a watch-and-wait strategy or proceed directly to surgery, though this application remains exploratory and requires prospective validation before clinical use. Radiomic features may quantify subtle pathological changes that are not easily discernible to the human eye, potentially offering complementary information to support clinical discussions, though direct evidence of clinical benefit has not been established in the present study. Although recent ensemble-based deep learning models, such as NEDL-GCP, have demonstrated promising performance in risk prediction tasks, explainable radiomics-based approaches retain important advantages in settings where the availability of clinical data is limited. In this study, to mitigate the limitations of relying on a single model, we explored multiple combinations of feature selection methods and classifiers. This strategy aligns with the emerging paradigm in multi-view classification research, which emphasizes the integration of complementary information from diverse perspectives to enhance robustness and interpretability.

In this study, the entire process, from preprocessing to feature extraction, model training, and evaluation, was conducted within a unified pipeline with patient-level separation to prevent overlap between the training and validation data. The combination that achieved the best performance in internal comparisons was selected as the final model, and its performance was confirmed on an independent test set by calculating the sensitivity, specificity, accuracy, and F1-score. In summary, applying radiomics to posttreatment rectal MRI provided useful information for distinguishing CR from Non-CR, with the RFE + SVM combination achieving the best performance, showing a mean 5-fold cross-validation AUC of 0.89 in the training set. Future work should expand the dataset to include external and multi-institutional cohorts to assess generalizability. Integrating MRI radiomics with endoscopic findings and digital rectal examination results may enable the development of a clinically applicable model for patient management.

To address the limitation of using data from a single institution and to enhance generalizability, a detailed protocol for future multicenter validation has been established. In subsequent studies, we plan to develop a multimodal nomogram that integrates the MRI-derived radiomics score (Rad-score) with endoscopic tumor regression grade (TRG) findings and DRE results. Beyond conventional performance metrics, model validation will extend to the assessment of calibration using calibration plots and the evaluation of clinical utility through decision curve analysis (DCA), thereby quantitatively estimating the net clinical benefit and strengthening the model’s potential for real-world clinical application.

This study has several limitations that should be acknowledged. First, this was a retrospective single-center study, which may limit the generalizability of the findings to other institutions and patient populations. Second, the sample size was relatively limited, with only 47 CR cases compared to 176 Non-CR cases, resulting in a substantial class imbalance that may affect model stability and the robustness of the reported performance. Although class weighting and patient-level stratified sampling were applied to mitigate this issue, the limited number of CR cases remains a constraint. Third, external validation using an independent multicenter cohort was not performed in the present study, and the generalizability of the proposed model therefore requires confirmation in future studies. In addition, formal calibration analysis and decision curve analysis were not performed in the present study. Although the model demonstrated promising discriminatory performance, further evaluation of calibration and clinical utility is necessary to determine whether the predicted probabilities are reliable and whether the model provides meaningful net clinical benefit in real-world decision-making. These analyses should be incorporated in future studies, ideally in the context of external validation using independent cohorts. Fourth, formal inter-observer variability assessment for ROI segmentation was not conducted, as all ROIs were delineated by a single experienced radiologist and subsequently reviewed by a board-certified colorectal surgeon. The absence of interobserver reproducibility testing is a recognized limitation in radiomics studies, as manual segmentation is known to introduce variability that may affect feature stability and model generalizability. Although the dual-review process was employed to minimize segmentation bias, the extent to which radiomic features remain stable across different delineators was not formally quantified in the present study. This limitation may affect the reproducibility of the reported findings and should be considered when interpreting the clinical applicability of the proposed model. Future studies incorporating interobserver analysis, perturbation-based stability testing, or automated and semi-automated segmentation methods would help improve the robustness and reproducibility of radiomic feature extraction. TotalEnergy may be affected by native voxel spacing, voxel volume, the number of voxels, and lesion size in addition to the underlying intensity distribution. In the present study, the images were analyzed at their native resolution without voxel resampling. Therefore, potential effects of differences in voxel spacing and tumor volume on TotalEnergy cannot be excluded. Although min–max scaling was applied after feature extraction for model development, this procedure was intended for numerical feature scaling and did not eliminate the potential dependence of TotalEnergy on the original voxel geometry or lesion volume. Future studies should evaluate the stability of TotalEnergy after voxel-spacing standardization and adjustment for lesion volume.

5. Conclusion

This study demonstrated that MRI radiomics-based machine learning is a promising approach for classifying complete response and Non-complete response after neoadjuvant chemoradiotherapy in locally advanced rectal cancer. The RFE + SVM combination achieved the best performance, and intensity- and texture-based radiomic features were identified as the most informative predictors. These findings suggest that quantitative radiomics analysis of post-treatment MRI has exploratory potential to complement conventional image interpretation; however, given the retrospective single-center design and the absence of external validation, the role of this approach in supporting watch-and-wait decision-making remains to be established. Further prospective and multicenter validation is warranted to confirm generalizability and clinical utility.

ORCID iD

Kwang Gi Kim https://orcid.org/0000-0001-9714-6038

Ethical considerations

This study was approved by the Institutional Review Board of Gachon University Gil Medical Center (IRB No. GDIRB2025-234) on August 26, 2025.

Data Availability Statement

The data used to support the findings of this study are available upon request from the corresponding authors.*