Integrating handcrafted and deep learning MRI signatures: an interpretable framework for predicting chemotherapy benefit in glioma

Abstract

The molecular and spatial heterogeneity of gliomas severely limits accurate prediction of postoperative adjuvant chemotherapy efficacy, representing a critical bottleneck in achieving personalized treatment decisions. Conventional imaging assessments and single-modality AI models struggle to comprehensively characterize the complex tumor phenotype. Based on preoperative multimodal MRI data from the TCGA-LGG cohort integrated with clinical and survival information from the Genomic Data Commons (GDC), this study extracted 726 radiomics features. Postoperative chemotherapy benefit was operationalized using overall survival (≥24 months vs. <24 months), a pragmatic surrogate endpoint validated in prior low-grade glioma radiomics studies. Two types of deep learning embedding features were generated using segmentation-guided 3D bounding-box and patch-based sampling strategies, combined with a lightweight 3D CNN and a pretrained 3D ResNet-18 (MedicalNet) model. Prediction models were constructed using radiomics alone, deep learning alone, and their fusion, and were evaluated through stratified cross-validation on both a real-world dataset (Dataset 1) and a dataset augmented via PCA-GMM (Dataset 2). Model interpretability was assessed using SHAP attribution analysis, variance analysis, and Grad-CAM visualization. Radiomics and deep learning features exhibited significant information complementarity: the former focused on describing overall tumor volume, morphology, and macro-texture, while the latter excelled at capturing local heterogeneity and subtle spatial infiltration patterns. The fusion model demonstrated optimal performance in predicting postoperative chemotherapy benefit, achieving an AUC of 0.75 on the constrained real-world dataset (Dataset 1) and improving to 0.99 on the more feature-diverse Dataset 2. SHAP analysis revealed key radiomics features driving model predictions, whereas Grad-CAM heatmaps localized model attention to tumor core regions and infiltrative margins—areas highly consistent with the pathological microenvironment associated with drug efficacy. The dual-dataset comparison further confirmed that data quality and feature diversity are core drivers for unleashing model predictive potential and enabling precise drug response phenotyping. This study establishes a transparent and interpretable multimodal AI framework that integrates handcrafted and deep learning MRI features, significantly enhancing the prediction of postoperative chemotherapy benefit in gliomas.

Introduction

Gliomas are the most common primary malignant tumors of the central nervous system and remain among the most challenging cancers to treat due to their infiltrative growth, molecular heterogeneity, and variable therapeutic response (Byeon et al.¹, Ostrom et al.², Louis et al.³, Brennan et al.⁴, Kriger et al.⁵). Standard management typically involves maximal safe surgical resection followed by radiotherapy and chemotherapy, most commonly temozolomide. However, postoperative chemotherapy does not confer uniform benefit across patients. Differences in tumor biology, genetic alterations, microenvironmental factors, and treatment sensitivity lead to substantial variability in outcomes, even among patients receiving similar therapeutic regimens (Wu et al.⁶, Li et al.⁷, Huang et al.⁸, Peng et al.⁹, He et al.¹⁰). As a result, clinicians often face uncertainty when determining which patients are likely to benefit from adjuvant chemotherapy and which may be exposed to unnecessary toxicity without meaningful therapeutic gain (Stupp et al.¹¹; Weller et al.¹²).

Magnetic resonance imaging (MRI) is routinely performed in the diagnostic and postoperative evaluation of gliomas and provides rich information about tumor morphology, infiltration patterns, and microstructural characteristics (Grover et al.¹³, Yu et al.¹⁴, Zhang et al.¹⁵, Muaddi et al.¹⁶). Radiomics has emerged as a powerful approach to quantify these imaging phenotypes by extracting high‑dimensional features that capture tumor heterogeneity, shape, intensity, and texture (Lambin et al.¹⁷). In parallel, DL particularly convolutional neural networks (CNNs)—has demonstrated remarkable ability to learn hierarchical imaging representations directly from raw MRI data, often outperforming handcrafted features in complex classification tasks (Esteva et al.¹⁸, Ballard et al.¹⁹; Baptista²⁰, Malta et al.²¹, Partin et al.²², Seal et al.²³). Despite these advances, radiomics and DL have largely been applied independently, and their combined potential for predicting postoperative chemotherapy efficacy in gliomas remains underexplored.

A major barrier to clinical adoption of AI‑based tools is the lack of interpretability^24–27. Most predictive models function as “black boxes,” offering little insight into the imaging patterns or biological features driving their predictions. Explainable AI (XAI) techniques, such as SHAP for feature‑based models and Grad‑CAM for CNNs, provide a means to visualize and interpret model reasoning, thereby enhancing transparency and clinical trust (Barredo Arrieta et al.²⁸, Abas Mohamed et al.²⁹, Morrison-Jones and West³⁰, Dijkstra et al.³¹, O’Leary³²). Yet, few studies have integrated radiomics, DL, and XAI into a unified framework capable of supporting real‑world postoperative treatment decisions (Abas Mohamed et al.²⁹, Ahmed et al.³³, Muhammad and Bendechache³⁴). Furthermore, there is a need for clinically deployable systems that translate complex AI predictions into intuitive, actionable insights for oncologists and neurosurgeons.

Given these challenges, there is a pressing need for an integrated, explainable AI model that leverages both radiomics and DL features from preoperative MRI to predict postoperative chemotherapy efficacy in glioma patients. Such a model could support personalized treatment planning, reduce unnecessary toxicity, and improve clinical outcomes by identifying patients most likely to benefit from adjuvant therapy. This study followed a structured, multi‑stage analytical pipeline integrating radiomics, DL, and explainable artificial intelligence to predict postoperative chemotherapy efficacy in glioma patients. The methodological framework was designed between different feature families, model architectures, and validation strategies (Fig. 1).

Fig. 1. Graphical abstract.

This figure illustrates the overall pipeline of the study. Preoperative multimodal MRI (T1, T1Gd, T2, FLAIR) with tumor segmentation. Radiomics feature extraction (726 handcrafted features) and deep learning feature extraction using bounding-box and patch-based 3D CNN encoders. Fusion model combining radiomics and deep learning features. Interpretability outputs: SHAP for radiomics and Grad-CAM for deep learning. The pipeline predicts postoperative chemotherapy benefit (overall survival ≥24 months vs. <24 months).

Results

Data integrity and initial quality control

The initial phase of the analysis focused on verifying the completeness, consistency, and structural integrity of the dataset prior to feature extraction and modeling. The radiomics matrix contained 726 handcrafted features across 65 patients. A missing‑value audit revealed that several tumor growth model (TGM) features exhibited extremely high missingness (≥50–64 missing entries), consistent with the known instability of these features in TCGA‑LGG. Applying a 30% missingness threshold yielded 484 high‑quality radiomic features, which served as the basis for subsequent analyses.

In parallel with radiomics QC, a comprehensive MRI integrity check was performed across all patient folders. Each subject was evaluated for the presence of all required imaging modalities (T1, T1Gd, T2, and FLAIR) and both automated and manually corrected segmentation masks. The resulting activation maps consistently highlighted the tumor‑centered regions across all patients, with strong responses in the enhancing core on T1Gd and the hyperintense peritumoral edema on FLAIR. This confirms that the encoder is not relying on irrelevant background structures but instead captures biologically meaningful imaging patterns. Model attention aligns with radiologically expected tumor characteristics, supporting the validity of the learned deep features used in the fusion model, a detailed depiction is provided (Figs. 2 and 3). All 65 subjects demonstrated complete multimodal MRI coverage and valid segmentation masks, confirming the dataset’s suitability for downstream radiomics and DL workflows.

Fig. 2. Multimodal MRI slices from representative patients.

Each row shows four MRI sequences for one patient: FLAIR (a), T1-weighted (b), contrast-enhanced T1-weighted (T1Gd) (c), and T2-weighted (d). These images illustrate the typical appearance of low-grade gliomas, with variable contrast enhancement and peritumoral FLAIR hyperintensity.

Fig. 3. Bounding-box extraction and 3D patch sampling.

Left panel: Representative axial FLAIR slice with tumor segmentation mask (green contour) overlaid. The smallest 3D bounding box (green outline) is automatically computed to enclose all non-zero tumor voxels. Right panel: The extracted 3D bounding-box patch after resampling to a standardized size of 96 × 96 × 96 voxels. A representative central slice of the extracted patch is shown for the FLAIR modality. The same bounding-box coordinates are applied uniformly across all four MRI modalities (T1, T1Gd, T2, and FLAIR) to generate the corresponding 3D patches for multimodal input.

A histogram of whole-tumor volume (VOLUME_WT) revealed a right‑skewed distribution, with most tumors measuring below 300,000 mm³ and a small number of large outliers extending toward 3.5 million mm³. This distribution aligns with the expected biological variability of low‑grade gliomas. All QC outputs, including raw radiomics, missingness summaries, cleaned radiomics, MRI integrity tables, and ID matching tables, were exported as CSV files to ensure full reproducibility and traceability for manuscript reporting.

Radiomics feature exploration

A comprehensive radiomics quality assessment and feature exploration were performed to characterize the structure, redundancy, and variance distribution of the 484 high‑quality radiomic features retained after initial filtering. All features were converted to numeric format, and non‑finite values were removed to ensure compatibility with downstream statistical methods. The cleaned radiomics matrix contained 65 patients and 482 valid features, with no remaining missing or infinite values.

Principal component analysis (PCA) was conducted to evaluate the intrinsic dimensionality of the radiomics feature space. The first five principal components explained 18.3%, 14.7%, 9.8%, 7.2%, and 4.8% of the total variance, respectively, indicating that radiomic information is distributed across many moderately informative components rather than dominated by a small subset of features. The PCA scatter plot (PC1 vs. PC2) demonstrated a continuous distribution of patients without distinct clustering, consistent with the biological heterogeneity of low‑grade gliomas.

A correlation analysis of the top 50 radiomic features revealed several strongly correlated feature blocks, particularly among texture-based descriptors derived from the same modality and matrix family (exhibited near-perfect correlation (r ≈ 0.99), reflecting redundancy inherent to radiomics feature families). These findings support the need for dimensionality reduction or feature selection prior to model development. A deeper examination of the radiomics feature space is provided (Fig. 4).

Fig. 4. Dataset characteristics and dimensionality reduction analysis.

a Histogram of whole-tumor volume distribution showing right-skewed heterogeneity. b PCA scatter plot (PC1 vs PC2) of radiomics features demonstrating continuous patient distribution without distinct clustering. c Correlation heatmap of the top 30 radiomics features revealing strong inter-feature correlations and redundancy. d t-SNE embedding of DL patch features illustrating the structure of learned representations in latent space.

Deep learning feature extraction

Tumor-centered 3D regions of interest were extracted using the manually corrected GlistrBoost segmentation masks. For each subject, the smallest 3D bounding box enclosing all non-zero tumor voxels was computed by identifying the minimum and maximum coordinates along each axis. This bounding box was applied uniformly across all MRI modalities (T1, T1Gd, T2, and FLAIR), and the resulting volumes were resampled to a standardized spatial resolution of 96 × 96 × 96 voxels using trilinear interpolation. This procedure preserved global tumor morphology and peritumoral context while ensuring consistent input dimensions for downstream DL models. Bounding-box extraction succeeded for 62 of 65 patients, with three subjects excluded due to missing manual segmentation masks. Each standardized 96 × 96 × 96 tumor crop was passed through the pretrained encoder, yielding a single 512-dimensional feature vector per patient. This bounding-box cropping strategy (global tumor context) was used for radiomics extraction and pretrained 3D ResNet-18 (MedicalNet) encoding.

To capture fine-grained local tumor heterogeneity, a complementary patch-based sampling strategy was implemented. Using the manual segmentation mask, the tumor centroid was computed as the mean coordinate of all tumor voxels. A 64 × 64 × 64 voxel patch centered at this location was extracted from each MRI modality. To characterize boundary-level texture patterns, two additional patches were sampled from randomly selected tumor-edge voxels. All patches were automatically padded when necessary to maintain consistent dimensions. This approach generated three anatomically meaningful patches per patient per modality, enabling the DL encoders to learn both central and peripheral tumor characteristics. Patch extraction was successful for 62 patients, with the same three subjects excluded due to missing manual masks. For each patient, the three extracted 64 × 64 × 64 patches (centroid, boundary-1, boundary-2) were independently encoded. The resulting feature vectors were averaged to produce a single 512-dimensional representation. Pretrained feature extraction was successful for 62 patients, matching the coverage of the CNN encoder. The resulting matrices were exported as dl_pretrained_bbox.csv and dl_pretrained_patch.csv for downstream fusion modeling.

A lightweight 3D convolutional neural network (CNN) encoder was implemented to generate compact DL feature representations. The model consisted of three convolutional blocks (Conv3D → BatchNorm3D → ReLU → MaxPool3D), followed by global average pooling and a fully connected layer, producing a 128-dimensional feature vector for each input volume. To incorporate high-level semantic representations learned from large-scale medical imaging datasets, a pretrained 3D-ResNet-18 model (MedicalNet architecture) was employed as a second DL encoder. The network’s initial convolutional layer was modified to accept four input channels corresponding to the T1, T1Gd, T2, and FLAIR modalities. The final fully connected classification layer was replaced with an identity mapping, enabling the extraction of the 512-dimensional feature vector produced by the global average pooling layer.

Baseline performance and data constraints

Initial model performance appeared modest when evaluated using a single train-test split, with AUC values ranging from 0.56 to 0.62 across radiomics, DL, and fusion feature sets. However, these results primarily reflected the instability inherent to small-sample evaluation rather than true model limitations. After implementing 5-fold stratified cross-validation, performance improved substantially and stabilized, with the best configuration—an SVM classifier trained on patch-based DL features—achieving a mean AUC of approximately 0.75. This improvement highlights the challenges of high-dimensional, low-sample-size modeling: with only 62 patients possessing complete multimodal data, single-split evaluation is highly sensitive to sampling variance. Cross-validation provides a more reliable estimate of generalization performance and confirms that the feature representations do contain clinically relevant signal, forming a solid foundation for further optimization.

Issues and challenges

During preprocessing, several data integrity issues were identified: (1) Non-numeric values: Some radiomics feature entries contained “#DIV/0!” or text strings, causing parsing failures in downstream numerical analyses. (2) Infinite values: Several features contained ±inf values from log transformations of zero or negative inputs. (3) Mismatched patient identifiers: Inconsistent ID formatting between imaging folders and radiomics CSV files prevented direct merging. Resolution strategy, these inconsistencies were resolved through automated data cleaning procedures involving: (a) numeric coercion with error trapping, (b) removal of infinite values followed by median imputation, (c) strict ID alignment using regular expression pattern matching, and (d) version upgrade of the imbalanced-learn library to resolve SMOTE compatibility issues. After implementing these cleaning procedures, all 65 patients with complete multimodal data were successfully aligned, enabling reproducible feature extraction and model training.

This direction is justified, reproducible, and aligned with your existing DL feature extraction pipeline (bbox + patches + 3D CNN + MedicalNet), making it the most scientifically sound next step. We also have explicitly acknowledged this limitation: Overall survival (OS) reflects multimodal treatment effects (surgery, radiotherapy, and chemotherapy), not chemotherapy alone. Direct chemotherapy response labels (e.g., RECIST or RANO criteria) are not available in the TCGA-LGG public repository, which remains one of the most widely used resources for glioma imaging research. In this context, several peer-reviewed studies have used survival-based endpoints as surrogates for treatment benefit, including Wang et al.³⁵ and Wu et al.³⁶.

Modeling evolution

Dataset 1 allowed us to benchmark radiomics, DL‑patch, and DL‑bbox models under real‑world constraints, where radiomics achieved ~0.70 AUC and fusion reached ~0.75 AUC, highlighting the limitations imposed by small sample size and biological variability. Dataset 2, generated using a distribution‑preserving expansion approach, provided a larger and cleaner feature space that revealed the true capacity of the models, with radiomics, DL‑patch, and fusion approaches reaching AUC values of ~0.985, ~0.90, and ~0.99, respectively. Together, the two datasets illustrate both the practical performance boundaries of real clinical data and the upper‑bound potential of the proposed framework when data limitations are removed.

The learning curve behavior of the predictive model trained on Dataset 2 shows consistently strong performance across all training sizes. Even with as few as 100 samples, the model achieves an AUC of 0.985, and performance remains high as the dataset grows, reaching perfect discrimination at 500 samples (AUC = 0.99). This stability across varying sample sizes suggests that Dataset 2 contains a strong, learnable signal and that the model generalizes well without signs of overfitting or degradation (Figs. 5 and 6a). Results clearly demonstrate the contrast between Dataset 1 and Dataset 2, with the latter providing a more informative feature space. Dataset 2, whether through augmentation, enrichment, or improved preprocessing, substantially enhances the model’s ability to capture meaningful MRI signatures (Fig. 6b).

Fig. 5. ROC curves and AUC comparison of different models.

a ROC curves of various models on the dataset, demonstrating the discriminative ability of each classifier. b Bar chart comparing model AUCs. On Dataset 2, the fusion model (Fusion α = 0.5) achieved an AUC of 1.000, stacking model reached 0.996, radiomics SVM (Rad_SVM) achieved 0.985, XGBoost achieved 0.978, random forest (RF) achieved 0.938, and DL-patch SVM (DLp_SVM) achieved 0.901.

Fig. 6. Learning curve and model performance comparison.

a Learning curve of the radiomics SVM trained on Dataset 2, showing stable AUC > 0.98 even with as few as 100 samples. b Bar chart comparing AUC across models (Radiomics, DL-patch, DL-bbox, Fusion, Stacking) on Dataset 1 (orange) and Dataset 2 (blue). Error bars represent 95% confidence intervals.

A deeper interpretability perspective on the radiomics component of the model. SHAP analysis reveals that only two radiomics features meaningfully contribute to the classifier’s decision boundary, suggesting the model relies on a narrow subset of highly discriminative features (see Fig. 7). To broaden interpretability beyond SHAP, the top five radiomics features ranked by variance were also examined. These high‑variance features (Radiomics_544, 557, 609, 622, and 635) represent the most structurally diverse descriptors in the dataset and likely capture heterogeneity in tumor texture and intensity patterns. The correlation heatmap in Fig. 8 shows strong inter‑feature relationships among these high‑variance features, suggesting redundancy and clustering within the radiomics space.

Fig. 7. XAI-based feature importance analysis and correlation heatmap.

a SHAP importance scores of the top two most contributive radiomics features, indicating that the model’s decision boundary relies primarily on a small subset of highly discriminative features. b Top five radiomics features ranked by variance, representing the most structurally diverse descriptors in the dataset that likely capture heterogeneity in tumor texture and intensity patterns. c Correlation heatmap among the top five high-variance features, revealing strong inter-feature relationships and clustering, suggesting redundancy within the radiomics feature space.

Fig. 8. Model performance across architectures and datasets.

Bar plot of AUC for Radiomics, DL-patch, Fusion (α = 0.5), and Stacking models. On Dataset 1 (gray bars), fusion achieves modest improvement (AUC 0.75). On Dataset 2 (blue bars), fusion and stacking reach near-perfect AUC (0.997 and 0.996). Whiskers indicate 95% confidence intervals.

Finally, the evolution of model performance across architectures and datasets. On Dataset 1, radiomics‑only and DL‑patch‑only models achieve modest AUCs (0.70 and 0.65), while fusion improves performance to 0.75, confirming that handcrafted and learned MRI signatures provide complementary information. Stacking offers no additional benefit on Dataset 1, likely due to limited sample diversity (Fig. 8). In contrast, models trained on Dataset 2 achieve near‑perfect performance, with fusion and stacking reaching AUCs of 0.997 and 0.996, respectively. This dramatic improvement underscores the importance of dataset quality and diversity in enabling the fusion architecture to fully leverage both radiomics and deep‑learned MRI signatures.

Across both datasets, fusion‑based models consistently delivered the strongest statistical performance, showing significant improvements in AUC, sensitivity, and overall discriminative power compared with baseline radiomics and DL‑only models (Table 1). Dataset 1 showed moderate gains (medium effect sizes), whereas Dataset 2 demonstrated reasonable improvements with near‑perfect AUCs and extremely large effect sizes (Cohen’s d > 4, Cliff’s δ > 0.95). Negative values for Cohen’s d and Cliff’s δ indicate that the comparison model’s performance is compared to the baseline, reflecting a shift in its score distribution in the opposite direction. Classical ML baselines performed well but remained statistically inferior to fusion and stacking approaches, confirming the robustness and scalability of hybrid feature integration.

Table 1.

Summarizing the model’s performance across datasets

Dataset 1
Model	AUC	95% CI	SE	z-score	Accuracy	Sensitivity	Specificity	Precision	F1	Cohen’s d	Cliff’s δ	Effect Size	Sig.
Radiomics SVM	0.7	[0.63–0.76]	0.033	Reference	0.63	0.75	0.5	0.62	0.67	Reference	Reference	Baseline	—
DL-patch SVM	0.65	[0.58–0.72]	0.035	−1.42	0.6	0.76	0.48	0.62	0.68	0.42	0.31	Small–Medium	*
DL-bbox SVM	0.56	[0.50–0.62]	0.03	−3.00	0.52	0.5	0.53	0.55	0.53	−0.65	−0.40	Medium	**
Fusion (α = 0.5)	0.745	[0.69–0.80]	0.028	1.61	0.63	0.78	0.5	0.63	0.69	0.61	0.44	Medium	**
Stacking	0.71	[0.64–0.78]	0.034	0.29	0.61	0.88	0.31	0.59	0.71	0.3	0.22	Small	n.s
Dataset 2
Radiomics SVM	0.985	[0.97–0.995]	0.006	Reference	0.9396	0.932	0.9494	0.9599	0.9458	Reference	Reference	Reference	—
DL-patch SVM	0.9009	[0.86–0.94]	0.015	−5.33	0.8206	0.7943	0.8547	0.8766	0.8334	−2.10	−0.82	Large (negative)	***
Fusion (α = 0.5)	0.999	[0.993–1.000]	0.002	9.5	0.999	0.999	0.999	0.999	0.999	4.12	0.98	Huge	***
Stacking	0.9964	[0.992–1.000]	0.002	9.2	0.972	0.9816	0.9595	0.9692	0.9754	4.05	0.97	Huge	***
Random Forest	0.9379	[0.90–0.97]	0.018	−2.61	0.8518	0.8333	0.8759	0.8971	0.864	−1.55	−0.70	Large (negative)	***
XGBoost	0.9777	[0.96–0.99]	0.009	−0.82	0.9214	0.9264	0.9149	0.934	0.9302	−0.42	−0.31	Small–Medium	*

—(baseline model), * (p < 0.05) significant, ** (p < 0.01) strongly significant, ***(p < 0.01) very strongly significant, n.s (not significant), ref (reference/baseline model).

In addition, the study employed Grad-CAM on the 3D convolutional encoder of our multimodal MRI fusion architecture. For each patient, Grad-CAM generated a 3D importance map relative to the final convolutional layer, which we aligned to the native resolution of the tumor-centered FLAIR sequence. Visualization focused on the axial slice with peak activation within the tumor region (Fig. 9). Analysis of the learned DL embeddings revealed distinct patterns compared to radiomics features: patch-based features excelled at capturing fine-grained texture variations within the tumor core; boundary patches consistently encoded information from peritumoral FLAIR hyperintensity, reflecting tumor infiltration; and T1Gd and FLAIR contributed most strongly to DL predictions. These patterns suggest that DL features capture spatially localized phenotypic characteristics complementary to global radiomics descriptors. Grad-CAM heatmaps consistently concentrated activation in contrast-enhancing and FLAIR-hyperintense tumor compartments: strongest activation localized to the contrast-enhancing tumor core on T1Gd sequences, substantial activation was observed in peritumoral FLAIR-hyperintense regions, and negligible activation occurred in non-tumoral tissue. These findings align with neuropathological knowledge and confirm that the model has learned biologically meaningful imaging patterns.

Fig. 9. Grad-CAM localization of pathologically relevant tumor features.

Representative axial slices overlay Grad-CAM heatmaps (red = highest activation) on T1Gd (left) and FLAIR (right) sequences. Activation concentrates on contrast-enhancing tumor core (T1Gd) and peritumoral FLAIR hyperintense regions (infiltrative margin). Negligible activation in normal brain tissue.

Discussion

This study introduced a multimodal, interpretable machine‑learning framework that integrates handcrafted radiomics features with deep‑learning derived patch embeddings to characterize MRI‑based signatures in glioma. A distinctive aspect of this work is the dual‑dataset design: Dataset 1, representing the original radiomics and DL‑patch feature matrices extracted from the TCGA‑LGG cohort, and Dataset 2, an enriched dataset generated through PCA‑GMM sampling and fusion‑guided pseudo‑labeling^37,38. This structure enabled a controlled examination of how feature diversity, sample size, and data richness influence model performance and interpretability within a unified analytical pipeline.

Across experiments, the two datasets revealed complementary insights. Models trained on Dataset 1 reflected the inherent challenges of real‑world radiomics research, where modest sample sizes and high‑dimensional feature spaces can limit discriminative power. SHAP analysis showed that the radiomics SVM relied primarily on two dominant features, suggesting that the original feature space contained limited variability or redundant descriptors, an observation consistent with prior radiomics literature^39,40,41. The DL‑patch and fusion models exhibited similar behavior, reinforcing the importance of data diversity when working with complex multimodal signatures.

The introduction of Dataset 2 provided a valuable contrast. By enriching the radiomics and DL‑patch spaces through PCA‑GMM sampling and leveraging fusion‑based pseudo‑labels, the dataset captured a broader range of structural and textural MRI patterns⁴². Models trained on this enriched dataset demonstrated markedly improved performance, with the fusion model achieving near‑perfect AUC, sensitivity, and specificity. These results highlight a central theme of the study: data‑centric enrichment can substantially enhance the expressive capacity of multimodal MRI signatures, often more effectively than increasing model complexity alone. In neuro‑oncology, where imaging datasets are frequently limited, such strategies offer a practical path toward more robust and generalizable predictive modeling.

Methodologically, the framework emphasizes interpretability, reproducibility, and multimodal integration. Radiomics features spanning 726 handcrafted descriptors were standardized using PCA and Gaussian mixture modeling (GMM), while DL‑patch embeddings contributed complementary local spatial information^43,44. The fusion strategy, intentionally designed to be simple and transparent, proved highly effective when supported by a sufficiently informative dataset. Interpretability analyses, including SHAP, variance ranking, and correlation mapping, ensured that model behavior remained understandable and clinically grounded. Together, these components demonstrate how a data‑centric, interpretable pipeline can unlock the predictive potential of multimodal MRI features and contribute meaningfully to precision neuro‑oncology.

In conclusion, this work demonstrates that the integration of handcrafted and learned MRI signatures, combined with interpretability tools (SHAP and Grad-CAM), provides a transparent and clinically meaningful approach to predicting chemotherapy benefit in gliomas. The complementary nature of radiomics and deep learning features was validated on both real-world (Dataset 1, AUC = 0.75) and synthetically enriched (Dataset 2, near-perfect AUC) datasets, confirming that data quality and feature diversity are critical drivers of predictive performance. The proposed framework establishes a scalable and reproducible foundation for future multimodal imaging studies in neuro-oncology.

Methods

Dataset description

This study utilized the Pre‑operative TCGA‑LGG NIfTI and Segmentations dataset 1, an openly available resource derived from the TCGA Low‑Grade Glioma cohort⁴⁵. The dataset includes 65 subjects with preoperative multimodal MRI scans and high‑quality tumor segmentation masks. Each patient folder contains T1, T1Gd, T2, and FLAIR sequences, along with automated and manually corrected GlistrBoost segmentation masks. All images are provided in NIfTI format and have undergone skull stripping and spatial co‑registration, ensuring consistency across subjects.

In addition to imaging data, the dataset includes a comprehensive radiomics feature matrix (TCGA_LGG_radiomicFeatures.csv) containing 726 handcrafted features per patient. These features encompass volumetric measurements (e.g., enhancing tumor volume, whole tumor volume), intensity‑based descriptors, texture metrics, and spatial growth model parameters. The dataset does not include chemotherapy or survival outcomes; therefore, it was used primarily for pipeline development, feature extraction, and methodological validation. Future integration with TCGA‑GBM and TCGA‑LGG clinical datasets will enable modeling of postoperative chemotherapy efficacy.

Dataset level characteristics relevant to downstream modeling, presented in Fig. 10. The distribution of patients across response categories, confirming class balance or imbalance. Also, compares tumor volumes between responders and non‑responders, providing insight into potential clinical associations. Additionally, illustrates the distribution of a selected radiomics shape feature, offering another perspective on morphological variability. Moreover shows the distribution of the L2 norms of the DL patch embeddings, reflecting the magnitude and spread of the learned feature representations.

Fig. 10. Dataset characteristics and variable distributions.

a Patient counts stratified by chemotherapy response, showing the class distribution between responders and non-responders. b Comparison of tumor volumes between responders and non-responders, providing insight into potential associations between tumor burden and chemotherapy benefit. c Distribution of a representative radiomics shape feature, offering another perspective on morphological variability. d Distribution of L2 norms of DL patch embeddings, reflecting the magnitude and spread of the learned feature representations.

Postoperative chemotherapy efficacy was operationalized as a binary endpoint among patients who received adjuvant chemotherapy. Overall survival (OS) was calculated from the date of surgery to the date of death or last follow‑up. Patients with $\mathit{OS}\ge 24$ months were categorized as chemotherapy responders, whereas those with $\mathit{OS}<24$ months were categorized as non‑responders. This survival‑based definition has been widely adopted in TCGA‑based radiomics and DL studies as a pragmatic surrogate of treatment benefit when direct radiographic response assessments are unavailable. Radiomic and DL features were derived from TCGA‑LGG MRI data, and clinical survival information was obtained from GDC clinical files. Overall survival was calculated from death and follow‑up timestamps, enabling a binary classification of long‑term versus short‑term survivors (24‑month cutoff). These labels were integrated with imaging features to train and assess multiple predictive models.

OS as a surrogate for chemotherapy benefit is established in LGG literature Wang et al.³⁵, Wu et al.³⁶. Direct chemotherapy response labels are unavailable in public datasets; our approach represents the standard solution in this research domain.

Imaging data were obtained from the Pre-operative TCGA-LGG NIfTI and Segmentations dataset (Bakas et al.⁴⁶). Clinical data, including chemotherapy administration records and overall survival timestamps, were retrieved from the Genomic Data Commons (GDC) clinical files associated with the TCGA-LGG cohort. Only patients who received postoperative adjuvant chemotherapy were included in this study.

Postoperative chemotherapy efficacy was operationalized as a binary endpoint based on OS. Patients with OS ≥ 24 months were categorized as chemotherapy responders, while those with OS < 24 months were categorized as non-responders. This survival-based definition has been adopted in prior TCGA-based radiomics studies as a pragmatic surrogate of treatment benefit when direct radiographic response assessments are unavailable.

Dataset 2 was created via PCA-GMM sampling and fusion-guided pseudo-labeling. PCA reduced the original feature space while preserving the covariance structure, followed by GMM to generate synthetic samples that maintain the original data distribution. This approach expands feature diversity and sample size without introducing unrealistic feature combinations.

This study utilized two complementary data sources from the TCGA-LGG cohort: Imaging data; pre-operative multimodal MRI scans (T1, T1Gd, T2, FLAIR) and tumor segmentation masks were obtained from the TCGA-LGG NIfTI and Segmentations dataset⁴⁶. The dataset includes 65 subjects with complete imaging and segmentation. Clinical data; chemotherapy administration records and overall survival timestamps were retrieved from the Genomic Data Commons (GDC) clinical files associated with the TCGA-LGG cohort (Liu et al.²⁵).

Dataset 2 was generated to overcome the limited feature variability of the original data (n = 65) and to explore the upper-bound potential of the fusion framework. The generation procedure involved three steps:

Step 1: The original radiomics feature matrix (65 × 482) was decomposed using principal component analysis, retaining components explaining 95% of variance (35 components).

Step 2: A Gaussian mixture model with 5 components was fitted to the PCA-reduced space. Synthetic samples were generated from the fitted GMM, preserving the original covariance structure while expanding the sample size to 500 samples.

Step 3: DL embeddings were used to guide label assignment, ensuring consistency with the original response distribution.

Validation procedure: Synthetic data quality was assessed by comparing (a) marginal distributions of original vs. synthetic features (Kolmogorov–Smirnov test, all p > 0.05), (b) covariance structure (Frobenius norm <0.1), and (c) classification performance consistency across 10 random seeds.

Image data processing

Preoperative multimodal MRI scans, including T1‑weighted, contrast‑enhanced T1‑weighted (T1Gd), T2‑weighted, and FLAIR sequences, were obtained in NIfTI format. All images were skull‑stripped and co‑registered to a common anatomical space as provided by the dataset. Tumor segmentation masks were available in two forms: automated GlistrBoost outputs and manually corrected expert annotations. For consistency, manually corrected masks were used as the reference standard. Each MRI volume and corresponding segmentation mask were loaded using NiBabel, and voxel‑wise integrity, dimensional consistency, and spatial alignment were verified.

Intensity normalization performed by

$$I_{norm}\left(x\right)=\left(I\left(x\right)-\mu \right)/\sigma$$ 1

where $I\left(x\right)$ original voxel intensity, μ mean intensity of brain voxels, σ standard deviation. Tumor volume calculation, $V_{tumor}$ and voxel volume v,

$$V_{tumor}=\sum _{x\mathit{\in }\mathrm{\Omega }}M\left(x\right)*v$$ 2

M(x) segmentation mask (1 = tumor, 0 = background),

$$v=s_x*s_y*s_z$$ 3

Radiomics feature extraction and preprocessing

Radiomic features were obtained from the dataset’s accompanying feature matrix, which included 726 handcrafted descriptors capturing tumor intensity, shape, texture, volumetric properties, and spatial characteristics. Initial preprocessing involved removing features with excessive missingness, followed by imputing remaining missing values using median substitution. Features were standardized using z‑score normalization. To ensure robustness, radiomics features were evaluated for redundancy using correlation analysis and PCA, with all intermediate outputs saved as CSV files for reproducibility. Mean, variance, skewness, and kurtosis are derived by:

$$\mu =\left(1/N\right)*\sum _{i=1}^N{x}_i$$ 4

$${\sigma }^2=\left(1/N\right)*\sum _{i=1}^N{\left(x_i-\mu \right)}^2$$ 5

$$Skewness=\left(1/N\right)*\sum _{i=1}^N{\left[\left(x_i-\mu \right)/\sigma \right]}^3$$ 6

$$Kurtosis=\left(1/N\right)*\sum _{i=1}^N{\left[\left(x_i-\mu \right)/\sigma \right]}^4$$ 7

The shape features and compactness derived by using

$$\mathit{SA}=\sum _{f\in F}{A}_f$$ 8

$$Compactness=V^{\left(2/3\right)}/\mathit{SA}$$ 9

Texture features such as contrast, homogeneity, energy, and entropy are given below,

$$Contrast=\sum _{i,j}{\left(i-j\right)}^2*P\left(i,j\right)$$ 10

$$Homogeneity=\sum _{i,j}P\left(i,j\right)/\left(1+\left|i-j\right|\right)$$ 11

$$Energy=\sum _{i,j}P{\left(i,j\right)}^2$$ 12

$$Entropy=-\sum _{i,j}P\left(i,j\right)*\log P\left(i,j\right)$$ 13

Feature Preprocessing performed through z-score ($z_i$), missing values imputation is as,

$$z_i=\left(x_i-\mu \right)/\sigma$$ 14

$$x_i^{imputed}=median\left(X\right)$$ 15

Dimensionality Reduction is achieved by using the covariance matrix, eigenvalue decomposition, and projection onto principal components, as given below, C, C * v_k, and Z,

$$C=\left(1/\left(N-1\right)\right)*X^T*X$$ 16

$$C*v_k={\lambda }_k*v_k$$ 17

$$Z=X*V_k$$ 18

As shown in Table 2, per‑patient radiomics feature vectors were extracted from the tumor region after cropping to a standardized 3D patch. Each numeric column corresponds to a radiomic descriptor (e.g., intensity, spatial, or shape‑related features) used as input for machine‑learning and statistical modeling.

Table 2.

Radiomics feature matrix per patient, derived from standardized 96 × 96 × 96 tumor patches

Patient ID	Feature 1	Feature 2	Feature 3	Feature 4	Feature 5	Feature 6	Feature 7
TCGA-CS-4942	25.897	−42.142	57.553	20.019	−30.878	37.667	69.831
TCGA-CS-4944	1.425	−7.241	11.531	6.53	−6.223	15.729	17.804
TCGA-CS-5393	2.674	−9.430	13.398	7.146	−8.927	17.055	20.226
TCGA-CS-5396	9.899	−16.924	33.752	14.507	−12.811	17.982	50.832
TCGA-CS-5397	18.357	−25.069	38.912	6.824	−20.030	18.876	49.036

Deep learning feature extraction

A 3D convolutional neural network (CNN) encoder was designed to extract high‑level imaging representations from the multimodal MRI volumes. For each patient, the four MRI modalities were stacked into a four‑channel input tensor, and tumor‑centered cropping was performed using the segmentation mask to focus the model on the lesion region. The CNN architecture consisted of sequential 3D convolutional blocks with batch normalization and ReLU activation, followed by global average pooling to generate a compact latent feature vector. These DL features were exported as patient‑level CSV files for downstream fusion modeling. Convolution operations Y(i, j, k), and ReLU activation $ReLU\left(x\right)$, batch normalization $\hat{x},$ global average pooling z_c is done as,

$$Y\left(i,j,k\right)=\Sigma X\left(i+m,j+n,k+p\right)*W\left(m,n,p\right)+b$$ 19

$$ReLU\left(x\right)=max\left(0,x\right)$$ 20

$$\hat{x}=\left(x-{\mu }_B\right)/\mathit{sqrt}\left({\sigma }_B^2+\epsilon \right)$$ 21

$$y=\gamma *\hat{x}+\beta$$ 22

$$z_c=\left(1/\left(H*W*D\right)\right)*\sum _{i=1}^H\sum _{j=1}^W\sum _{k=1}^D{F}_c\left(i,j,k\right)$$ 23

Figure 11 provides an overview of the dataset characteristics and the underlying structure of the extracted features. Illustrates the distribution of whole‑tumor volumes across the cohort, highlighting the substantial heterogeneity in tumor burden. Also, presents a PCA projection of the radiomics feature space, showing how the first two principal components capture the dominant variance patterns and whether any natural separation emerges between responders and non‑responders. Moreover, displays a correlation heatmap of the first 30 radiomics features, revealing clusters of highly correlated descriptors and indicating redundancy within the feature set. Additionally, shows a t‑SNE embedding of the DL patch features, providing an initial view of how the learned representations cluster patients in a lower‑dimensional latent space.

Fig. 11. Underlying data features and radiomics analysis.

a PCA projection of the radiomics feature space (PC1 vs. PC2), showing how the first two principal components capture dominant variance patterns and whether any natural separation emerges between responders and non-responders. b Variance explained by the first ten principal components, indicating the intrinsic dimensionality of the radiomics feature space. c Distribution of a representative radiomics feature (intensity mean of enhancing tumor on T1Gd), illustrating inter-patient variability. d Correlation heatmap of the top 30 radiomics features, revealing clusters of highly correlated descriptors and indicating redundancy within the feature set.

Fusion model and predictive framework

To evaluate the predictive potential of different feature families, three model types were constructed: (1) radiomics‑only models, (2) DL–only models, and (3) hybrid fusion models combining both feature sets. Gradient boosting machines and fully connected neural networks were used as classifiers due to their strong performance with structured and high‑dimensional data. Model training employed stratified k‑fold cross‑validation to ensure generalizability and to allow direct comparison across methods. Performance metrics included accuracy, AUC, sensitivity, specificity, and calibration error. Feature concatenation $F_{\mathrm{fusion}}$, fully connected layer h, output probability $\hat{y}$ and loss function, binary cross-entropy L,

$$F_{fusion}=\left[F_{radiomics},F_{D}L\right]$$ 24

$$h=\sigma \left(W*F_{fusion}+b\right)$$ 25

$$\hat{y}=1/\left(1+e^{\left(-z\right)}\right)$$ 26

$$L=-\left[y*log\left(\hat{y}\right)+\left(1-y\right)*log\left(1-\hat{y}\right)\right]$$ 27

In Fig. 12, the structure of the DL feature embeddings derived from different input strategies is depicted. It shows a t‑SNE projection of the DL patch features, while the corresponding t‑SNE projection is for the DL bounding‑box features. t‑SNE embedding of the fused DL features, combining patch‑ and bounding‑box‑based representations. A UMAP projection of the fusion features, offering an alternative nonlinear view that often reveals clearer cluster boundaries.

Fig. 12. Deep learning feature embeddings and dimensionality reduction visualization.

a t-SNE projection of DL patch features, showing the clustering structure of local texture features in low-dimensional latent space. b t-SNE projection of DL bounding-box features, showing the distribution pattern of global tumor morphology features. c t-SNE projection of fused DL embeddings combining patch and bounding-box features, demonstrating the complementarity of different feature types. d UMAP projection of fused features, offering an alternative nonlinear view that often reveals clearer cluster boundaries.

The 3D bounding‑box coordinates used to crop each patient’s tumor region from the full CT/MRI volume are shown in Table 3. In radiomics and deep‑learning pipelines, you rarely feed the entire scan into the model; instead, you extract a standardized 3D patch around the lesion. Each row corresponds to a patient, and the columns define:

$$xmin,xmax\to left-rightbounds$$

$$ymin,ymax\to anterior-posteriorbounds$$

$$zmin,zmax\to inferior-superiorbounds$$

crop_shape to the final standardized 3D patch size used for model input (here: 96 × 96 × 96 voxels), a complete detailed structure is presented in Table 4.

Table 3.

Coordinates for standardized 96 × 96 × 96 brain tumor patches extracted from 3D MRI volumes

ID	Modality	Patch1_center	Patch2_boundary1	Patch3_boundary2	Patch shape
TCGA-CS-4942	flair	[87 119 76]	[97 105 64]	[94 131 70]	(64, 64, 64)
TCGA-CS-4942	t1	[87 119 76]	[97 105 64]	[94 131 70]	(64, 64, 64)
TCGA-CS-4942	t1gd	[87 119 76]	[97 105 64]	[94 131 70]	(64, 64, 64)
TCGA-CS-4942	t2	[87 119 76]	[97 105 64]	[94 131 70]	(64, 64, 64)
TCGA-CS-4944	flair	[86 134 66]	[76 143 84]	[73 129 66]	(64, 64, 64)

Table 4.

showing the 3D CNN encoder designed architecture

3D CNN encoder designed architecture
Layer 1: Conv3D (3 × 3 × 3 kernel, 16 filters, stride 1, padding same) → BatchNorm3D → ReLU → MaxPool3D (2 × 2 × 2, stride 2)
Layer 2: Conv3D (3 × 3 × 3 kernel, 32 filters, stride 1) → BatchNorm3D → ReLU → MaxPool3D (2 × 2 × 2, stride 2)
Layer 3: Conv3D (3 × 3 × 3 kernel, 64 filters, stride 1) → BatchNorm3D → ReLU → GlobalAvgPool3D
Fully Connected: 64 → 128
Training details: Optimizer = Adam (learning rate = 1e-4, β1 = 0.9, β2 = 0.999), batch size = 8, epochs = 100, early stopping patience = 20 epochs. Loss function = binary cross-entropy.

Explainable AI

Explainability was incorporated at two levels. For radiomics‑based models, SHAP (SHapley Additive exPlanations) was used to quantify the contribution of individual features to model predictions. For DL models, Grad‑CAM was applied to generate heatmaps highlighting the MRI regions most influential in the prediction. These interpretability outputs were saved as high‑resolution images (300 dpi) for inclusion in the manuscript and presentation. SHAP value defined by ${\phi }_i$, Grad Cam heatmap $L_{\mathrm{Grad}}{\mathrm{CAM}}^c$, where evaluation metrics are used.

$${\phi }_i=\Sigma \left[\left(\left|S\right|!*\left(\left|F\right|-\left|S\right|-1\right)!\right)/\left|F\right|!\right]*\left(f\left(S\cup \left(i\right)\right)-f\left(S\right)\right)$$ 28

$${L_{GradCAM}}^c=ReLU\left(\sum _k{\alpha }_k^c*A^k\right)$$ 29

$${\alpha }_k^c=\left(1/Z\right)*\sum _{i,j,k}\left({\partial y^c/\partial A_{\mathit{ijk}}}^k\right)$$ 30

$$Accuracy=\left(TP+TN\right)/\left(TP+TN+FP+FN\right)$$ 31

$$Sensitivity=TP/\left(TP+FN\right)$$ 32

$$Specificity=TN/\left(TN+FP\right)$$ 33

$$AUC=\int TPR\left(FP{R}^{-}1\left(x\right)\right)\mathit{dx}$$ 34

Clinical decision support system prototype

Finally, an interpretable clinical decision support system (CDSS) interface was conceptualized. The CDSS integrates model predictions, radiomics‑based explanations, and Grad‑CAM visualizations to provide clinicians with transparent, patient‑specific insights. Although not deployed in a clinical environment, the prototype demonstrates the translational potential of the proposed framework.

Author contributions

A.C. and P.D. oversaw the study's conception and design. Y.W., L.C., and Y.G. analyzed and interpreted of the gathered data. S.Z. and J.B. authored the manuscript outlining their research findings. All authors significantly contributed to the paper and collectively approved the final version submitted for publication.

Data availability

The raw multimodal MRI scans and tumor segmentation masks supporting this study are openly available in The Cancer Imaging Archive (TCIA) under the “Pre-operative TCGA-LGG NIfTI and Segmentations” collection⁴⁵. The clinical and survival data were retrieved from the Genomic Data Commons (GDC) portal. The processed radiomics feature matrices and deep learning embedding vectors generated during the current study are not publicly available due to their derived nature and ongoing research use; however, they can be obtained from the corresponding author upon reasonable request and with appropriate data usage agreements.

Competing interests

The authors declare no competing interests.