Optimizing meningioma grading with radiomics and deep features integration, attention mechanisms, and reproducibility analysis

Abstract

Objective

This study aims to develop a robust and clinically applicable framework for preoperative grading of meningiomas using T1-contrast-enhanced and T2-weighted MRI images. The approach integrates radiomic feature extraction, attention-guided deep learning models, and reproducibility assessment to achieve high diagnostic accuracy, model interpretability, and clinical reliability.

Materials and methods

We analyzed MRI scans from 2546 patients with histopathologically confirmed meningiomas (1560 low-grade, 986 high-grade). High-quality T1-contrast and T2-weighted images were preprocessed through harmonization, normalization, resizing, and augmentation. Tumor segmentation was performed using ITK-SNAP, and inter-rater reliability of radiomic features was evaluated using the intraclass correlation coefficient (ICC). Radiomic features were extracted via the SERA software, while deep features were derived from pre-trained models (ResNet50 and EfficientNet-B0), with attention mechanisms enhancing focus on tumor-relevant regions. Feature fusion and dimensionality reduction were conducted using PCA and LASSO. Ensemble models employing Random Forest, XGBoost, and LightGBM were implemented to optimize classification performance using both radiomic and deep features.

Results

Reproducibility analysis showed that 52% of radiomic features demonstrated excellent reliability (ICC > 0.90). Deep features from EfficientNet-B0 outperformed ResNet50, achieving AUCs of 94.12% (T1) and 93.17% (T2). Hybrid models combining radiomic and deep features further improved performance, with XGBoost reaching AUCs of 95.19% (T2) and 96.87% (T1). Ensemble models incorporating both deep architectures achieved the highest classification performance, with AUCs of 96.12% (T2) and 96.80% (T1), demonstrating superior robustness and accuracy.

Conclusion

This work introduces a comprehensive and clinically meaningful AI framework that significantly enhances the preoperative grading of meningiomas. The model’s high accuracy, interpretability, and reproducibility support its potential to inform surgical planning, reduce reliance on invasive diagnostics, and facilitate more personalized therapeutic decision-making in routine neuro-oncology practice. Clinical trial number: Not applicable.

Introduction

Meningiomas are the most common type of primary tumor in the central nervous system. They are classified into three grades based on histopathological features: Grade I (benign), Grade II (atypical), and Grade III (anaplastic) [1]. Correctly grading meningiomas is essential for determining treatment options, predicting patient outcomes, and guiding clinical decisions. Traditionally, this grading is done through histopathological examination, which is time-consuming and can vary depending on the observer [2, 3]. This has led to increased interest in using non-invasive imaging techniques, particularly magnetic resonance imaging (MRI), to grade meningiomas reliably.

Radiomics, the extraction of quantitative features from medical images, has improved the ability to identify hidden tumor characteristics that are not visible to the human eye [4, 5]. Features like texture, shape, and intensity can provide additional insights into tumor behavior, offering a more precise way of grading [6, 7]. However, these features are sensitive to variations in the segmentation process, where differences in tumor boundaries can significantly affect the extracted features [8–12]. Other factors, such as MRI protocols and machine types, can also cause variations in these radiomic features [13–15]. Therefore, reproducibility assessments are crucial to ensure that these features are reliable across different segmentation methods, scanner types, and imaging conditions [16–19].

Deep learning methods, especially convolutional neural networks (CNNs), have become very successful in medical image analysis, including tumor classification and segmentation [20]. Deep learning models automatically learn patterns from large datasets, allowing them to identify complex relationships within the images [21–23]. However, one challenge with traditional deep learning methods is that the model may not focus on the most diagnostically important regions of the image, which is essential for accurate grading. Attention mechanisms, integrated into deep learning models, have been developed to overcome this challenge. These mechanisms allow the model to concentrate on the most relevant parts of the image, improving performance and interpretability, especially in medical imaging tasks where it is crucial to locate the most significant features [24–26].

This study introduces a hybrid framework for preoperative meningioma grading that combines radiomic feature extraction, attention-enhanced deep learning, and reproducibility assessment to overcome the limitations of traditional methods. Deep learning is applied both as an end-to-end classification pipeline and as a feature extractor, where intermediate features from pre-trained models (ResNet50, EfficientNet-B0) are fused with radiomic descriptors to improve diagnostic performance and interpretability. This dual strategy leverages the complementary strengths of radiomics (quantitative texture and shape analysis) and deep learning (hierarchical pattern recognition), resulting in a more robust and clinically meaningful grading system.

Key innovations include reproducibility validation using the intraclass correlation coefficient (ICC), the use of attention mechanisms to focus on diagnostically critical tumor regions, and integration of deep and radiomic features for improved accuracy. The framework was rigorously validated on a diverse multi-institutional dataset of T1-contrast-enhanced and T2-weighted MRI scans, achieving high sensitivity and AUC values. The attention maps enhanced transparency by highlighting regions most influential to model decisions. These contributions position the proposed model as a reliable and scalable tool for clinical adoption in differentiating low-grade (Grade I) from high-grade (Grade II/III) meningiomas.

Materials and methods

Dataset, inclusion, and exclusion criteria

For this study, we used a dataset of 2546 patients diagnosed with meningiomas, including both low-grade (Grade I) and high-grade (Grade II and III) tumors (Table 1). The dataset included 1560 low-grade meningiomas and 986 high-grade meningiomas. Patients were selected based on the availability of annotated brain MRI scans, which were classified by expert radiologists according to histopathological results. The inclusion criteria required that patients had clear MRI scans with sufficient quality for analysis and annotation, as well as a confirmed diagnosis of meningioma through histopathological examination. Figure 1 shows sample images from the brain MRI scans.

Table 1.

Clinical information of patients

Parameter	Low-grade meningiomas (Grade I)	High-grade meningiomas (Grade II and III)
Number of patients	1560	986
Mean age (years)	47.8 ± 12.3	55.2 ± 13.6
Gender distribution	52% male, 48% female	58% male, 42% female
Mean tumor size (cm)	2.7 ± 1.1	3.4 ± 1.3
Tumor location	Parietal: 30%, frontal: 25%, temporal: 20%, other: 25%	Parietal: 40%, frontal: 30%, temporal: 15%, other: 15%
MRI scans used	T1, contrast-enhanced, T2	T1, contrast-enhanced, T2
Mean follow-up (months)	22.5 ± 10.2	18.9 ± 9.8
Preoperative symptoms	Headache: 45%, seizures: 25%, no symptoms: 30%	Headache: 60%, seizures: 35%, no symptoms: 5%
Clinical outcome	No recurrence: 88%, recurrence: 12%	No recurrence: 64%, recurrence: 36%
Surgical approach	Craniotomy: 90%, endoscopic: 10%	Craniotomy: 85%, endoscopic: 15%
Histopathological features	Mitosis: 3 ± 1 per 10 HPF, brain invasion: 10%	Mitosis: 12 ± 3 per 10 HPF, brain invasion: 35%
Mean tumor volume (cm3)	30.2 ± 15.3	45.7 ± 20.1
Radiotherapy post-surgery	10%	40%
Recurrence time (months)	24.5 ± 5.3	18.2 ± 4.8

Fig. 1.

Sample brain MRI scans used in the study

In this study, Grade I tumors were labeled as low-grade, and both Grade II and Grade III tumors were grouped under the high-grade category. This dichotomization was based on two factors: (1) the limited number of Grade III cases in our dataset, which would introduce substantial class imbalance in a three-class classification setting and compromise model generalization; and (2) alignment with clinical practice, where preoperative risk stratification commonly distinguishes between low-grade and high-grade meningiomas to inform surgical planning and the need for adjunctive therapies. Although we recognize the underlying biological and clinical heterogeneity between WHO Grade II and Grade III meningiomas, consolidating these into a single high-grade category enhances the model’s clinical applicability and statistical robustness, particularly in light of real-world diagnostic paradigms that prioritize binary risk stratification for therapeutic decision-making.

The study included adult patients aged 18 or older who had been diagnosed with meningioma, confirmed through histopathological examination. Eligible patients were required to have MRI scans available for analysis, including T1-contrast-enhanced and T2-weighted images. Only patients with tumors larger than 1 cm in size were included. Patients were excluded if they had other central nervous system tumors or diseases, as this could affect the analysis. Cases with incomplete or low-quality MRI scans, such as those with motion artifacts or poor resolution, were also excluded to ensure accurate analysis. In addition, patients with unclear histopathological classifications or missing clinical data were not included in the study. Finally, individuals with incomplete demographic or clinical information were excluded, as this could reduce the reliability and generalizability of the findings.

Figure 2 shows the detailed steps of the study, starting with how patients were selected and the MRI imaging protocols were used. It then explains the stages of image preprocessing, tumor segmentation, and feature extraction (both radiomic and deep learning-based). Next, it covers how the features were combined, followed by reducing the number of features and selecting the most important ones. The figure then describes the process of training the model using advanced machine learning techniques, and ends with how the model’s performance was evaluated using accuracy, sensitivity, and AUC-ROC metrics.

Fig. 2.

Workflow of study: MRI analysis, feature extraction, and classification framework for meningioma grading

Image acquisition and MRI imaging protocols

MRI scans for this study were collected from multiple institutions, all following standardized protocols for T1-weighted, T2-weighted, and contrast-enhanced images. The dataset was compiled from 5 tertiary medical centers, each contributing between 412 and 615 patients, ensuring diversity in patient demographics and imaging equipment. MRI scans were performed using 1.5 T scanners, including models from Siemens, GE, and Philips. While acquisition protocols varied slightly across institutions, all centers followed standard clinical neuroimaging protocols for brain tumors, including axial T1-contrast-enhanced and T2-weighted sequences. To mitigate inter-institutional variability, all images underwent z score intensity normalization, spatial resampling to isotropic voxel dimensions (1 × 1 × 1 mm3), and orientation alignment prior to segmentation and analysis.

To ensure consistency across the dataset, the imaging parameters were harmonized, although slight differences in scanner models and protocols occurred due to the multi-center nature of the study. Table 2 provides a summary of the key MRI imaging parameters for both T1 and T2-weighted scans used in the study. It includes details like repetition time (TR), echo time (TE), flip angle, slice thickness, field of view (FOV), and matrix size for both imaging sequences. These parameters were standardized across all participating institutions to ensure uniformity in image acquisition and reduce variability between different MRI scanners.

Table 2.

MRI imaging protocols for T1 and T2-weighted images

Parameter	T1-weighted images	T2-weighted images
Repetition time (TR)	600–800 ms	3000–4000 ms
Echo time (TE)	10–15 ms	80–100 ms
Flip angle	15–20 degrees	90 degrees
Slice thickness	1–2 mm	1–2 mm
Field of view (FOV)	200–220 mm	200–220 mm
Matrix size	256 × 256 or 512 × 512	256 × 256 or 512 × 512

Although T1 pre-contrast images were available for a subset of patients, they were excluded from the analysis due to inconsistent availability across the cohort and their relatively lower diagnostic utility in delineating tumor boundaries. T1-contrast-enhanced and T2-weighted sequences were selected for their superior visibility of lesion morphology and consistent presence across all patients, facilitating uniform image preprocessing, segmentation, and feature extraction.

For the contrast-enhanced images, the same imaging settings used for the T1-weighted images were applied, along with an intravenous injection of a gadolinium-based contrast agent at a dose of 0.1 mmol/kg body weight. The scans were done 5–10 min after the injection to ensure the tumor was clearly enhanced. All MRI scans were taken with high resolution and a slice thickness of 1–2 mm to clearly define the tumor. The FOV was adjusted to capture the entire tumor, with slight changes made based on the patient’s anatomy. The imaging protocols were standardized across all institutions to reduce differences between scanners, ensuring the study’s analysis was consistent and the radiomic and deep learning evaluations were reliable.

Image preprocessing

Image normalization

Image normalization was used to standardize the intensity values across all MRI scans, ensuring consistency for further analysis. This process involved adjusting the pixel intensities of each image to a fixed range, usually between 0 and 1, to eliminate differences caused by varying scanner models or imaging protocols. Normalization helped reduce the impact of external factors, like changes in field strength and coil sensitivity, making the feature extraction and deep learning model performance more reliable.

Image resizing and augmentation

To make the MRI images compatible with transfer learning models, all images were resized to a uniform resolution of 224 × 224 × 3 pixels. This was important for maintaining consistency across the dataset and for using pre-trained convolutional neural networks (CNNs). These pre-trained models, which were initially trained on large datasets like ImageNet, require a fixed input size to work properly. Resizing the images allowed the models to process them efficiently and use their learned features to improve performance in classifying meningiomas. To further enhance the dataset, data augmentation techniques like random rotations, flips, and translations were applied. This increased the variety of the dataset and helped the model generalize better, especially since the study had fewer high-grade meningioma cases. Augmentation also helped prevent the model from overfitting by exposing it to more variations during training, which improved its ability to handle new, unseen data. In addition, it ensured the model could detect small changes in tumor size, shape, and orientation, which are essential for accurate classification. This comprehensive preprocessing was key to improving the model’s performance and ensuring reliable results.

No explicit skull stripping was applied prior to segmentation, as tumor annotations were manually performed by expert radiologists using ITK-SNAP, ensuring that only tumor-containing regions were delineated and non-brain structures, including the skull, were excluded from analysis. Radiomic features were extracted solely from these segmented tumor regions, minimizing any influence of extracranial tissue. Furthermore, all images underwent z score intensity normalization within the brain volume to mitigate inter-scanner variability and improve the comparability of pixel intensity distributions across subjects and institutions.

Segmentation

Tumor segmentation using ITK-SNAP

Tumor segmentation was done using ITK-SNAP, a popular software tool for medical image segmentation. ITK-SNAP supports both manual and semi-automatic segmentation of anatomical structures in MRI scans. In this study, radiologists used ITK-SNAP to outline the tumor boundaries in T1, T2, and contrast-enhanced MRI images. The software helped accurately identify the tumor volume and ensured precise segmentation by allowing interactive editing and adjustments during the process.

Multi-segmentation strategy for reproducibility

To ensure the segmentation process was consistent, two experienced radiologists independently segmented each tumor twice. This was done to check how consistent their tumor boundary markings were and to reduce any differences between them. The radiologists traced the tumor boundaries on each MRI scan without seeing each other’s work. After the initial segmentations, they compared and reviewed the results. The final segmentation was agreed upon by both radiologists, making sure they both agreed on the tumor boundaries. This process helped ensure the segmentation was reliable, which is crucial for accurate feature extraction and model training. The variability in the segmentations was also measured using ICC.

Radiomic feature extraction and preprocessing

In this study, 215 quantitative radiomic features were extracted from MRI data of low-grade and high-grade meningiomas using the Standardized Environment for Radiomics Analysis (SERA) software. SERA is a validated platform that follows the guidelines of the Image Biomarker Standardization Initiative (IBSI) and is commonly used in multi-center studies to standardize radiomics across different platforms. The study focused on 79 first-order features and 136 higher-order 3D features from a total of 487 available standardized features in SERA.

The first-order features include 29 morphological features that describe the tumor’s shape and size, 2 Local Intensity Features that represent the voxel intensity compared to nearby voxels, 18 Statistical Features summarizing the intensity distribution, 23 Intensity Histogram Features that show the frequency of different voxel intensities, and 7 Intensity Volume Histogram Features that link intensity levels with tumor volume. The higher order 3D features provide insights into the texture and spatial structure of the tumor. These include 50 Co-occurrence Matrix Features that capture texture heterogeneity, 32 Run-Length Matrix Features that measure texture uniformity, 16 Size Zone Matrix Features that define areas with similar intensity, 16 Distance Zone Matrix Features that measure the spatial relationships between intensity zones, 5 neighborhood grey-tone difference matrix features that examine local intensity variations, and 17 neighboring grey-level dependence matrix features that describe intensity dependencies between adjacent voxels.

The MRI images were preprocessed to standardize voxel spacing and normalize intensity values, ensuring consistency across all scans. Tumor regions were manually segmented with high precision to define the ROI. Radiomic feature extraction was then performed on each ROI using the SERA platform, which ensures uniformity across datasets by following guidelines for intensity normalization and binning. A total of 215 radiomic features were extracted to capture a broad range of tumor characteristics. To improve model accuracy and reduce overfitting, features with low variance or high correlation were removed. This careful selection process ensured that only the most informative and independent features were kept for further analysis, enhancing the model’s robustness.

Deep feature extraction, attention mechanism, and feature fusion

In this study, deep learning models were used to extract high-level features from MRI images to classify low and high-grade meningiomas. We employed ResNet50 and EfficientNet-B0 architectures pre-trained on the ImageNet dataset for deep feature extraction from the segmented tumor regions. These models were not fine-tuned on the medical imaging data; instead, we extracted intermediate-layer feature maps to leverage their general-purpose representational power. Prior research has demonstrated that such transfer learning approaches can be effective in extracting informative features from medical images, particularly when used in combination with classical machine learning classifiers. Using these pre-trained networks as feature extractors rather than classifiers, we minimized overfitting and preserved domain-agnostic texture and shape patterns relevant for tumor grading. After passing through the Average Pooling layer, the output from both networks consisted of feature vectors—2048 dimensions for ResNet50 and 1280 dimensions for EfficientNet-B0. These vectors contained rich, high-level information necessary for tumor classification. To enhance the discriminative capacity of deep features, we embedded convolutional block attention modules (CBAMs) within both ResNet50 and EfficientNet-B0 architectures. CBAM operates through two sequential sub-modules: a channel attention module that emphasizes the importance of informative feature maps, and a spatial attention module that highlights salient regions within each map. This dual mechanism allows the network to adaptively recalibrate intermediate features during training, enabling the model to attend more effectively to relevant tumor areas. The attention-enhanced features extracted from these modified networks were then passed to the classification stage. In addition, to further interpret the model’s predictions, we employed Gradient-weighted Class Activation Mapping (Grad-CAM) as a post hoc visualization tool, illustrating which tumor regions influenced the network’s final decision. Figure 3 illustrates the structure of the network, showing how features were extracted and fused from the two models.

Fig. 3.

Structure of deep feature extraction and ensemble model integration

Feature integration

In this study, a comprehensive approach was used to combine both radiomic features and deep learning features to improve the classification accuracy of low- and high-grade meningiomas. The process involved merging traditional radiomic features, extracted directly from the MRI images, with high-level deep features obtained from pre-trained deep learning models such as ResNet50 and EfficientNet-B0. This integration allowed the model to benefit from both the detailed, quantitative information provided by radiomics and the powerful, context-rich features learned by deep learning models, ultimately enhancing the accuracy of tumor classification.

Combination of radiomic and deep features

To build a strong set of features, radiomic features were first extracted from the tumor regions in the MRI scans. These features capture important details about the tumor’s texture, shape, and intensity. At the same time, deep learning features were taken from the middle layers of pre-trained models, like the Average Pooling layer, which provide more abstract and detailed representations of the tumor areas. By combining both radiomic and deep learning features, the model was able to understand the tumor better, using both simple pixel-level details and more complex, higher level information from the deep learning models. This made the tumor classification more accurate and reliable.

Feature fusion strategy

After combining the radiomic and deep features, a feature fusion strategy was applied to improve the model’s accuracy. The features from both sources were merged using a technique that combined the information in a way that highlighted their complementary strengths. This fusion process aimed to reduce redundancy and ensure that only the most useful features from each source were included in the final model. Once the features were fused, the integrated set was fed into machine learning classification models for further processing. This approach helped the model perform better in distinguishing between low- and high-grade meningiomas.

Reproducibility assessment

Ensuring the reproducibility of radiomic and deep learning feature extraction is essential to confirm the reliability and consistency of the results, especially when tumor segmentation is involved. In this study, statistical measures were used to assess the consistency of the features across repeated tumor segmentations and feature extraction processes. This helped evaluate how reliable and stable the feature extraction process was, ensuring that the results could be trusted for further analysis and model development.

Reliability and consistency of the tumor segmentation process

To assess the reliability and consistency of the tumor segmentation process, the ICC was used. Tumors were independently segmented twice by two experienced radiologists, and the ICC was calculated to measure how much agreement was there between the two segmentations. A high ICC value indicates that the tumor segmentation is reliable and consistent, ensuring that the regions of interest (ROIs) for feature extraction are well-defined and reproducible. This is crucial to ensure that the extracted features are not overly influenced by small variations in manual segmentation, making them reliable for model training and evaluation.

To further validate the consistency of the features extracted from the segmented tumor regions, an evaluation of feature consistency was conducted. This process assessed whether the extracted features—both radiomic and deep learning-based—remained stable when tumor segmentation was repeated. The features were evaluated for their ability to consistently distinguish between low-grade and high-grade meningiomas across repeated segmentations. In addition, cross-validation was applied to check that the features were robust and discriminatory across different subsets of the dataset. This step confirmed that the features were not only consistent but also maintained their ability to predict tumor grade accurately, thus enhancing the reproducibility of the study. Based on the ICC values, radiomic features were classified into four levels of reliability: excellent (ICC > 0.90), good (0.75 < ICC ≤ 0.90), moderate (0.50 < ICC ≤ 0.75), and poor (ICC ≤ 0.50). Only features with excellent or good reliability were selected for further analysis to ensure robust and reproducible results. ICC calculations were carried out using an in-house Python-based code developed specifically for this study, ensuring precision and consistency in the statistical assessments. This process ensured that only the most reliable features were chosen for subsequent analysis, strengthening the overall robustness of the study’s findings.

Feature integration and classification framework

In this study, a hybrid framework was developed to improve the diagnostic accuracy for classifying low- and high-grade meningiomas. This approach combines both radiomic features and deep learning-derived features, taking advantage of the strengths of each. Radiomic features are clinically interpretable and capture key tumor characteristics such as shape, intensity, and texture, which are often used by radiologists. On the other hand, deep features, extracted from pre-trained models like ResNet50 and EfficientNet-B0, offer a data-driven, nuanced representation of complex patterns within the tumor images. By merging these two feature types, the framework provides a comprehensive and multifaceted view of the tumor, combining expert-driven insights with machine learning-derived patterns.

Given the high dimensionality of the combined feature set, dimensionality reduction and feature selection techniques were used to optimize the classification process and reduce the risk of overfitting. Principal component analysis (PCA) was applied to reduce the dimensions of the feature set, identifying the most important components that explain the majority of the variance in the data. In addition, LASSO (Least Absolute Shrinkage and Selection Operator) regularization was employed for feature selection. LASSO helps by shrinking the coefficients of less relevant features to zero, leaving only the most important features based on their ability to predict the tumor grade. This process ensured that the final set of features was both informative and manageable, enhancing the model’s performance.

For the classification task, three advanced machine learning models were used: Random Forest, XGBoost, and LightGBM. Random Forest, an ensemble method based on decision trees, is known for its ability to provide stable predictions and resist overfitting by averaging predictions across multiple trees. XGBoost, a gradient-boosted decision tree algorithm, is highly efficient and performs exceptionally well on structured data by optimizing both speed and accuracy during training. LightGBM, another gradient boosting algorithm, excels with large datasets, offering faster training times without compromising performance. These models were chosen for their complementary strengths, ensuring a robust and effective classification system for distinguishing between low- and high-grade meningiomas.

Model training and evaluation

The dataset was divided into three parts: 70% for training, 10% for validation, and 20% for testing. Hyperparameter tuning was conducted for both traditional machine learning models and deep learning models to optimize their performance. For the traditional models, such as Random Forest, XGBoost, and LightGBM, techniques like Grid Search and Randomized Search were used to find the best values for key hyperparameters, including learning rate, number of estimators, maximum depth of decision trees, and regularization parameters. For the deep learning models, specifically DenseNet121 and EfficientNet-B0, the learning rate was set to 0.001 with the Adam optimizer, a batch size of 32, and 1500 epochs. To reduce the risk of overfitting, regularization techniques were applied, including a dropout rate of 0.5 and weight decay. Dynamic learning rate scheduling and early stopping were also implemented to prevent overfitting during training. In addition, for fine-tuning, some layers of the pre-trained networks were unfrozen, allowing the model to better adapt to the specific task and improve its ability to extract relevant features for meningioma classification.

Model performance was evaluated using three metrics: accuracy, sensitivity (recall), and AUC-ROC. Sensitivity was particularly important for detecting high-grade meningiomas, ensuring that the model could identify them effectively. The AUC-ROC curve measured the model’s ability to distinguish between low- and high-grade tumors.

The experiments were run on a system with an NVIDIA Tesla V100 GPU (32 GB), Intel Xeon CPU (2.60 GHz, 32 cores), and 256 GB of RAM. The models were implemented using TensorFlow 2.0 and Scikit-learn, with training and evaluation performed on a Linux-based operating system.

Results

Reliability and categorization of radiomic features based on ICC values

The results presented in Table 3 show how radiomic features were categorized based on their intraclass correlation coefficient (ICC) values, reflecting their reliability for further analysis. Out of the total 215 features, the majority—112 features (52%)—demonstrated excellent reliability with ICC values greater than 0.90, indicating high consistency and reproducibility. Another 50 features (23%) were categorized as having good reliability (ICC between 0.75 and 0.90), further supporting the robustness of the feature set.

Table 3.

Reliability categorization of radiomic features based on ICC values

Feature category	Total features	Excellent reliability (ICC > 0.90)	Good reliability (0.75 < ICC 0.90)	Moderate reliability (0.50 < ICC 0.75)	Poor reliability (ICC 0.50)
Morphology Features (Morph)	29	10	5	7	7
Local Intensity Features (LOC)	2	2	0	0	0
Statistics Features (STAT)	18	6	4	4	4
Intensity Histogram Features (IH)	23	10	6	2	5
Intensity Volume Histogram Features (IVH)	7	2	3	1	1
Co-occurrence Matrix Features (CM)	50	35	10	3	2
Run-Length Matrix Features (RLM)	32	20	10	1	1
Size Zone Matrix Features (SZM)	16	10	4	1	1
Distance Zone Matrix Features (DZM)	16	11	2	1	2
Neighborhood Grey Tone Difference Features (NGT)	5	1	1	2	1
Neighboring Grey Level Dependence Features (NGL)	17	5	5	5	2
Total	215	112	50	27	26

However, a smaller portion of features showed moderate or poor reliability. Specifically, 27 features (12%) had moderate reliability, while 26 features (12%) had poor reliability (ICC ≤ 0.50). Among the reliable features, Morphology Features (Morph) and Co-occurrence Matrix Features (CM) were the most prevalent, suggesting that these features are strong candidates for use as biomarkers in meningioma grading. On the other hand, categories such as Neighborhood Grey Tone Difference Features (NGT) and Intensity Volume Histogram Features (IVH) showed lower reliability. This suggests that these features might need refinement or more consistent imaging protocols before they can be considered reliable for predictive modeling.

Overall, these findings emphasize the importance of selecting features with high reliability to ensure the accuracy and reproducibility of subsequent analyses.

Figure 4 offers a comprehensive view of the various radiomic feature categories used in this study, showcasing the types of features and their corresponding reliability, as indicated by their ICC values. The features are organized into categories such as Morphology, Local Intensity, Co-occurrence Matrix, and others. Each category contributes to the overall analysis, with their reliability playing a key role in their effectiveness for meningioma grading. The visual breakdown helps to highlight which feature types perform best in terms of consistency and reliability for accurately classifying tumor grades.

Fig. 4.

Feature overview and categorization

Figures 5 and 6 display the reliability distribution of the deep features extracted from ResNet50 and EfficientNet-B0, respectively. The analysis revealed notable differences in the percentage distribution of features across various reliability categories for both networks. For EfficientNet-B0, a significant 61% of the features were classified as having poor reliability, while only 14% exhibited excellent reliability and 20% showed moderate reliability. This indicates that a substantial portion of the extracted features may require further refinement or better consistency in image acquisition protocols. In contrast, ResNet50 displayed a more balanced distribution, with 11% of features falling into the excellent category, 9% in the good category, and 17% in the moderate category. These results suggest that ResNet50 may provide more consistent and reproducible deep features compared to EfficientNet-B0, which can be critical for ensuring the robustness of the model, particularly when integrating these features with radiomic data for tumor classification.

Fig. 5.

ICC-based feature reliability distribution for EfficientNet-B0

Fig. 6.

ICC-based feature reliability distribution for ResNet50

Radiomic features analysis

The results of this study highlight the benefits of combining radiomic features with deep learning-derived features and utilizing ensemble models to improve the classification accuracy for meningioma grading. The study explored various approaches, including traditional machine learning models applied to both radiomic and deep features, deep feature extraction pipelines, and end-to-end ensemble deep learning frameworks.

Figure 7 illustrates the performance of radiomic features extracted from T2-weighted and T1-contrast-enhanced MRI images when analyzed using machine learning models. For both imaging modalities, XGBoost with RFE achieved the highest performance, with testing AUC values of 87.39% for T2-weighted images and 88.59% for T1-contrast-enhanced images. LightGBM with RFE also performed well, particularly for T1-contrast images, where it achieved a testing AUC of 87.26%.

Fig. 7.

Performance of radiomic features extracted from T2-weighted and T1-contrast-enhanced MRI

In terms of other evaluation metrics, XGBoost with RFE performed consistently, achieving an accuracy of 86.23% for T2-weighted images and 87.35% for T1-contrast-enhanced images, with sensitivity scores of 86.82% and 86.90%, respectively. In contrast, Random Forest models showed lower performance across both feature selection methods (RFE and LASSO), with testing AUC values ranging from 79.29 to 85.22%, and sensitivity scores remaining below 80%. These results demonstrate the superiority of XGBoost in combination with feature selection techniques for classifying meningiomas based on MRI features.

Deep feature extraction

The evaluation of deep features extracted from ResNet50 and EfficientNet-B0 using machine learning classifiers, as shown in Figs. 8 and 9, demonstrates the superior performance of EfficientNet-B0 over ResNet50. For T2-weighted MRI images, EfficientNet-B0 with Recursive Feature Elimination (RFE) consistently outperformed ResNet50, achieving a testing AUC of 93.17%, compared to 90.21% for ResNet50. Similarly, for T1-contrast-enhanced images, EfficientNet-B0 with RFE achieved a testing AUC of 94.12%, while ResNet50 reached 90.21%.

Fig. 8.

Performance of deep features extracted from EfficientNet-B0 using T2-weighted and T1-contrast-enhanced MRI

Fig. 9.

Performance of deep features extracted from ResNet50 using T2-weighted and T1-contrast-enhanced MRI

When evaluating accuracy, EfficientNet-B0 showed slightly better performance than ResNet50 across both image modalities. EfficientNet-B0 achieved testing accuracies of 94.80% for T1-contrast-enhanced images and 93.70% for T2 images, compared to ResNet50's accuracies of 92.25 and 89.79%, respectively. These results underline the effectiveness of EfficientNet-B0 in capturing higher level, more discriminative features, making it a stronger model for tumor classification compared to ResNet50.

Combination of radiomic and deep features

Figure 10 illustrates the performance of machine learning models using combined radiomic and deep features extracted from EfficientNet-B0 and ResNet50. This hybrid approach significantly enhanced classification performance for both T2 and T1-contrast-enhanced MRI images. For T2 images, XGBoost with RFE achieved the highest testing AUC of 95.19%, closely followed by LightGBM with RFE, which achieved an AUC of 95.10%. For T1-contrast-enhanced images, XGBoost with RFE again delivered the best performance, with a testing AUC of 96.87%.

Fig. 10.

Performance of models using combined features from T2-weighted and T1-contrast-enhanced MRI

In terms of accuracy, the combination of radiomic and deep features further boosted performance, with XGBoost achieving 94.44% accuracy for T2 images and 96.12% accuracy for T1-contrast-enhanced images. These results highlight the advantages of integrating radiomic and deep learning features, as it captures complementary information that enhances the model’s ability to classify and grade tumors more effectively.

To assess the predictive value of volumetric features alone, we conducted an auxiliary experiment using maximum tumor diameter, tumor volume, and tumor area as input to a Random Forest classifier. This simplified model achieved an AUC of 82.4%, indicating that tumor size and volume are informative for differentiating meningioma grades. However, the performance remained inferior to our full hybrid model, underscoring the added value of integrating radiomic and deep learning features that capture complex imaging characteristics beyond volumetric differences.

Direct and ensemble deep learning models

Figure 11 presents the evaluation of direct deep learning models and their ensembles. EfficientNet-B0 outperformed ResNet50 across both T2 and T1-contrast-enhanced MRI modalities, achieving a testing AUC of 94.16% for T2 images and 94.52% for T1-contrast-enhanced images. However, ensemble deep learning models, which combined the predictions from both EfficientNet-B0 and ResNet50, demonstrated the highest overall performance. For T2 images, the ensemble model achieved a testing AUC of 96.12%, and for T1-contrast-enhanced images, it reached an AUC of 96.80%. These results emphasize the enhanced predictive accuracy of ensemble models by effectively utilizing the strengths of each individual architecture.

Fig. 11.

Performance of end-to-end and ensemble deep learning models using T2-weighted and T1-contrast-enhanced MRI

To visualize and compare classification outcomes from the combined T1 and T2 feature sets, we present a performance heatmap across six ensemble models using two feature selection techniques (RFE and PCA) (Fig. 12). The highest testing AUC (94.99%) and testing sensitivity (93.24%) were achieved using XGBoost with RFE, indicating strong generalizability. While this combined setup marginally outperforms the individual T1- or T2-based models in some cases, the overall performance increase was modest. This can be attributed to redundancy between T1 and T2 features and potential feature noise introduced during concatenation. Nevertheless, combining modalities enhanced model stability and sensitivity, a desirable trait in clinical settings for high-risk lesion detection.

Fig. 12.

Heatmap of classification performance across different ensemble models using combined radiomic and deep features from T1-contrast-enhanced and T2-weighted MRI sequences

Comparison of metrics between T2 and T1-contrast-enhanced images

A comparison of the metrics obtained from T2 and T1-contrast-enhanced images shows that T1-contrast-enhanced images generally lead to better performance across various models. Specifically, the AUC values were higher for T1-contrast-enhanced images, particularly for models like XGBoost and EfficientNet-B0. For instance, XGBoost with RFE achieved an AUC of 96.87% for T1-contrast images, compared to 95.19% for T2 images. Similarly, EfficientNet-B0 with RFE achieved a testing AUC of 94.12% for T1-contrast-enhanced images, while it was 93.17% for T2 images. This pattern was also evident in accuracy, where T1-contrast-enhanced images consistently outperformed T2 images, especially in models like XGBoost and EfficientNet-B0. The superior performance of T1-contrast-enhanced images suggests that the enhanced contrast provides clearer differentiation of tumor characteristics, offering more discriminative features for classification.

Figure 13 displays the ROC curves, highlighting the maximum AUC values achieved by each model for both T1-contrast-enhanced and T2 MRI images. These curves demonstrate the models’ ability to distinguish between low- and high-grade meningiomas, with higher AUC values indicating better performance in classification. Figure 14 showcases the performance of the models after 1000 epochs of training. It illustrates the reduction in loss over time, demonstrating how effectively the models improved their ability to differentiate between different grades of meningiomas. This figure highlights the models’ learning progress and their eventual capability to make accurate predictions based on the MRI images after extensive training.

Fig. 13.

ROC curves showing maximum AUC for each model using T1 and T2 images

Fig. 14.

Training and loss curves of end-to-end networks after 1000 epochs

Figure 15 shows the attention maps for a selection of sample meningioma images. These maps highlight the areas of the MRI images that the deep learning models, such as ResNet50 and EfficientNet-B0, focus on during prediction. By visualizing the attention, we can observe which specific regions within the images the models consider most important when classifying the tumor grades. This helps make the decision-making process of the models more interpretable, providing insights into the key features the models use to differentiate between low- and high-grade meningiomas. The attention maps not only improve the transparency of the models but also help to understand how they analyze and process the images for accurate classification.

Fig. 15.

Attention maps highlighting regions of interest in selected meningioma images

Discussion

In this study, we present a robust framework for preoperative meningioma grading, integrating radiomics feature extraction, deep learning models enhanced with attention mechanisms, and ensemble learning strategies. The model utilizes contrast-enhanced T1 and T2-weighted MRI images to classify meningiomas into low-grade (Grade I) and high-grade (Grade II and III) tumors. The strength of our approach lies in the reproducibility of the extracted features, the integration of both radiomics and deep learning features, the use of ensemble learning techniques, and the advantage of a large and diverse dataset. These factors together ensure the robustness, generalizability, and clinical applicability of our method.

Feature reproducibility

A key aspect of this study is the use of the ICC to assess the reproducibility of radiomic features extracted from MRI images. Segmentation was performed independently by two radiologists, and features with ICC values greater than 0.75 were considered reliable for inclusion in the deep learning models. This rigorous reproducibility assessment ensures that the selected features are stable and reliable across different clinical settings, mitigating the potential variability in radiomics analysis. This approach contrasts with previous studies, such as Zhu et al. [27], which did not prioritize reproducibility in feature selection. By selecting highly reproducible features, our model is more robust when applied to diverse datasets or clinical settings where imaging quality or interpretation may vary.

Integration of radiomics and deep learning features

Our model distinguishes itself by combining radiomics features with deep learning features. Previous studies, such as Hu et al. [28], have demonstrated the utility of multi-parametric MRI radiomics models for meningioma grading, but typically focus on either radiomics or deep learning in isolation. Our framework integrates both, leveraging the strengths of each. Radiomics captures low-level image characteristics, while deep learning excels at extracting high-level, complex patterns that radiomics may miss. This combination allows our model to better address tumor heterogeneity and improve grading accuracy. Furthermore, our results show that this hybrid approach outperforms models relying solely on either radiomics or deep learning. For example, our multi-modal approach, which combines both contrast-enhanced T1 and T2-weighted images, enables the model to capture a more comprehensive representation of tumor features compared to single-sequence models, like the one developed by Chen et al. [29].

Ensemble learning techniques

A significant innovation in this study is the use of ensemble learning techniques in deep learning models. Unlike single-model approaches, which may be prone to overfitting or bias, ensemble learning combines the outputs of multiple models to enhance performance and robustness. By combining the strengths of different architectures, such as EfficientNet-B0 and ResNet50, the ensemble approach reduces the risk of errors from any single model, ensuring more stable and reliable predictions. Our ensemble model outperforms traditional models in terms of predictive accuracy and generalization. For instance, the study by Karabacak et al. [30], which used a radiomics-driven machine learning model with an AUC of 0.838, did not implement ensemble learning. Our ensemble-based model achieved a higher AUC (0.91) in our validation cohort, highlighting the advantage of combining multiple models to capture different aspects of tumor morphology and biology.

Dataset size and diversity

The size and diversity of our dataset were crucial for enhancing model performance. A large and diverse dataset ensures that the model is trained on a wide variety of tumor characteristics and patient demographics. This not only improves generalization but also increases robustness in clinical settings. Our dataset, which includes a large number of patients across different meningioma grades, enables the model to learn nuanced patterns, improving predictive accuracy. Compared to studies like Hamerla et al. [31], which used 138 patients, our larger dataset provides more opportunities for feature exploration and model training, leading to better prediction performance. Our model achieved an AUC of 96.80 in the independent testing cohort, demonstrating the benefits of a large and diverse dataset for accurate and reliable predictions.

Performance comparison with other studies

In comparison with other studies, our model shows superior performance across multiple metrics. For instance, Zhu et al. [27] achieved an AUC of 0.811 using deep learning features alone, while our model, combining both radiomics and deep learning features with ensemble learning, achieved an AUC of 96.80. Furthermore, our model’s sensitivity (95.46%) and specificity were significantly better than those of Zhu et al. [27], who reported sensitivity and specificity values of 89.8 and 76.9%, respectively. Similarly, the multi-parametric model by Hu et al. [28] achieved an AUC of 0.84, and Karabacak et al. [30] reported an AUC of 0.838. Our hybrid model surpasses these figures, ensuring greater clinical interpretability through the inclusion of attention mechanisms, which provide visual insights into the regions of interest the models focus on during predictions. Our results also confirm the intuitive expectation that tumor size and volume contribute meaningfully to grade differentiation. Nevertheless, an independent evaluation using only these features yielded substantially lower predictive performance compared to our integrated framework. This emphasizes the importance of incorporating multiscale image features—such as texture, shape irregularity, and learned deep representations—to comprehensively characterize tumor biology and improve classification accuracy.

Limitations and future directions

While our model demonstrates excellent performance in the validation cohort, several limitations should be considered. First, external validation using more diverse datasets from different clinical settings is needed to assess the model’s generalizability. In addition, the study relied solely on MRI sequences, and incorporating other imaging modalities like PET or CT scans could improve diagnostic accuracy. Another limitation is the potential bias introduced by manual segmentation, despite the use of reproducibility testing with ICC. Future research could explore automated segmentation techniques to reduce variability. Finally, while our model performs well in grading, integrating additional clinical and pathological data, such as biomarkers, may further enhance its prognostic capabilities. Future work should focus on multi-center validation and real-time clinical applications to maximize the model’s impact in practice.

Conclusion

In this study, we proposed an innovative framework for preoperative grading of meningiomas by integrating radiomics features with deep learning. Using contrast-enhanced T1 and T2-weighted MRI images, our model incorporates attention mechanisms to improve prediction accuracy significantly. The ICC was used to assess feature reproducibility, ensuring that only highly reliable features were included in the model. In addition, the application of ensemble learning techniques further enhanced model performance, helping to overcome the limitations of individual models. Our approach outperformed existing models, demonstrating superior interpretability and robustness. The combination of radiomics and deep learning, along with attention mechanisms, allows for better clinical decision-making by providing more transparent insights into the features driving predictions. Compared to previous studies, our model offers a more reliable and clinically applicable solution for meningioma grading. These findings underscore the potential of our framework in enhancing non-invasive grading of meningiomas, making it a promising tool for preoperative diagnosis and clinical decision-making. The integration of diverse imaging modalities and advanced learning techniques positions our model as a powerful asset in the field of neuro-oncology.

Author contributions

R.J.A., D.S., A.Y., M.M.R., B.J., K.J., A.K., W.M.T., M.A., M.J.J., A.M.A., and R.A-S: investigation; project administration; methodology; software; formal analysis; writing—original draft. M.M. and B.F.: conceptualization; data curation; project administration; conceptualization; supervision; writing—review and editing. All authors read and approved the manuscript.

Funding

None.

Data availability

The datasets generated and analyzed during the current study are available from the corresponding author upon reasonable request.

Declarations

Ethics approval and consent to participate

The need for ethical approval was waived off by the ethical committee of Kashan Shahid Beheshti hospital, Kashan, Iran.

Consent to participate

Not applicable.

Competing interests

The authors declare no competing interests.