Application of radiomics to meningiomas: A systematic review

Abstract

Background

Quantitative imaging analysis through radiomics is a powerful technology to non-invasively assess molecular correlates and guide clinical decision-making. There has been growing interest in image-based phenotyping for meningiomas given the complexities in management.

Methods

We systematically reviewed meningioma radiomics analyses published in PubMed, Embase, and Web of Science until December 20, 2021. We compiled performance data and assessed publication quality using the radiomics quality score (RQS).

Results

A total of 170 publications were grouped into 5 categories of radiomics applications to meningiomas: Tumor detection and segmentation (21%), classification across neurologic diseases (54%), grading (14%), feature correlation (3%), and prognostication (8%). A majority focused on technical model development (73%) versus clinical applications (27%), with increasing adoption of deep learning. Studies utilized either private institutional (50%) or public (49%) datasets, with only 68% using a validation dataset. For detection and segmentation, radiomic models had a mean accuracy of 93.1 ± 8.1% and a dice coefficient of 88.8 ± 7.9%. Meningioma classification had a mean accuracy of 95.2 ± 4.0%. Tumor grading had a mean area-under-the-curve (AUC) of 0.85 ± 0.08. Correlation with meningioma biological features had a mean AUC of 0.89 ± 0.07. Prognostication of the clinical course had a mean AUC of 0.83 ± 0.08. While clinical studies had a higher mean RQS compared to technical studies, quality was low overall with a mean RQS of 6.7 ± 5.9 (possible range −8 to 36).

Conclusions

There has been global growth in meningioma radiomics, driven by data accessibility and novel computational methodology. Translatability toward complex tasks such as prognostication requires studies that improve quality, develop comprehensive patient datasets, and engage in prospective trials.

Key Points.

1. Radiomics for meningioma spans several domains, from detection to prognostication.

2. New deep learning architectures are driving radiomics development and performance.

3. Limitations in radiomic study quality are impacting clinical translatability.

Importance of the Study.

Image-based phenotyping through radiomics has redefined the value of medical imaging. Meningioma serves as a prototypical oncologic example demonstrating the impact of combining quantitative imaging features with artificial intelligence methods to enable precision medicine. Linking phenotype with genotype has the potential to become a more accessible source of biomarkers, given limitations in the availability and utilization of genomic/molecular platforms. This study is one of the first to characterize the applications of radiomics to meningioma, capture trends in computational methodologies, and assess barriers to clinical translatability. Our analysis reveals a dichotomization in how radiomics is being conducted, with rapid growth in technical computational development compared to clinically oriented studies. We outline domains in which meningioma radiomic models may be approaching clinical relevance and those that require additional investment. A proactive strategy against siloed technological development as well as revisions to image-based standardization guidelines can contribute to the accelerated clinical integration of radiomics.

Imaging is a cornerstone in cancer care, serving as an invaluable tool for diagnosis and treatment of malignancies.^1,2 Radiomics, a quantitative approach to medical images, is a powerful new technology to phenotype tumors, with the potential to deliver more individualized care. This approach applies mathematical algorithms and artificial intelligence principles to capture complex patterns in tumor images that are imperceptible by the human eye.^3–6 Similar to other big data approaches such as genomics and proteomics, radiomic imaging features may serve as biomarkers to track and influence clinical endpoints.⁷ The potential of radiomics has been demonstrated across multiple cancers including lung, breast, prostate, and nervous system tumors: Radiomic models have been used to determine cancer subtypes, predict molecular and genetic variants, and prognosticate outcomes such as distant metastasis and survival.^8–15

Meningioma serves as an excellent oncologic model system to understand the current utilization and evolution of radiomics in cancer care, given the nuances in diagnosis, risk stratification, and treatment approach of this tumor.^16–18 Clinical decision-making for meningiomas is traditionally driven by the World Health Organization (WHO) grade.¹⁹ However, significant variability in tumor behavior is sometimes observed in this grading scheme, with almost 20% of low-grade meningiomas having an early recurrence while certain high-grade meningiomas exhibit indolent behavior and may not recur.^20–23 This has prompted interest in using molecular characteristics and growth rates to guide treatment.^24–27 In practice, molecular platforms and testing are not universally accessible, add cost, and may capture only a fragment of a heterogeneous tumor.^28,29 In comparison, imaging is far more ubiquitously available for clinicians treating brain tumors. Image-based phenotyping that reflects the biological characteristic and molecular signature of tumors is therefore poised to redefine how precision care can be delivered to patients if such methods can achieve high reliability. We examine the diverse applications for radiomics to meningiomas, analyze how radiomic models are constructed and validated, quantify the performance of radiomic approaches, and characterize barriers to clinical translatability.

Materials and Methods

Search Strategy and Study Selection

We searched 3 databases, PubMed, Embase, and Web of Science to aggregate all radiomic analyses applied to meningioma up to December 20, 2021. Detailed search terms can be found in Supplementary File 1 and preferred reporting items for systematic reviews and meta-analyses guidelines were followed.³⁰ Inclusion criteria were: (1) any patient population with a meningioma, (2) quantitative feature extraction and analysis of intracranial meningioma MRI or CT images, and (3) model developed on at least 5 images. Case reports as well as in vitro and animal studies were excluded. An institutional review board review was not required for this study.

Two reviewers (RVP, SY) independently screened abstracts and corresponding full texts through 2 rounds, with all disputes resolved through a third party (WLB).

Data Extraction

Qualitative and quantitative variables pertaining to meningioma radiomics were decided a priori and iterated on during data extraction. In publications with multiple experimental aims, each experiment was analyzed separately. Two reviewers (RVP, SY) independently subdivided experiments as clinical versus technical, with clinical experiments focused on: (1) applicability to patient populations, (2) discussion of clinical impact, and (3) developed for translatability and technical experiments focused on: (1) computational model development, (2) hyperparametric optimization of models, and (3) emphasis on model methodology. Data on experiment aims, global locations, datasets utilized, imaging modalities, computational model methodology, and performance statistics were extracted. Publication citations were quantified using Google Scholar on July 3, 2022.

Quality Assessment

We assessed the methodologies, strengths, limitations, quality, and translatability of included publications using the radiomics quality score (RQS). The RQS is a validated tool developed to standardize evaluation criteria of radiomics investigations, focusing on image acquisition protocols, model development/validation, and data accessibility.³¹ It is composed of a 16-category inventory with a total possible score between −8 and 36. Two reviewers with experience in radiomics (RVP, SY) independently scored each experiment using RQS criteria. RQS scores were averaged in cases where scoring was discrepant by ≤2 points. If RQS scoring was discrepant by >2 points, conflicts were resolved through a third party independent of the scoring process (WLB).

Statistical Analysis

Quantitative data aggregation and statistics were performed in R (R Foundation for Statistical Computing 3.6.1) and Python (Spyder 4.1.5, Python 3.8.1) software. Chi-squared and Student’s t-test were used to compare experiment characteristics, RQS, and radiomic model performance metrics for clinical versus technical papers. Performance metrics were compiled only for experiments reporting meningioma-specific data. All statistical tests were 2-tailed and P < .05 was considered significant.

Results

Our search yielded 13 371 unique articles, with 170 publications containing 190 individual experiments meeting inclusion criteria (Figure 1A; Supplementary File 2; Table 1). We compartmentalized radiomic applications to meningiomas into 5 categories: Meningioma detection and segmentation (21%), classification of meningiomas versus other neurological pathology (54%), meningioma grading (14%), radiological correlation of meningioma tumor features (3%), and prognostication of meningioma outcomes (8%, Figure 1B).

Figure 1.

(A) Preferred reporting items for systematic reviews and meta-analyses flowchart of study design. (B) Applications of radiomics to meningiomas. (C) Number of experiments per year, subdivided by type of analysis. (D) Global heatmap of publications on meningioma radiomics.

Table 1.

Characteristics of Meningioma Radiomic Experiments

	Clinical (n = 52)	Technical (n = 138)	Total (n = 190)	P-value
Datasets
Institutional, n (%)	50 (96)	54 (39)	104 (55)	<.001
Public, n (%)	2 (4)	82 (59)	84 (44)
Institutional + Public, n (%)	0 (0)	2 (2)	2 (1)
Images*
Institutional, median (range)	194 (43–10,038)	100 (20–9,000)	108 (20–10,038)
Public, median (range)	143	708 (5–865)	708 (5–865)
Cases*
Institutional, median (range)	134.5 (8–1,278)	50 (1–5,088)	123 (1–5,088)
Public, median (range)	35	82 (15–82)	82 (15–82)
Experiment aim
Detection, n (%)	7 (13)	33 (23.9)	40 (21)	<.001
Classification, n (%)	7 (13)	95 (68.8)	102 (54)
Grading, n (%)	19 (37)	8 (5.8)	27 (14)
Feature correlation, n (%)	4 (8)	2 (1.5)	6 (3)
Prognostication, n (%)	15 (29)	0 (0)	15 (8)
Models
Unsupervised ML, n (%)	0 (0)	11 (8)	11 (6)	<.001
Supervised ML, n (%)	33 (64)	50 (36)	83 (44)
Deep learning, n (%)	9 (17)	66 (48)	75 (39)
Regression, n (%)	8 (15)	7 (5)	15 (8)
Other, n (%)	2 (4)	4 (3)	6 (3)
Meningioma sub-analysis performed†, n (%)	52 (100)	93 (67)	145 (76)	<.001
Studies with training and testing datasets, n (%)	35 (67)	95 (69)	130 (68)	P = .791
Radiomics quality score, mean ± SD (mean % total score)	8.3 ± 6.6 (37)	6.1 ± 5.4 (32)	6.7 ± 5.9 (33)	P = .037

^*Meningioma-specific data only included for number of images and cases.

^†Proportion of experiments reporting meningioma-specific radiomic performance data if dataset includes multiple tumor types. Chi-squared and student’s t-test used.

Meningioma radiomics studies have undergone exponential growth, with the number of experiments published in 2021, 15 times that of 2015. Experiments focusing on the classification of meningiomas have experienced the fastest growth while those studying feature correlation and prognostication have had a more recent increase in investigation (Figure 1C). These studies are being conducted globally: India, China, and the United States have the most publications at 43, 40, and 12, respectively (Figure 1D).

Variations in Data Access and Use

We observed a dichotomization of imaging data sources used to create meningioma radiomics models (Figure 2A). Institutional data defined as private datasets collected at individual academic institutions represented 46% of datasets utilized. 4% of studies used multi-institutional datasets. Public datasets, open-source repositories of data, were used 49% of the time. The largest public dataset utilized was one collected between 2005 and 2010 at Nanfang Hospital, Guangzhou, China and Tianjian Medical University General Hospital, China (Nanfang/Tianjian, 26%). The Nanfang/Tianjian dataset is composed of 3064 T1-contrast MRI images of brain tumors (including 708 images from 82 patients with meningiomas) and manual segmentation masks of tumor regions.³² Additional public datasets included the Harvard Medical School Whole Brain Atlas (HMS, 7%),³³ Multimodal Brain Tumor Image Segmentation Benchmark (BraTS, 7%),³⁴ and the cancer imaging archive (4%).³⁵ Notably, only 2 experiments (1%) drew from both public and institutional datasets for analyses.

Figure 2.

(A) Imaging datasets used in included experiments. (B) Global heatmap of imaging datasets, shading reflects fraction of public versus institutional datasets used (Institutional: single and multi-center). (C) Imaging modalities used—T1 (64%): T1, T1C, MP-RAGE, T1-Subtraction, T2 (27%): T2, fluid attenuated inversion recovery, gradient recalled echo, Diffusion (8%): diffusion weighted image, apparent diffusion coefficient, diffusion tensor image, diffusion kurtosis image, Other (1%): CT, proton density, susceptibility weighted image. (D) Meningioma images and patient cases per study (2 experiments used a combined public/institutional dataset—binned as public or institutional based on which dataset contributed a larger number of images). (E) Statistical and analytical models used. (F) Distribution of clinical versus technical experiments, datasets, and models used.

The utilization of institutional and public datasets for radiomic studies varied geographically. Publications from China utilized institutional datasets 4 times more than public datasets (32 versus 8 publications). In the United States and India, there was a relatively even split in utilization of the 2 data sources. In developing countries such as Iran, Turkey, and Pakistan, investigators were more likely to use public datasets (Figure 2B).

Coupled with the variation in dataset use, there was a wide range in the number of meningioma patient cases and corresponding images used to develop radiomic models. A median of 82 patient cases (range: 1–5088) and 309 patient images (range: 5–10 038) were used, driven by differences in institutional and public datasets (Figure 2D). Institutional datasets had a median of 123 patient cases (range: 1–5088) and 108 patient images (range: 20–10 038) while public datasets had a median of 82 patient cases (range: 15–82) and 708 patient images (range: 5–865, Table 1). The median number of patient cases and images in public datasets can be attributed to frequent use of the Nanfang/Tianjian dataset composed of 82 meningioma cases and 708 meningioma images. Data augmentation techniques such as image translation, shear, rotation, and brightness were used in 15% of experiments, contributing to the difference in the number of meningioma cases versus images.³⁶

Meningioma radiomic models largely used the same imaging modality, with 99% (188/190) of experiments using MRI sequences or techniques. Two studies incorporated CT images alongside MRI sequences. T1-weighted MRI sequences (T1 non-contrast, T1-contrast, T1 3D-gradient echo, and T1 subtraction map) were the most commonly used sequence to extract radiological features (64%). This was followed by T2-weighted sequences (27%: T2, T2 fluid-attenuated inversion recovery, T2 gradient recalled echo), diffusion sequences (8%: diffusion-weighted image, apparent diffusion coefficient image, diffusion tensor image, diffusion kurtosis image), and other (1%: CT, susceptibility-weighted image, proton density). While 74% (142/190) of meningioma radiomic models were developed on one imaging sequence, 26% used multi-sequence integration, reflecting the rise of computational methods able to handle multimodal data. T1 sequences were most frequently used in isolation while T2 and diffusion sequences were often studied in conjunction with other MRI sequences (Figure 2C).

Evolution of Computational Models

Many different models have been applied to process meningioma radiomics data driven by the development of advanced computational approaches in the last 2 decades (Figure 2E). Supervised machine learning methods, such as random forest, least absolute shrinkage and selection operator, and support vector machine were the most frequently used model method (44% of experiments). Deep learning was used in 39% of experiments and spanned architectures including custom convolutional neural networks, AlexNet, VGG, GoogLeNet, Inceptionv3, ResNet, PSPnet, and DenseNet. Additional methods included linear and multivariable regression (8%), unsupervised machine learning (6%), and other statistical approaches such as histogram analysis (3%). Since 2017, there has been a rapid uptake of deep learning for radiomics, with 63% of experiments in 2021 utilizing a deep learning architecture.

Trends in Radiomics Research Focus

After binning experiments based on the primary study motive, 73% of experiments had a technical focus compared to 27% with a clinical focus. Study objective was associated with type of dataset used, with 96% of clinical analyses using institutional data versus 40% of technical experiments. Furthermore, there was a significant difference in radiomic model architectures developed in technical versus clinical experiments. Clinical analyses applied supervised machine learning (64%) more than deep learning (17%) and regression (15%). In comparison, technical experiments were heavily enriched in deep learning models developed on both institutional and public datasets (48%) versus supervised machine learning (36%) and regression (5%) models (Figure 2F; Table 1).

The growth in meningioma radiomics appears to be driven by technical experiments, with 70% of published studies in 2021 being classified as technical (Figure 3A). Both clinical and technical publications were cited at roughly the same rate, averaging 10 citations per year. However, citations of technical studies were skewed right, with some reaching greater than 70 citations per year (Figure 3B).

Figure 3.

(A) Clinical and technical experiments over time. (B) Citations per year of clinical and technical publications. (C) Reported statistical data across meningioma radiomics categories (only studies with meningioma specific data included), P < .05 (*), P < .01 (**).

Meningioma Radiomics Model Performance

Performance of meningioma radiomic models was evaluated across radiomic applications and clinical versus technical studies. Metrics from the 33% of technical studies which did not report meningioma-specific data were not included in the final synthesis of performance (Figure 3C; Table 1).

For models developed to detect and segment meningiomas, the average accuracy was 93.1 ± 8.1%, (range 73%–100%). Average dice coefficient, a measure of similarity to ground truth detection/segmentation, was 88.8 ± 7.9% (range 69.7%-100%). Clinical versus technical studies did not differ significantly in accuracy or dice coefficient.

Classification of meningiomas occurred against tumor and non-tumor nervous system pathologies. In total, 50% of all classification experiments assessed the differentiation of meningioma from glioma and pituitary tumors. The mean classification accuracy of radiomic models was 95.2 ± 4.0% (range 81.9%–100%). Technical experiments had significantly higher classification accuracy (95.7 ± 3.7%) compared to clinical experiments (89.8 ± 3.2%, P < .01). Classification sensitivity was 91.0 ± 9.1% (range 52.3%–100%) and specificity was 94.5%±9.1% (range 58.3%–100%).

Grading of meningiomas sorted tumors by WHO grading criteria.¹⁹ 74% of models sorted meningiomas into low (WHO grade 1) versus high (WHO grade 2–3) grade; 15% distinguished between each WHO grade, and 11% partitioned meningiomas between grade 1 and 2. The mean area-under-the-curve (AUC) for meningioma grading was 0.851 ± 0.078 (range 0.690–0.970). Grading sensitivity was 79.7 ± 12.4% (range 54%–100%) and specificity was 84.0 ± 11.3% (range 62.5%–100%). Clinical experiments had a significantly higher grading specificity (87.3 ± 9.2%) compared to technical experiments (71.8 ± 10.4%, P < .05).

Radiomic models developed for feature correlation focused on meningioma firmness, fibrous quality, and Ki-67 proliferative index. The mean AUC across these domains was 0.89 ± 0.07 (range 0.83–0.97) with a sensitivity of 92.9%±7.9% (range 82.6%–100%) and specificity of 81.2% ± 6.5% (range 75%–88%).

Prognostication of meningioma clinical course included predicting tumor brain invasion, treatment failure, post-operative cerebral edema, post-operative seizures, tumor recurrence, and survival. Across these heterogeneous aims, the mean AUC was 0.83 ± 0.08 (range 0.71–0.99) with a sensitivity of 77.8 ± 11.3% (range 61.5%–90%) and specificity of 75.4 ± 13.0% (range 57%–88.2%). A breakdown of performance metrics for studies performing feature correlation and prognostication can be found in Supplementary File 3.

Quality Assessment

Despite the rising number of publications, the overall quality of meningioma radiomics studies remained low, with an average RQS score of 6.7 ± 5.9 points (range: −8–20), equivalent to 33% of the total possible score. There was no significant difference in RQS scores across meningioma radiomic applications and experiments which used public versus institutional datasets (Figure 4A).

Figure 4.

(A) Radiomics Quality Score (RQS) by study class (institutional data only for grading, feature correlation, and prognostication). (B) RQS for clinical versus technical studies, P < .05 (*). (C) RQS breakdown by percent of points earned in each category.

Radiomic experiment quality varied between clinical and technical experiments. Clinical analyses had a significantly greater average RQS of 8.3 ± 6.6 (range: −8–20) compared to technical analyses at 6.1 ± 5.4 (range: −7–14, P = .037) (Figure 4B, Table 1).

The RQS score breakdown highlights areas in which meningioma radiomic experiments excelled as well as areas for improvement (Figure 4C). Many experiments documented imaging protocols (87% points earned) and had multiple segmentations of tumor volumes (61% points earned). During model development, experiments did well in reducing radiological features to avoid overfitting (77% points earned) and reporting discrimination statistics such as AUC values (53% points earned).

Validation of radiomic models on testing datasets showed moderate quality (31% points earned). While 68% of experiments included training and testing data, many derived this split using one dataset or overlapping sampling, impacting the RQS score (Table 1).

Poor RQS performance was concentrated in categories that address data heterogeneity, model verification, and clinical translatability. Few experiments utilized multiple scanners to address inter-scanner differences (2% points earned), and none had images from individual patients at multiple time points. While many experiments extracted radiological features, they were rarely combined with biological or clinical variables (11% points earned). Many models did not describe clear statistical cutoffs (28% points earned) or calibration statistics (4% points earned). Very few experiments discussed biological correlates underlying their results (21% points earned), assessed clinical applicability/utility of their models (3% points earned), or compared current gold standard technologies (0.5% points earned). No experiments were conducted as prospective trials and no cost-effectiveness analysis was performed. Finally, imaging data, tumor segmentations, and code utilized to create radiomic models were frequently held privately, impacting open access sharing of methodology and models (20% points earned).

Discussion

Quantitative analysis of radiological images is an emerging force in oncology, with the potential to transform clinical management.³⁷ There has been exponential growth in global engagement in radiomics, with groups leveraging large institutional and public datasets to uncover valuable tumor radiological features. Such image-based phenotyping may augment decision-making, from tumor detection, classification, grading, feature prediction, and prognostication. While emerging radiomic approaches hold great potential toward accessibility and applicability, the pattern of growth also highlights challenges, previously seen in other “-omics” efforts like genomics, that are roadblocks to translation.³⁸

As meningioma radiomics evolved over the past 15 years, 2 distinct research aims emerged: Technical and clinical. Technical studies focus on the creation of computational models, using the rich data in radiological images of meningiomas to train and fine-tune these technologies. In contrast, clinical studies focus on the utility of radiomics, assessing model development through the lens of therapeutic relevance and impact. The difference in research aims tailors technical and clinical approaches to distinct radiomic applications. A majority of technical studies focused on meningioma detection/segmentation and brain tumor classification, tasks that take advantage of computer vision and high-throughput data pipelines. Clinical studies were more likely to address grading, feature correlation, and prognostication, areas that require more complex clinical data that reflects the heterogeneity of patient populations and treatment plans. With 73% of published experiments taking a technical approach, meningioma radiomic efforts are skewed toward detection, segmentation, and classification. It is important to ensure this divergence does not lead to the development of powerful computational models which have a limited path to clinical application.³⁹

The availability of public datasets of radiological images has helped drive growth in meningioma radiomics, especially with technical studies. About half of the reported studies were performed on public datasets, with a quarter using a single public dataset: Nanfang/Tianjian.³² Nanfang/Tianjian was unique as it included meningioma tumor masks, eliminating a time-consuming data processing step for users. Though public datasets can be an important source to augment institutional data, this was not observed in the literature. Only two publications developed models on both institutional and public data.^40,41 RQS subcategories reinforced this observation, with a limited number of studies creating or utilizing open-access data. Integrating multiple data sources can introduce data heterogeneity and grow training/testing splits which improves the validation process for radiomic features.⁴² However, a limitation in public datasets surrounds the variables available. Semantic features such as patient age, gender, presentation, and treatment are often excluded. Additionally, there is a discrepancy between the number of meningioma patient cases and images found in these datasets. The Nanfang/Tianjian dataset has an average of 8.6 images per patient, far fewer than expected for a high-fidelity MRI sequence including thin slices or wider brain coverage.⁴³ Public datasets are therefore a valuable source for technical development but make it difficult to create more clinically relevant models. This is evident given the sources of data used in clinical radiomic experiments, with 96% of data coming from institutional sources compared to 4% from public datasets.

There has been a significant shift in how meningioma radiomics is being performed. Classically, radiomics involves extracting quantitative features from a segmented tumor image, selecting features of interest, and correlating features with a particular endpoint.⁴² Regression and machine learning models use these selected radiomic features as inputs, requiring structured pre-defined data. Deep learning has changed this workflow as it is not dependent on contoured tumor regions or manual feature selection.⁴⁴ Rather, it operates as a “black box” without human intervention, developing relationships between features that are not easily traceable to produce a given outcome. Deep learning for meningioma radiomics was first published in 2017, a timeframe that coincides with the release of novel computer vision architectures such as AlexNet (2012), VGG16 (2014), and GoogLeNet (2014).⁴⁵ Compared to traditional supervised and unsupervised machine learning methods, these platforms push the boundaries in computational efficiency, data throughput, and performance—areas that have led to deep learning outpacing all other computational methods for meningioma radiomics. However, the “hidden features” of deep learning are fundamentally different from the radiomic features and reporting framework previously established through the image biomarker standardization initiative.⁴⁶ Given the pace of deep learning implementation, this evolution in radiomics requires new guidelines to ensure reproducibility and clinical validity.

The assessment of meningioma radiomics quality reveals opportunities for future development. The overall RQS was 33%, with no difference across meningioma radiomic applications. Radiomic studies generally had high-quality imaging protocols, performed feature reduction, and included validation datasets. The RQS analysis indicates that there is a significant global effort being applied to meningioma radiomics with several barriers that impact translatability. Many studies did not develop models with clinically heterogenous data representative of patient populations. Very few studies accounted for inter-scanner differences and clinical variables connected to patient presentation. This has historically been a challenge preventing translation of radiomics: Models become tuned experimentally but underperform on external data.⁴⁷ Along with this, reporting of radiomics statistical data have been irregular. Without clear calibration guidelines, image sequence sensitivity analysis, and performance metrics, it is difficult to test, validate, and iterate on published studies. These areas have consistently limited the quality of radiomics evidence, with no significant improvement in RQS scores over time (Supplementary File 4).

Some of the challenges in image heterogeneity and data reporting can be overcome by creating models on multi-institutional datasets that account for variability in acquisition protocols, scanner quality, and patient sampling. Effective multi-institutional collaboration also requires innovative solutions that overcome the barriers and cost of data sharing. Federated learning is one such paradigm that facilitates secure decentralized sharing of medical image data, enabling computational models to be trained on diverse data streams.⁴⁸ Tools like this will be instrumental to carry out prospective randomized trials with radiomics-based clinical decision support models: The gold standard for evidence that has not been performed for meningioma radiomics to date.

Nevertheless, meningioma radiomic models have achieved a remarkable level of performance and demonstrate potential as a tool to non-invasively augment clinical decision-making. Models developed for meningioma detection/segmentation and classification are achieving high accuracy at 93.1% and 95.2%, respectively. Performance in this range is critical for any future radiomics pipeline looking to process large volumes of data for clinical questions. However, when performance data are aggregated, there are interesting trends that emerge. Many technical experiments report accuracies, sensitives, and specificities at 100%, which may reflect overfitting. This level of performance likely poses an unrealistic expectation for real-world workflows. As an example, in radiomic performance on meningioma classification, technical studies reported significantly higher accuracy at 95.7% compared to clinical studies at 89.8%. When more heterogenous variables and images are introduced in clinical studies, models do not perform as well. This trend continues in radiomic feature correlation and prognostication compared to detection/segmentation and classification. As the radiomic task increases in clinical heterogeneity and complexity, there is a further reduction in AUC, sensitivity and specificity. While clinically useful radiomics requires models with reliably high performance, radiological features of meningiomas must be integrated with clinical variables to account for the breadth and depth of patient presentations.

There are several limitations of this analysis, some of which are inherent to retrospective reviews. While we reported aggregate performance data, we were unable to apply additional meta-analytical statistics due to poor reporting of primary data. There was variability in which performance metrics were reported, with many studies publishing point data without confidence intervals. This analysis also approaches radiomics from the perspective of radiological features and model development. A more detailed clinical needs assessment is required to support future prospective radiomic trials and technological integration. Finally, for meningioma tumor feature correlation and prognostication, our conclusions were limited by the relatively low number of published studies, making it difficult to accurately assess the trajectory and implications of radiomics in these areas. Collectively, we demonstrate the potential for radiomics in oncology and propose concrete, readily achievable, areas of improvement to accelerate the integration of artificial intelligence-based image phenotyping in clinical settings.

Funding

This work was supported by the Courtney Meningioma Research Fund.

Conflict of interest statement

The authors declare no conflicts of interest.

Authorship statement

Study conception, design, and search strategy: RVP, SY, RYH, WLB. Literature search, screening, and data extraction: RVP, SY. Study screening conflict resolution: RYH, WLB. Data analysis, interpretation, manuscript production: RVP, SY, RYH, WLB. All authors had full access to data included in this manuscript and approved submission.