Abstract
Simple Summary
Identifying GBM patients with very short survival could contribute to adapting the therapeutic approach. According to our results, high-precision models can be elaborated using basic MRI sequences available at any center, combined with advanced image analysis. Although there are several previous publications related to this topic, a survival threshold that may be clinically relevant has not been proposed. The importance of our study lies in selecting patients with total or near-total resection, the short-term survival end-point applied of six months, and the employment of user-friendly software that allows clinicians to explore new statistical methodologies and carry out complex tasks such as the extraction of radiomic features. Promoting the use of these technological tools will motivate other clinical researchers to get involved and take advantage of radiomics and artificial intelligence, tools that have come to reinforce our analytical capacity.
Abstract
Radiomics, in combination with artificial intelligence, has emerged as a powerful tool for the development of predictive models in neuro-oncology. Our study aims to find an answer to a clinically relevant question: is there a radiomic profile that can identify glioblastoma (GBM) patients with short-term survival after complete tumor resection? A retrospective study of GBM patients who underwent surgery was conducted in two institutions between January 2019 and January 2020, along with cases from public databases. Cases with gross total or near total tumor resection were included. Preoperative structural multiparametric magnetic resonance imaging (mpMRI) sequences were pre-processed, and a total of 15,720 radiomic features were extracted. After feature reduction, machine learning-based classifiers were used to predict early mortality (<6 months). Additionally, a survival analysis was performed using the random survival forest (RSF) algorithm. A total of 203 patients were enrolled in this study. In the classification task, the naive Bayes classifier obtained the best results in the test data set, with an area under the curve (AUC) of 0.769 and classification accuracy of 80%. The RSF model allowed the stratification of patients into low- and high-risk groups. In the test data set, this model obtained values of C-Index = 0.61, IBS = 0.123 and integrated AUC at six months of 0.761. In this study, we developed a reliable predictive model of short-term survival in GBM by applying open-source and user-friendly computational means. These new tools will assist clinicians in adapting our therapeutic approach considering individual patient characteristics.
Keywords: glioblastoma, radiomics, texture analysis, survival, machine learning
1. Introduction
Glioblastoma (GBM) continues to be the most threatening primary brain neoplasm, with a median survival of approximately 15 months [1]. Currently, despite the standard treatment that includes maximum safe surgical resection followed by adjuvant chemoradiation therapy [2,3], its prognosis remains ominous, and our knowledge of this neoplasm is still limited.
Predicting a patient’s survival is of vital importance for determining the ideal choice of treatment and management. Currently, several prognostic factors are commonly used to predict the prognosis of these patients, including age, sex, Karnofsky performance status (KPS), molecular profile, extent of resection, preoperative tumor volume, volume of non-enhancing tumor and degree of necrosis [4]. However, some of these features depend on radiologists’ interpretation, which justifies the increasing need for an unbiased and quantitative radiological evaluation.
Magnetic resonance imaging (MRI) plays a fundamental role in neuro-oncology for diagnosing and assessing response to treatment and is being increasingly used as a non-invasive predictive tool. On the other hand, the term "radiomics" refers to the process of obtaining quantitative features based on the intensity, volume, shape and texture variations of the radiological images and creating algorithms that find the association of these variables with the survival and outcome of the patients [5]. Through radiomics, converting medical images into high-dimensional data allows us to expose the underlying pathophysiology, especially intratumor heterogeneity [6]. This extraction process captures tumor characteristics undetectable to the human eye and gives added value to clinical visual perception. Radiomics incorporate several essential disciplines, including radiology for image interpretation, computerized vision for extracting quantitative variables, and machine learning for classification and regression tasks [7]. Such integration has been demonstrated to exceed expert human abilities in multiple tasks, including diagnosis and outcome prediction.
Recognizing patients who would not benefit from standard treatment and identifying those who need a more aggressive approach at the time of diagnosis is essential for managing GBM through personalized medicine [8]. There are several publications that, through the integration of radiomics and artificial intelligence, seek to establish survival prediction models in GBM based on preoperative MRI [8,9,10,11,12,13,14]. In the vast majority of studies, patients are classified according to their survival into two or three categories, depending on whether they exceed 10 or 15 months of survival. This approach aims to identify medium- and long-term survivors who could theoretically be subsidiaries of aggressive therapies. Furthermore, in most studies, the extent of resection is not used as a discriminatory factor, and biopsies and partial and subtotal resections are included.
This fact precludes the implementation of such predictive models in newly diagnosed GBM patients. Our study aims to use the radiological characteristics from structural preoperative multiparametric magnetic resonance imaging (mpMRI) to construct a predictive model of short-term survival in patients in whom total or near-total resection of the enhancing tumor has been performed followed by standard treatment.
2. Materials and Methods
2.1. Study Population
A retrospective collection of patients who underwent surgery with a diagnosis of GBM was carried out in two institutions between January 2019 and January 2020. In addition, a second cohort of patients was selected from available public databases: the BraTS (Multimodal Brain Tumor Segmentation) Challenge 2020 [15,16,17], and three other sources available through The Cancer Image Archive [Ivy Glioblastoma Altas Project (Ivy -GAP) [18], the National Cancer Institute’s Clinical Proteomic Tumor Analysis Consortium Glioblastoma Multiforme (CPTAC-GBM) and The Cancer Genome Atlas Glioblastoma Multiforme (TCGA-GBM) [19]]. The inclusion criteria were pathologically confirmed glioblastomas, availability of preoperative MRI with structural/conventional sequences [T2-weighted images (T2WI), fluid-attenuated inversion recovery (FLAIR), T1-weighted images (T1WI) and contrast-enhanced T1-weighted images (T1CE)] with adequate resolution, known survival status, and clinical information (age and type of surgical resection). Only those cases in which gross total resection (100% of the enhancing tumor volume) or near-total resection (>95% of the enhancing tumor volume) were included. Patients were randomly allocated into training and testing data sets following a proportion of 70/30. The primary endpoint was overall survival (OS), which was defined as the number of days from the initial pathological diagnosis to death (censored = 1) or the last date that they were known to be alive (censored = 0). Public data sets do not have patient identifiers, hence, no institutional review board approval was required. Nevertheless, the study was approved by the institutional review boards and ethics committees of the other two participating centers. Additionally, all institutional patients provided written informed consent. The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments.
2.2. Image Data Description and Preprocessing
All BraTS scans were acquired with different clinical protocols and various scanners from multiple (n = 19) institutions. Details of the protocol acquisition of the scans from TCIA and institutional cases are shown in Table S1.
Image pre-processing consists of several steps. First, mpMRI scans were converted to Neuroimaging Informatics Technology Initiative (NifTI) format. Then, the scans were placed in a common orientation [“LPS” (left-posterior-superior) in the radiological convention or “RAI” (right-anterior-inferior” in the neurological convention)]. Later, the scans for every subject were registered to SRI24 anatomical atlas space [20]. N4 bias correction [21] was applied as a temporary step to facilitate optimal registration but was not included at the end of the process since it might obliterate the MRI signal, particularly on the FLAIR modality [17].
The T1W1, T2WI and FLAIR scans were registered to the transformed T1CE scan, resulting in coregistered resampled volumes of 1 × 1 × 1 mm isotropic voxels. The brain was then extracted from all co-registered scans using a pretrained deep learning-based model [22]. Finally, intensity Z-scoring normalization was carried out. All pre-processing pipelines were generated using The Cancer Imaging Phenomics Toolkit (CaPTk) [23].
2.3. Tumor Segmentation and Feature Extraction
The method used to generate the segmentation labels of the different tumor regions is called GLISTRboost (Boosted GLioma Image SegmenTation and Registration) [24], which is defined as a hybrid generative-discriminative tumor segmentation method. This segmentation algorithm comprises a glioma growth model, a discriminative part based on a gradient boosting multiclass classification scheme and a Bayesian strategy [14]. Segmentation labels or volumes of interest (VOIs) were as follows: enhancing tumor (ET), non-enhancing tumur/necrosis (NET), and edema (ED). Segmentations were evaluated by two experts (S.C., S.G.-G.) and corrected manually if necessary.
Using the extraction tool of CaPTk, a total of 15,720 characteristics were computed from the tumor subregions (i.e., ET, NET and ED) and the four mpMRI modalities following the Image biomarker standardization initiative (IBSI) [25] definitions. These extracted features included intensity features or first-order statistics, histogram-based features, and volumetric, morphologic and textural features, including those based on the gray level cooccurrence matrix (GLCM), gray level run-length matrix (GLRLM), neighborhood gray-tone difference matrix (NGTDM), gray level size-zone matrix (GLSZM) and lattice-based features. A detailed description of these characteristics is shown in Table S2.
2.4. Data Processing and Feature Selection
After the extraction of the radiomic features, the data were pre-processed by data cleaning (removing the variables with more than 5% missing values) and imputation (using the average value method). Afterward, 13,265 features were z-score normalized based on the mean and standard deviation. Then, a feature selection process was necessary to reduce these high-dimensional imaging features to avoid overfitting. Feature selection helps to optimize the generalizability and reproducibility of the models subsequently built. The feature selection process was performed using the training data set exclusively. A two-step selection method was used as follows. Spearman’s correlation coefficient was calculated for each pair of radiomic features. Then, the features with Spearman’s correlation coefficient > 0.95 with each other were discarded, retaining a single feature in each set. Later, features were reduced by including only the variables with a significance <0.001 with the OS in days. Thus, the number of features was reduced to a set of 260.
2.5. Statistical Analysis and Machine Learning
Predicting OS was achieved by two different strategies. The first is a binary classification task between short and long survival. For this purpose and after feature reduction, several classification algorithms were assessed for patient stratification. As a previous step, machine learning (ML)-based filters were used: Gini index (GINI), fast correlation-based filter (FCBF) and information gain (InfoGain). Hence, the top 10 features were selected. Then, six different ML classifiers were trained: logistic regression (LR), naive Bayes, k-nearest neighbors (kNN), random forest (RF), support vector machine (SVM) and a multilayer perceptron algorithm (neural network -NN). We used the default settings of the hyperparameters detailed in the Table S3. The target response for each model was the patient’s OS grouped into two classes to distinguish patients who survived <6 months (short-term survivors) from others. Then, the results were quantitatively validated on the testing data set. The performance of the ML classifiers was measured by the area under the receiver operating characteristic curve (AUC), accuracy, precision, F1 score and recall. All performance metrics were reported as the average value over classes.
The second statistical strategy was conducted using the random survival forest (RSF) approach from the R package “randomForestRSC” [26], an ensemble-tree method that adapts random forests to right-censored data and survival analysis. RSF does not rely on restrictive assumptions such as proportional hazards and automatically handles nonlinear effects and interactions of high-dimensional data. Features were ranked by positive importance using a variable hunting algorithm as a feature selector. Model hyperparameters were as follows: number of trees = 500, node size = 2, number of splits = 10 and log-rank as the splitting rule. Training predictions were performed using 5-fold cross-validation. We also evaluated the model’s ability to generalize those predictions on the testing group.
When the primary outcome is survival (time to event), RSF produces a cumulative hazard function (CHF) from each decision tree that is averaged in an ensemble out-of-bag CHF (OOB-CHF). The predicted ensemble mortality is the mean OOB-CHF estimated by the RSF model for each subject, and it was used to calculate each patient’s estimated mortality risk. We used the results from the RSF model to build a mortality risk score and split the sample into high- and low-risk groups. The OOB-CHF cut-off values defining the risk groups were calculated through the “cutp” function of the “survMisc” package [27]. The log-rank test was used to compare the survival Kaplan–Meier curves between the patients in the high- and low-risk groups.
Finally, Cox proportional hazard regression models were fitted to the training data set using the dichotomized risk score (high- and low-risk groups) from the RSF model as an explanatory variable, the patient’s age and a combination of both. Then, the models were validated in the testing data set. The performance assessment of the survival models was performed by calculating the prediction errors using the integrated Brier score (IBS) defined as the average squared distances between the observed survival status and the predicted survival probability by the “pec” package [28]. Additionally, the discriminatory capacity of the model was evaluated by calculating the concordance index (CI), which refers to the probability that, for a pair of randomly allowed samples, the sample with the highest risk prediction experiences an event before the sample with the smallest risk. Furthermore, the integrated area under the time-dependent ROC curve (iAUC) was calculated for all models using the “risksetROC” package [29]. The standard approach of ROC curve analysis considers event (death) status and predictor value for an individual as fixed over time. Because the status and explanatory variables change over time, we used the risksetROC package that estimates the iAUC under incident sensitivity and dynamic specificity definition and produces accuracy measures for censored data under proportional or nonproportional hazard assumption of Cox regression estimator [30]. Following the objective of our study, we also calculated the iAUC at six months for all models.
Statistical and survival analyses were performed with R version 4.0.5 (R Foundation for Statistical Computing, Vienna, Austria). The differences in age, OS, the proportion of right-censored cases and short-term (<6 months) survival cases were assessed using Student’s t-test, Mann–Whitney U test and two-proportions Z-test, respectively. For the binary classification model, we used Orange version 3.28.0 (University of Ljubljana, Ljubljana, Slovenia) [31]. The radiomics quality score (RQS) was calculated for this study according to the recommendations by Lambin et al. [32]. A p value < 0.05 was considered to indicate a statistically significant difference. The image processing and statistical analysis workflow are shown in Figure 1.
Figure 1.
Study workflow. GBM = glioblastoma, GTR = gross-total resection, NTR = near-total resection, BraTS = Brain tumor segmentation challenge 2020, TCIA = The cancer image archive, GLISTRboost = Boosted GLioma Image SegmenTation and Registration, ED = edema, NET = nonenhancing tumor, ET = enhancing tumor, OS = overall survival, InfoGain = Information gain, GINI = Gini Index, FCBF = Fast correlation-based filter, ML = machine learning, AUC = area under the curve, CA = classification accuracy, IBS = integrated Brier score, iAUC = integrated AUC.
3. Results
3.1. Patient Population
Two hundred and three patients were enrolled in this study. The mean age was 61.49 ± 11.76 (range 27.81–86.65). The median OS was 407 days [interquartile range (IQR) = 351.5]. A total of 7.9% (16) of patients were right-censored cases, and 17.24% (35) registered an OS of less than six months. The patients were randomly assigned to a training data set of 143 patients and a testing data set of 60 patients. There were no significant differences in age, preoperative tumor volume, OS, or proportion of right-censored and short-term survival cases between the training and test data sets (Table 1).
Table 1.
Patient characteristics.
| Data Set | n | Age (SD) | Tumor Volume (cm3) | OS (IQR) | Censored (%) | OS < 6 m (%) |
|---|---|---|---|---|---|---|
| BraTS | 119 | 62 ± 12 | 30.09 ± 11.77 | 374 (364) | 0 | 21.8% (26) |
| TCIA | 34 | 60.3 ± 10 | 42.55 ± 15.71 | 414 (482) | 5.9% (2) | 17.6% (6) |
| Institution 1 | 22 | 65.3 ± 10 | 40.77 ± 18.12 | 451 (307) | 22.7% (5) | 13.6% (3) |
| Institution 2 | 28 | 57.9 ± 13 | 42.14 ± 17.16 | 466 (217) | 32.1% (9) | 0 |
| After random split (70/30) | ||||||
| Training | 143 | 62.1 ± 11 | 40.62 ± 10.27 | 409 (311) | 7.7% (11) | 17.5% (25) |
| Testing | 60 | 60.1 ± 13 | 39.27 ± 11.35 | 404 (392) | 8.3% (5) | 16.7% (10) |
| Statistical comparison between cohorts |
t = 0.10, p = 0.273 | t = 0.12, p = 0.902 | U = 0.33, p = 0.743 | χ2 = 0.02, p = 0.877 | χ2 = 1.17, p = 0.279 | |
BraTS = Brain Tumor Segmentation Challenge 2020, TCIA = The Cancer Image Archive, SD = standard deviation, IQR = interquartile range.
3.2. Classification Task and Survival Groups
The variable selection filters made it possible to determine the top ten radiomic features (Table 2). Based on these characteristics and using the ML classifiers, patients were classified into short-term survivors (<6 months).
Table 2.
Radiomic features ranked by different scoring methods for short-term survival prediction.
| Filter | ||
|---|---|---|
| Information Gain | Gini Index | FCBF |
| T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Skewness | T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Range_Max | T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Skewness |
| T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Kurtosis | T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Kurtosis | T2WI_ED_Lattice_Intensity_Bins-20_Radius-1_QuartileCoefficientOfVariation_StdDev |
| T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Range_Max | T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Skewness | T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Energy_Skewness |
| T2WI_ED_Lattice_Intensity_Bins-20_Radius-1_QuartileCoefficientOfVariation_StdDev | FLAIR_ET_Lattice_Morphologic_EquivalentSphericalRadius_Variance | T1CE_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_NinetyFifthPercentile_Kurtosis |
| FLAIR_ET_Lattice_Morphologic_EquivalentSphericalPerimeter_Variance | FLAIR_ET_Lattice_Morphologic_EquivalentSphericalPerimeter_Variance | T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Range_Max |
| FLAIR_ET_Lattice_Morphologic_EquivalentSphericalRadius_Variance | FLAIR_NET_Lattice_Histogram_Bins-20_Bins-20_Bin-12_Frequency_Median | FLAIR_ET_Lattice_Morphologic_EquivalentSphericalPerimeter_Variance |
| FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_InterQuartileRange_StdDev | T2WI_ED_Lattice_Intensity_Bins-20_Radius-1_Mean_Skewness | FLAIR_ET_Lattice_Morphologic_EquivalentSphericalRadius_Variance |
| FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_InterQuartileRange_Variance | T2WI_ED_Lattice_Intensity_Bins-20_Radius-1_QuartileCoefficientOfVariation_StdDev | FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_InterQuartileRange_StdDev |
| FLAIR_NET_Lattice_Histogram_Bins-20_Bins-20_Uniformity_Min | FLAIR_NET_Lattice_Histogram_Bins-20_Bins-20_Uniformity_Min | FLAIR_NET_Lattice_Histogram_Bins-20_Bins-20_Bin-12_Frequency_Median |
| A10824 FLAIR_NET_Lattice_Histogram_Bins-20_Bins-20_Bin-12_Frequency_Median |
FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_InterQuartileRange_Variance | FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_InterQuartileRange_Variance |
FCBF, fast correlation-based filter, T1CE = contrast-enhanced T1-weighted images, T2WI = T2 weighted images, FLAIR = Fluid-attenuated inversion recovery, ET = enhancing tumor, NET = non-enhancing tumor, ED = edema.
The optimal results were obtained by applying the information gain as a feature selector. Thus, in the training cohort, AUC values were achieved with a range between 0.802 and 0.978, a classification accuracy between 81.8% and 94.4%, and a precision between 82.8% and 94.8% (Table S4).
In the testing data set, the naive Bayes classifier obtained the best results, with an AUC of 0.769, a classification accuracy of 80%, and a precision of 81% (Table 3 and Figure 2).
Table 3.
Model performance grouped by feature selection filter and machine learning classifier on the testing data set.
| Classifier | Filter | AUC | CA | Precision | F1 |
|---|---|---|---|---|---|
| Naive Bayes | Information Gain | 0.769 | 0.800 | 0.812 | 0.805 |
| Gini Index | 0.784 | 0.767 | 0.810 | 0.781 | |
| FCBF | 0.743 | 0.783 | 0.803 | 0.791 | |
| k-Nearest Neighbor | Information Gain | 0.600 | 0.817 | 0.806 | 0.776 |
| Gini Index | 0.639 | 0.800 | 0.771 | 0.763 | |
| FCBF | 0.670 | 0.767 | 0.713 | 0.724 | |
| Neural Network | Information Gain | 0.691 | 0.800 | 0.771 | 0.763 |
| Gini Index | 0.682 | 0.767 | 0.691 | 0.705 | |
| FCBF | 0.722 | 0.817 | 0.806 | 0.776 | |
| Random Forest | Information Gain | 0.574 | 0.733 | 0.683 | 0.701 |
| Gini Index | 0.666 | 0.750 | 0.696 | 0.713 | |
| FCBF | 0.700 | 0.783 | 0.738 | 0.735 | |
| Support Vector Machine | Information Gain | 0.709 | 0.750 | 0.608 | 0.671 |
| Gini Index | 0.630 | 0.750 | 0.608 | 0.671 | |
| FCBF | 0.730 | 0.800 | 0.777 | 0.747 | |
| Logistic Regression | Information Gain | 0.648 | 0.783 | 0.614 | 0.688 |
| Gini Index | 0.643 | 0.783 | 0.614 | 0.688 | |
| FCBF | 0.656 | 0.783 | 0.614 | 0.688 |
AUC: area under the curve; CA: classification accuracy; FCBC: fast correlation-based filter.
Figure 2.
Performance assessment of the binary classification model on the testing data set. left: area under the receiver operating characteristic (ROC) curve and right: calibration plot. kNN = k-nearest neighbor, SVM = support vector machine, TP = true positives, FP = false positives, P = positives.
3.3. Random Survival Forest to Predict OS
In the radiomic model based on RSF, the variable-hunting algorithm selected 17 radiomic features to predict OS in the training data set (Table 4).
Table 4.
Radiomic features selected by the variable hunting algorithm for Random Forest Survival prediction.
| Nº | Feature |
|---|---|
| 1 | T1CE_NET_Lattice_GLSZM_Bins-20_Radius-1_LargeZoneHighGreyLevelEmphasis_Mean |
| 2 | FLAIR_ED_Lattice_NGTDM_Busyness_Max |
| 3 | T1CE_NET_Lattice_GLSZM_Bins-20_Radius-1_LargeZoneHighGreyLevelEmphasis_Median |
| 4 | FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Bin-14_Frequency_Max |
| 5 | FLAIR_NET_Lattice_Intensity_Bins-20_Radius-1_NinetiethPercentile_Mean |
| 6 | T1CE_ET_Lattice_Intensity_Bins-20_Radius-1_StandardDeviation_StdDev |
| 7 | FLAIR_NET_Lattice_Morphologic_PixelsOnBorder_Variance |
| 8 | T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Bin-0_Probability_Kurtosis |
| 9 | T1CE_ET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Bin-9_Probability_Median |
| 10 | T1CE_ET_Lattice_GLSZM_Bins-20_Radius-1_ZoneSizeMean_Variance |
| 11 | T2WI_ED_Lattice_Intensity_Bins-20_Radius-1_Mean_Skewness |
| 12 | FLAIR_ET_Lattice_Morphologic_Perimeter_Skewness |
| 13 | FLAIR_NET_Lattice_Histogram_Bins-20_Radius-1_Bins-20_FifthPercentileMean_Max |
| 14 | FLAIR_ET_Lattice_Morphologic_EquivalentSphericalRadius_Variance |
| 15 | T1CE_ET_Lattice_GLRLM_Bins-20_Radius-1_RunLengthNonuniformity_Kurtosis |
| 16 | FLAIR_ED_Lattice_Intensity_Bins-20_Radius-1_InterQuartileRange_Median |
| 17 | FLAIR_ED_Lattice_Histogram_Bins-20_Radius-1_Bins-20_Sum_Max |
T1CE = contrast-enhanced T1-weighted images, T2WI = T2 weighted images, FLAIR = Fluid-attenuated inversion recovery, ET = enhancing tumor, NET = non-enhancing tumor, ED = edema, GLSZM = Gray Level Size-Zone Matrix, NGTDM = Neighborhood Gray-Tone Difference Matrix, GLRLM = Gray Level Run-Length Matrix.
Based on these characteristics, the mortality risk score was calculated using the OOB-CHF. The cut-off point used was 0.684 (Table S5). This cut-off point allowed patients to be stratified into low-risk and high-risk groups [HR = 2.19, (95% CI: 1.54–3.12), log-rank p = < 0.001, C-Index = 0.61, IBS = 0.096]. In the testing data set, patients were also stratified using the same cut-off point [HR 2.16, (95% CI: 1.21–3.89), log-rank p = 0.008, C-Index = 0.61, IBS = 0.123] (Figure 3). The multivariate Cox regression models in which age was incorporated as an explanatory variable are shown in Table 5.
Figure 3.
Kaplan–Meier plots showing differences in overall survival for patients in training (A) and testing (B) data sets stratified into low- or high-risk groups by random survival forest (RSF)-based ensemble mortality. Prediction error curves show Cox regression model performance in the training (C) and testing (D) groups. HR= Hazard Ratio, RSF = Random Survival Forest, IBS = Integrated Brier Score, iAUC = Integrated area under the curve, 6m – AUC = six months - area under the curve.
Table 5.
Univariate and Multivariate Cox regression analysis.
| Univariate Cox Regression Analysis | ||||||||
|---|---|---|---|---|---|---|---|---|
| Variable | β | HR | 95% CI | p | C-Index | IBS | iAUC | 6m-iAUC |
| Training data set | ||||||||
| Age | 0.03 | 1.03 | 1.02–1.05 | <0.001 | 0.61 | 0.089 | 0.604 | 0.599 |
| Radiomic RSF Score (High risk) | 0.78 | 2.19 | 1.54–3.12 | <0.001 | 0.61 | 0.096 | 0.591 | 0.712 |
| Testing data set | ||||||||
| Age | 0.03 | 1.03 | 1–1.06 | 0.023 | 0.60 | 0.128 | 0.592 | 0.643 |
| Radiomic RSF Score (High risk) | 0.77 | 2.16 | 1.21–3.89 | 0.009 | 0.61 | 0.123 | 0.568 | 0.761 |
| Multivariate Cox Regression Model | ||||||||
| Model | Likelihood Ratio Test | C-Index | IBS | iAUC | 6m-AUC | |||
| χ 2 | df | p | ||||||
| Training data set | ||||||||
| Age + Radiomic RSF Score (High risk) | 36.93 | 2 | <0.001 | 0.66 | 0.084 | 0.650 | 0.730 | |
| Testing data set | ||||||||
| Age + Radiomic RSF Score (High risk) | 11.35 | 2 | 0.003 | 0.68 | 0.118 | 0.6278 | 0.769 | |
The iAUC of the radiomic model was 0.591 in the training data set and 0.568 in the test data set. By incorporating age as a variable in the model, the iAUC increased to 0.650 in the training data set and 0.627 in the testing data set.
After setting the survival time to 6 months, the predictive accuracy of the radiomic model improved to an iAUC of 0.712 in the training data set and 0.761 in the testing data set (Figure S1).
The RQS was used to evaluate the methodological quality of our study. We obtained a score of 19/36 (53%). A detailed report of RQS items is shown in Table S6.
4. Discussion
In the present study, we elaborated a prediction model of short-term survival with high predictive capacity using the radiomic features of structural preoperative multiparametric MRI of GBM patients.
We believe that the main strength of our study is based on a selection of patients who underwent total or near-total resection of the enhancing tumor. We considered this methodologic aspect due to the undeniable link between the extent of resection and survival in these patients [33]. In most previous studies, the extent of resection was not used as a selection criterion, including partial resections and biopsies in their series, without making any adjustment during the analysis phase. The exception is the studies by Bakas et al. [14] and Fathi et al. [34], in which the entire cohort of patients underwent complete resection and standard chemoradiotherapy treatment.
Another crucial point of our work is to set our objective to identify short-term survival patients, in contrast to previously published studies where 10 and 12–15 months were used as cut-off points for defining short- and long-term survival, respectively [9,10,12,35,36,37,38,39]. The only reference we found is in the work of Prasanna et al. [40], who classified patients in long (>18 months) versus short-term (<7 months) survival, based on peritumoral region radiomic features. The rationale of our approach lies in the desire to predict the survival of patients diagnosed with GBM by non-invasive methods and to identify those with very short survival. In these patients, the futility of our treatments would lead us to offer patients and their families the option of not taking aggressive measures or, on the contrary, opening new lines of research since those cases would be poor responders to the standard therapies applied currently.
As another strength of our work, we can mention the use of open-source software. The CapTk and Orange programs have a very intuitive yet robust user interface, thanks to which clinicians can access advanced image processing technics and data mining tools. Thanks to these programs, we have performed complex tasks such as automatic tumor segmentation, image processing, radiomic feature extraction, and exploring different ML-based algorithms.
Concerning statistical analysis, we used a dual approach. On the one hand, we have used a binary classification system using different ML-based algorithms. Additionally, we used state-of-the-art survival analysis techniques such as random survival forest and time-dependent ROC curve analysis focused on short-term survival that contribute to corroborating the stability of the models produced here.
We also highlight that the results of our predictive models have been achieved using only structural MRI [14]. These results could even be improved after the inclusion of studies based on diffusion and perfusion sequences [41]. However, basic MRI is available in most centers, and according to our results, the lack of special sequences is not a limitation in the search for useful radiological patterns in clinical practice.
An important aspect to discuss is the biological correspondence of the variables employed by the prediction models. There is notable variability concerning the radiomic characteristics used by previous studies, which is one of the most significant obstacles in reproducing and validating their results. In our study, most of the selected variables came from the T1CE sequence followed by FLAIR and T2WI, while the different tumor subregions (i.e., ET, NET and ED) were represented in the models in a balanced way. In our series, first-order features and morphological characteristics appeared to be important for OS prediction.
We are aware of the limitations of our work, such as the lack of clinical and molecular data that can be incorporated into predictive models. Even so, age as an explanatory variable was incorporated into our models due to its significant association with the OS of these patients, proving that its mere incorporation into the analysis improved the performance of the models. Despite having a relatively small sample size, various statistical techniques have been applied to overcome the "curse of dimensionality". Taking into consideration that MRI studies come from numerous sources, the processing method for image standardization that we have chosen aims to be simple and reliable and has been used by several studies [23,34,42].
Another limitation of predictive models based on radiomic features is the lack of assessment of the repeatability and reproducibility of the features calculated from the magnetic resonance images. We have included a detailed description of the feature extraction software, digital image manipulation, and image acquisition parameters to improve reporting quality. However, it is necessary to carry out new studies specifically designed to solve this problem.
Unquestionably, the combination of texture analysis and artificial intelligence is starting to facilitate knowledge about the biological behaviour of GBMs through the study of their patient-dependent heterogeneity. However, the rapid development of big data tools and the tremendous complexity of advanced medical image analysis dangerously threaten widening the gap between data experts and clinicians. Then, it is a paradox that radiomics, defined by Lambin et al. [32] as “the bridge between medical imaging and personalized medicine”, is now out of reach of those who treat real patients every day. Therefore, our study arises from a real need and aims to find a solution to a clinically relevant problem: identifying GBM patients with short survival after complete resection. Although our results can be improved, we show that there are currently computer tools and public data sets available to everyone to develop reliable predictive models. Hence, our duty as clinicians is to become immersed in developing these models, since our pragmatism can never be replaced, even by the most complex algorithm.
Indeed, our results are encouraging, and the precision achieved is similar to the previous literature. However, this article represents an early age of a promising future in which the ultimate link between image, diagnosis and prognosis could finally be decoded to provide instant, useful and precise information to individual patients based on their specific features. Multi-institutional studies [43] would allow the generalization of predictive models, or even adapt the mechanisms of data pre-processing, extraction, and analysis to the MRI from each center, since the standardization of acquisition protocols is not feasible. Finally, we believe that in this catastrophic disease, the quality of life of our patients should be our first consideration, and maximum exploitation of available neuroimaging techniques should be pursued to optimize management strategies, avoiding unnecessarily aggressive therapies in those patients who will not benefit from them.
5. Conclusions
In the present study, we evaluated the capability of the radiomic features of preoperative mpMRI and machine learning-based classification and regression analysis to predict short-term survival in GBM patients. Our model shows a classification accuracy of 80% and an iAUC of 0.76 to predict OS < 6 months in the test data set. We believe that these new tools will serve clinicians to understand the biological behaviour of individual GBMs, and we must take advantage of them.
Supplementary Materials
The following are available online at https://www.mdpi.com/article/10.3390/cancers13205047/s1, Figure S1: Time-dependent receiver operating characteristics (ROC) curve of the Random Survival Forest survival model in the testing data set. Table S1: The specific parameters of structural multiparametric MRI. Table S2: Radiomic features description. Table S3: Hyperparameters of machine learning classifiers used in this study. Table S4: Model performance grouped by feature selection filter and machine learning classifier on training data set. Table S5: Kaplan-Meier analysis based on Random Forest Survival risk groups. Table S6: Methodologic quality evaluation of our study by the Radiomic Quality Score.
Author Contributions
Conceptualization, S.C.; methodology, A.P.-N., D.G.-P.; software, S.C., M.G.-G.; validation, S.G.-G., S.C., formal analysis, S.C., M.V.-C.; investigation, S.C., T.Z.; data curation, R.S.; writing—original draft preparation, S.C.; writing—review and editing, I.A., L.J.-R., P.G.; visualization, S.G.-G.; supervision, R.S.; project administration, R.S. All authors have read and agreed to the published version of the manuscript.
Funding
This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.
Institutional Review Board Statement
The study was conducted according to the guidelines of the Declaration of Helsinki, and approved by the Drugs & Research Ethics Committee (CEIm) of the University Hospital Río Hortega, Ref.: 21-PI085, 5 July 2021.
Informed Consent Statement
Informed consent was obtained from all subjects involved in the study.
Data Availability Statement
The data sets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.
Conflicts of Interest
The authors declare no conflict of interest.
Footnotes
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
- 1.Koshy M., Villano J.L., Dolecek T.A., Howard A., Mahmood U., Chmura S.J., Weichselbaum R.R., McCarthy B.J. Improved survival time trends for glioblastoma using the SEER 17 population-based registries. J. Neurooncol. 2012;107:207–212. doi: 10.1007/s11060-011-0738-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Stupp R., Hegi M.E., Mason W.P., van den Bent M.J., Taphoorn M.J.B., Janzer R.C., Ludwin S.K., Allgeier A., Fisher B., Belanger K., et al. Effects of radiotherapy with concomitant and adjuvant temozolomide versus radiotherapy alone on survival in glioblastoma in a randomised phase III study: 5-year analysis of the EORTC-NCIC trial. Lancet. Oncol. 2009;10:459–466. doi: 10.1016/S1470-2045(09)70025-7. [DOI] [PubMed] [Google Scholar]
- 3.Sanai N., Berger M.S. Surgical oncology for gliomas: The state of the art. Nat. Rev. Clin. Oncol. 2018;15:112–125. doi: 10.1038/nrclinonc.2017.171. [DOI] [PubMed] [Google Scholar]
- 4.Gutman D.A., Cooper L.A.D., Hwang S.N., Holder C.A., Gao J., Aurora T.D., Dunn W.D., Scarpace L., Mikkelsen T., Jain R., et al. MR Imaging predictors of molecular profile and survival: Multi-institutional Study of the TCGA Glioblastoma Data Set. Radiology. 2013;267:560–569. doi: 10.1148/radiol.13120118. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5.Chaddad A., Kucharczyk M.J., Daniel P., Sabri S., Jean-Claude B.J., Niazi T., Abdulkarim B. Radiomics in glioblastoma: Current status and challenges facing clinical implementation. Front. Oncol. 2019;9:1–9. doi: 10.3389/fonc.2019.00374. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Gillies R.J., Kinahan P.E., Hricak H. Radiomics: Images Are more than pictures, they are data. Radiology. 2016;278:563–577. doi: 10.1148/radiol.2015151169. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Seow P., Wong J.H.D., Ahmad-Annuar A., Mahajan A., Abdullah N.A., Ramli N. Quantitative magnetic resonance imaging and radiogenomic biomarkers for glioma characterisation: A systematic review. Br. J. Radiol. 2018;91:20170930. doi: 10.1259/bjr.20170930. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8.Osman A.F.I. A multi-parametric MRI-Based radiomics signature and a practical ML Model for stratifying glioblastoma patients based on survival toward precision oncology. Front. Comput. Neurosci. 2019;13:58. doi: 10.3389/fncom.2019.00058. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Kickingereder P., Burth S., Wick A., Götz M., Eidel O., Schlemmer H.-P., Maier-Hein K.H., Wick W., Bendszus M., Radbruch A., et al. Radiomic Profiling of glioblastoma: Identifying an Imaging predictor of patient survival with improved performance over established clinical and radiologic risk models. Radiology. 2016;280:880–889. doi: 10.1148/radiol.2016160845. [DOI] [PubMed] [Google Scholar]
- 10.Bae S., Choi Y.S., Ahn S.S., Chang J.H., Kang S.-G., Kim E.H., Kim S.H., Lee S.-K. Radiomic MRI Phenotyping of glioblastoma: Improving survival prediction. Radiology. 2018;289:797–806. doi: 10.1148/radiol.2018180200. [DOI] [PubMed] [Google Scholar]
- 11.Shboul Z.A., Alam M., Vidyaratne L., Pei L., Elbakary M.I., Iftekharuddin K.M. Feature-guided deep radiomics for glioblastoma patient survival prediction. Front. Neurosci. 2019;13:966. doi: 10.3389/fnins.2019.00966. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12.Chaddad A., Daniel P., Sabri S., Desrosiers C., Abdulkarim B. Integration of radiomic and multi-omic analyses predicts survival of newly diagnosed IDH1 wild-type glioblastoma. Cancers. 2019;11:1148. doi: 10.3390/cancers11081148. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Baid U., Rane S.U., Talbar S., Gupta S., Thakur M.H., Moiyadi A., Mahajan A. Overall survival prediction in glioblastoma with radiomic features using machine learning. Front. Comput. Neurosci. 2020;14:1–9. doi: 10.3389/fncom.2020.00061. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Bakas S., Shukla G., Akbari H., Erus G., Sotiras A., Rathore S., Sako C., Min Ha S., Rozycki M., Shinohara R.T., et al. Overall survival prediction in glioblastoma patients using structural magnetic resonance imaging (MRI): Advanced radiomic features may compensate for lack of advanced MRI modalities. J. Med. Imaging. 2020;7:031505. doi: 10.1117/1.JMI.7.3.031505. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Menze B.H., Jakab A., Bauer S., Kalpathy-Cramer J., Farahani K., Kirby J., Burren Y., Porz N., Slotboom J., Wiest R., et al. The Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) IEEE Trans. Med. Imaging. 2015;34:1993–2024. doi: 10.1109/TMI.2014.2377694. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Bakas S., Reyes M., Jakab A., Bauer S., Rempfler M., Crimi A., Shinohara R.T., Berger C., Ha S.M., Rozycki M., et al. Identifying the best machine learning algorithms for brain tumor segmentation, progression assessment, and overall survival prediction in the BRATS challenge. arXiv. 20181811.02629 [Google Scholar]
- 17.Bakas S., Akbari H., Sotiras A., Bilello M., Rozycki M., Kirby J.S., Freymann J.B., Farahani K., Davatzikos C. Advancing the cancer genome atlas glioma MRI collections with expert segmentation labels and radiomic features. Sci. Data. 2017;4:170117. doi: 10.1038/sdata.2017.117. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 18.Puchalski R.B., Shah N., Miller J., Dalley R., Nomura S.R., Yoon J.-G., Smith K.A., Lankerovich M., Bertagnolli D., Bickley K., et al. An anatomic transcriptional atlas of human glioblastoma. Science. 2018;360:660–663. doi: 10.1126/science.aaf2666. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 19.Clark K., Vendt B., Smith K., Freymann J., Kirby J., Koppel P., Moore S., Phillips S., Maffitt D., Pringle M., et al. The Cancer Imaging Archive (TCIA): Maintaining and Operating a public information repository. J. Digit. Imaging. 2013;26:1045–1057. doi: 10.1007/s10278-013-9622-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 20.Rohlfing T., Zahr N.M., Sullivan E.V., Pfefferbaum A. The SRI24 multichannel atlas of normal adult human brain structure. Hum. Brain Mapp. 2010;31:798–819. doi: 10.1002/hbm.20906. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Tustison N.J., Avants B.B., Cook P.A., Zheng Y., Egan A., Yushkevich P.A., Gee J.C. N4ITK: Improved N3 bias correction. IEEE Trans. Med. Imaging. 2010;29:1310–1320. doi: 10.1109/TMI.2010.2046908. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 22.Thakur S., Doshi J., Pati S., Rathore S., Sako C., Bilello M., Ha S.M., Shukla G., Flanders A., Kotrotsou A., et al. Brain extraction on MRI scans in presence of diffuse glioma: Multi-institutional performance evaluation of deep learning methods and robust modality-agnostic training. Neuroimage. 2020;220:117081. doi: 10.1016/j.neuroimage.2020.117081. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23.Davatzikos C., Rathore S., Bakas S., Pati S., Bergman M., Kalarot R., Sridharan P., Gastounioti A., Jahani N., Cohen E., et al. Cancer imaging phenomics toolkit: Quantitative imaging analytics for precision diagnostics and predictive modeling of clinical outcome. J. Med. Imaging. 2018;5:011018. doi: 10.1117/1.JMI.5.1.011018. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24.Bakas S., Zeng K., Sotiras A., Rathore S., Akbari H., Gaonkar B., Rozycki M., Pati S., Davatzikos C. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Volume 9556. Springer; Cham, Switzerland: 2016. GLISTRboost: Combining multimodal MRI Segmentation, registration, and biophysical tumor growth modeling with gradient boosting machines for glioma segmentation; pp. 144–155. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 25.Zwanenburg A., Vallières M., Abdalah M.A., Aerts H.J.W.L., Andrearczyk V., Apte A., Ashrafinia S., Bakas S., Beukinga R.J., Boellaard R., et al. The Image biomarker standardization initiative: Standardized Quantitative radiomics for high-throughput image-based phenotyping. Radiology. 2020;295:328–338. doi: 10.1148/radiol.2020191145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Ishwaran H., Kogalur U.B. Fast Unified Random Forests for Survival, Regression, and Classification (RF-SRC) Volume 2 The R Foundation; Vienna, Austria: 2019. R Package version 2.12.1. [Google Scholar]
- 27.Dardis C. survMisc: Miscellaneous Functions for Survival Data. The R Foundation; Vienna, Austria: 2016. R Package Version 0.5.5. [Google Scholar]
- 28.Mogensen U.B., Ishwaran H., Gerds T.A. Evaluating Random forests for survival analysis using prediction error curves. J. Stat. Softw. 2012;50:1–23. doi: 10.18637/jss.v050.i11. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29.Heagerty P.J., Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 2005;61:92–105. doi: 10.1111/j.0006-341X.2005.030814.x. [DOI] [PubMed] [Google Scholar]
- 30.Kamarudin A.N., Cox T., Kolamunnage-Dona R. Time-dependent ROC curve analysis in medical research: Current methods and applications. BMC Med. Res. Methodol. 2017;17:53. doi: 10.1186/s12874-017-0332-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31.Demšar J., Curk T., Erjavec A., Gorup Č., Hočevar T., Milutinovič M., Možina M., Polajnar M., Toplak M., Starič A., et al. Orange: Data mining toolbox in Python. J. Mach. Learn. Res. 2013;14:2349–2353. [Google Scholar]
- 32.Lambin P., Leijenaar R.T.H., Deist T.M., Peerlings J., de Jong E.E.C., van Timmeren J., Sanduleanu S., Larue R.T.H.M., Even A.J.G., Jochems A., et al. Radiomics: The bridge between medical imaging and personalized medicine. Nat. Rev. Clin. Oncol. 2017;14:749–762. doi: 10.1038/nrclinonc.2017.141. [DOI] [PubMed] [Google Scholar]
- 33.Awad A.-W., Karsy M., Sanai N., Spetzler R., Zhang Y., Xu Y., Mahan M.A. Impact of removed tumor volume and location on patient outcome in glioblastoma. J. Neurooncol. 2017;135:161–171. doi: 10.1007/s11060-017-2562-1. [DOI] [PubMed] [Google Scholar]
- 34.Fathi Kazerooni A., Akbari H., Shukla G., Badve C., Rudie J.D., Sako C., Rathore S., Bakas S., Pati S., Singh A., et al. Cancer imaging phenomics via CaPTk: Multi-institutional prediction of progression-free survival and pattern of recurrence in glioblastoma. JCO Clin. Cancer Inform. 2020:234–244. doi: 10.1200/CCI.19.00121. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 35.Kickingereder P., Neuberger U., Bonekamp D., Piechotta P.L., Götz M., Wick A., Sill M., Kratz A., Shinohara R.T., Jones D.T.W., et al. Radiomic subtyping improves disease stratification beyond key molecular, clinical, and standard imaging characteristics in patients with glioblastoma. Neuro. Oncol. 2018;20:848–857. doi: 10.1093/neuonc/nox188. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36.Lu Y., Patel M., Natarajan K., Ughratdar I., Sanghera P., Jena R., Watts C., Sawlani V. Machine learning-based radiomic, clinical and semantic feature analysis for predicting overall survival and MGMT promoter methylation status in patients with glioblastoma. Magn. Reson. Imaging. 2020;74:161–170. doi: 10.1016/j.mri.2020.09.017. [DOI] [PubMed] [Google Scholar]
- 37.Kickingereder P., Götz M., Muschelli J., Wick A., Neuberger U., Shinohara R.T., Sill M., Nowosielski M., Schlemmer H.P., Radbruch A., et al. Large-scale radiomic profiling of recurrent glioblastoma identifies an imaging predictor for stratifying anti-angiogenic treatment response. Clin. Cancer Res. 2016;22:5765–5771. doi: 10.1158/1078-0432.CCR-16-0702. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38.Ingrisch M., Schneider M.J., Nörenberg D., Negrao de Figueiredo G., Maier-Hein K., Suchorska B., Schüller U., Albert N., Brückmann H., Reiser M., et al. Radiomic analysis reveals prognostic information in T1-weighted baseline magnetic resonance imaging in patients with glioblastoma. Invest. Radiol. 2017;52:360–366. doi: 10.1097/RLI.0000000000000349. [DOI] [PubMed] [Google Scholar]
- 39.Liu Y., Xu X., Yin L., Zhang X., Li L., Lu H. Relationship between Glioblastoma heterogeneity and survival time: An MR imaging texture analysis. AJNR. Am. J. Neuroradiol. 2017;38:1695–1701. doi: 10.3174/ajnr.A5279. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 40.Prasanna P., Patel J., Partovi S., Madabhushi A., Tiwari P. Radiomic features from the peritumoral brain parenchyma on treatment-naïve multi-parametric MR imaging predict long versus short-term survival in glioblastoma multiforme: Preliminary findings. Eur. Radiol. 2017;27:4188–4197. doi: 10.1007/s00330-016-4637-3. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 41.Kim J.Y., Yoon M.J., Park J.E., Choi E.J., Lee J., Kim H.S. Radiomics in peritumoral non-enhancing regions: Fractional anisotropy and cerebral blood volume improve prediction of local progression and overall survival in patients with glioblastoma. Neuroradiology. 2019;61:1261–1272. doi: 10.1007/s00234-019-02255-4. [DOI] [PubMed] [Google Scholar]
- 42.Rathore S., Bakas S., Pati S., Akbari H., Kalarot R., Sridharan P., Rozycki M., Bergman M., Tunc B., Verma R., et al. Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries. Volume 10670 LNCS. Springer; Cham, Switzerland: 2018. Brain cancer imaging phenomics toolkit (brain-CaPTk): An interactive platform for quantitative analysis of glioblastoma; pp. 133–145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 43.Davatzikos C., Barnholtz-Sloan J.S., Bakas S., Colen R., Mahajan A., Quintero C.B., Capellades Font J., Puig J., Jain R., Sloan A.E., et al. AI-based prognostic imaging biomarkers for precision neuro-oncology: The ReSPOND consortium. Neuro. Oncol. 2020;22:886–888. doi: 10.1093/neuonc/noaa045. [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Supplementary Materials
Data Availability Statement
The data sets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.