Survival prediction in glioblastoma on post-contrast magnetic resonance imaging using filtration based first-order texture analysis: Comparison of multiple machine learning models

Abstract

Objective

Magnetic resonance texture analysis (MRTA) is a relatively new technique that can be a valuable addition to clinical and imaging parameters in predicting prognosis. In the present study, we investigated the efficacy of MRTA for glioblastoma survival using T1 contrast-enhanced (CE) images for texture analysis.

Methods

We evaluated the diagnostic performance of multiple machine learning models based on first-order histogram statistical parameters derived from T1-weighted CE images in the survival stratification of glioblastoma multiforme (GBM). Retrospective evaluation of 85 patients with GBM was performed. Thirty-six first-order texture parameters at six spatial scale filters (SSF) were extracted on the T1 CE axial images for the whole tumor using commercially available research software. Several machine learning classification models (in four broad categories: linear, penalized linear, non-linear, and ensemble classifiers) were evaluated to assess the survival prediction performance using optimal features. Principal component analysis was used prior to fitting the linear classifiers in order to reduce the dimensionality of the feature inputs. Fivefold cross-validation was used to partition the data iteratively into training and testing sets. The area under the receiver operating characteristic curve (AUC) was used to assess the diagnostic performance.

Results

The neural network model was the highest performing model with the highest observed AUC (0.811) and cross-validated AUC (0.71). The most important variable was the age at diagnosis, with mean and mean of positive pixels (MPP) for SSF = 0 being the second and third most important, followed by skewness for SSF = 0 and SSF = 4.

Conclusions

First-order texture features, when combined with age at presentation, show good accuracy in predicting GBM survival.

Introduction

Glioblastoma multiforme (GBM) is the most common and aggressive primary adult malignant brain tumor.¹ The prognosis of GBM is quite dismal, despite available therapeutic strategies, including maximal safe surgical resection followed by chemoradiation and temozolomide.² The median survival of GBM is only 15 months, though some patients may survive beyond three years.³

The poor prognosis is likely secondary to underlying tumoral heterogeneity.⁴ Conventional magnetic resonance imaging (MRI) is commonly used for treatment planning and response assessment. However, visual analysis of MRI images is frequently limited in assessing the heterogeneity of GBM. Brain biopsy is often employed for preoperative diagnostic workup, including glioma grading and genetic testing in patients with suspected GBM. However, apart from being invasive, the biopsy is prone to sampling error and may not entirely capture tumor heterogeneity.⁵ Clinical parameters including age, Karnofsky performance status (KPS), preoperative tumor volume, the extent of tumor resection, and postoperative adjuvant therapy have also been shown to predict prognosis.⁶,⁷ The VASARI (Visually AcceSAble Rembrandt Images) scoring system, which includes 30 descriptors of imaging features of brain tumors, is another method that can provide an objective assessment of high-risk imaging features in GBM.⁸,⁹

Magnetic resonance texture analysis (MRTA) is a relatively new technique that can be a valuable addition to these clinical and imaging parameters in predicting prognosis and may serve as an independent prognostic marker. When applied to MRI images, MRTA can be used to extract multiple quantitative, shape, or volume parameters from the entire tumor, including enhancing, necrotic, or edematous tumoral segments.¹⁰ These quantitative indices are thought to reflect underlying cellular heterogeneity and have been associated with tumor grade, histology, genomics, and survival.¹¹,¹² MRTA is noninvasive and can provide significant additional tumoral information from routinely acquired images.

First-order texture parameters are quantitative descriptors of gray-level pixel intensity (histogram).¹³ The filtration-based histogram technique extracts first-texture parameters after applying varying levels of spatial smoothing (different spatial scale filter (SSF) values).¹⁴

Prior studies have evaluated the role of MRTA in predicting survival in patients with GBM. However, all these studies utilized a selected combination of features based on shape, volume, first-, second-, and higher-order texture features, or deep features that were extracted from multiparametric or advanced imaging sequences (Appendix 1).¹²,^15–27 Advanced sequences are not performed routinely at all centers, require additional expertise and expense, and are time-consuming. Additionally, the extraction of higher-order features requires multiple post-processing steps and is time-consuming. In this study, we aimed to evaluate if only first-order histogram statistical parameters derived from routinely available T1 contrast-enhanced (CE) images at the time of initial presentation, along with age, can stratify long- from short-term survivors in patients with GBM.

Methods

Data collection

The study was approved by the local Institutional Review Board, and patient consent was waived off due to the retrospective nature of the study. Between 2005 and 2016, patients with a pathologically confirmed diagnosis of GBM (N = 165) were identified. All patients with the availability of acceptable diagnostic quality preoperative MRI images and the presence of a contrast-enhancing tumor were evaluated. Since the aim was to determine prediction between short- and long-term survival, only patients with available survival information and known short- (<12 months) or long-term (>24 months) survival were included. Survival time was assessed as the duration from the initial histopathological examination to the date of demise or censored point if the patient was still alive. Exclusion criteria included patients with non-available preoperative MRI, non-enhancing tumors, motion artifact, or survival between 12 and 24 months. This yielded a total of 85 patients with glioblastoma. The clinical characteristics of the selected patients are given in Table 1.

Table 1.

Patient characteristics.

	Short-term survivor (N = 53)	Long-term survivor (N = 32)
Age (years), M ± SD	65.2 ± 11.4	55.6 ± 12.6
Sex	M = 31; F = 22	M = 14; F = 18
Survival duration (months), M ± SD	6 ± 3.9	40.8 ± 21

M: male; F: female; SD: standard deviation.

Image acquisition

Pre-therapy scanning was performed on either a 1.5T (n = 76) or 3T (n = 9) MRI system (Siemens, Erlangen, Germany). The routine protocol for brain tumor imaging at our institution includes pre-contrast axial T1-weighted, T2-weighted, FLAIR, diffusion-weighted imaging with ADC maps, gradient echo followed by tri-planar T1-weighted CE images. Post-contrast T1-weighted images (slice thickness 3–5 mm) were acquired after administration of gadobenate dimeglumine (Multihance; Bayer Healthcare Pharma, Berlin, Germany) or gadobutrol (Gadavist; Bayer Healthcare Pharma) injected at the rate of 0.1 mL/kg body weight.

Tumor segmentation/region of interest delineation

One researcher independently segmented the region of interest (ROI). Each ROI was confirmed by two fellowship-trained neuroradiologists in consensus. Manual segmentation of tumors was performed on T1-weighted CE axial slices using a commercial research software (TexRAD v3.3, TexRAD Ltd, part of Feedback plc, Cambridge, UK) by employing a free-hand polygon drawing tool. All axial slices containing the tumor were included to perform total tumor volume (volumetric) segmentation. The ROI included the whole tumor (necrotic and solid enhancing regions of the tumor with the exclusion of edema; Figure 1). If more than one lesion was seen in the same slice, the lesion with the largest dimension was segmented.

Figure 1.

Tumor segmentation and feature extraction. Whole tumor segmentation with extraction of texture features at fine spatial scale filter (SSF=2), medium filter (SSF=4), and coarse filter (SSF=6).

Texture analysis

MRTA was performed using the same TexRAD software. The software employs a filtration-based histogram technique and extracts first-order texture features at different SFFs: no filter (SSF = 0), fine filter (SSF = 2), medium filter (SSF = 3–5), and coarse filter (SSF = 6) filters. These filter values correspond to the width (in millimeters) at which the image is enhanced or highlighted. Details regarding the methodology and post-processing software have been discussed in prior studies.²⁸,²⁹ Following the application of the Laplacian of Gaussian (LoG) band-pass filter, image texture is further quantified by deriving a histogram. Six statistical parameters for each SSF are calculated, followed by mathematical interpolation of slice data. These include mean (average value of the gray-level pixels or brightness); standard deviation (SD; dispersion of gray-level intensities from the mean); entropy (randomness of gray-level distribution); skewness (asymmetry of the distribution of variables around mean); kurtosis (sharpness of the peak of the histogram); and mean of positive pixels (MPP; mean of values >0).³⁰

Feature selection and reduction

Thirty-six textual features, as well as the patient’s age at diagnosis, were included as predictor variables. Age was included, as it has been shown in prior studies to be an important clinical prognostic variable.³¹ Since we did not have sufficient information regarding KPS scores and the extent of resection, these variables were not included. Models were formulated to predict short-term survival.

Classifier models

Twelve models were fit to the data, which can be classified into four categories: linear classifiers, penalized linear classifiers, nonlinear classifiers, and ensemble methods. The linear classifiers evaluated were linear regression and logistic regression. Principal component analysis (PCA) was used prior to fitting the linear classifiers in order to reduce the dimensionality of the feature inputs. Five principal components were computed, reducing the 36 features to only five potential predictor variables. PCA was performed using the recipes package in R v4.0.2 (The R Foundation for Statistical Computing, Vienna, Austria). Three penalized classifiers were evaluated: ridge regression, least absolute shrinkage and selection operator (LASSO), and elastic net. All variables were standardized prior to fitting the penalized classifiers in order to ensure that each feature was treated equally in the penalization. The nonlinear classifiers evaluated were regression trees, neural networks, and support vector machines (SVM). The ensemble methods evaluated were the AdaBoost model, the generalized boosting model, and the extreme gradient boosting model. Tuning was performed for all models using nested cross-validation.

Classifier model evaluation

Models were evaluated by their predictive performance, that is, the model’s ability to classify new cases correctly. Since the data set was relatively small, fivefold cross-validation was used instead of separate training and validation sets. The same cross-validation splits were used to evaluate predictive performance for all models. The predictive performance of the models was evaluated by utilizing the area under the receiver operating characteristic curve (AUC) for interpretability. AUC estimates the probability that a randomly selected subject who was a short-term survivor will have a greater predicted value than a randomly selected subject who was a long-term survivor. Higher AUC values indicate better predictive performance. Model fitting and cross-validated computation of AUC and other performance metrics were performed using the MachineShop package in R v4.0.2.

Survival analysis

The model with the best AUC was used to predict short- or long-term status. Kaplan–Meier curves were constructed using the best performing model. A log-rank test was used to test for statistical significance.

Results

Patient characteristics

There were 45 males and 40 female patients in the study group. The mean survival of the short-term survivor group was six months (SD = 3.9 months), while the mean survival of the long-term survivor group was 41 months (SD = 21 months).

Classifier model performance

Summary statistics for the cross-validated AUC for all the models are presented in Table 2. Based on these results, the neural network showed the best overall performance followed by ridge regression. The neural network model was used to construct survival analysis curves.

Table 2.

Summary statistics of cross-validated ROC AUC for all models fit.

Model	Cross-validated ROC AUC
	M	Median	SD	Min	Max
Neural network	0.696	0.714	0.107	0.519	0.811
Ridge regression	0.686	0.714	0.095	0.532	0.783
XGBoost	0.659	0.667	0.161	0.429	0.833
Linear with PCA	0.647	0.667	0.075	0.532	0.717
Logistic with PCA	0.641	0.652	0.082	0.506	0.714
SVM	0.625	0.636	0.104	0.481	0.727
Regression tree	0.621	0.667	0.064	0.533	0.669
Elastic net	0.597	0.500	0.132	0.500	0.750
AdaBoost	0.582	0.636	0.205	0.234	0.740
GBM	0.581	0.584	0.140	0.364	0.742
Random forest	0.581	0.583	0.188	0.286	0.758
LASSO	0.525	0.500	0.056	0.500	0.625

SVM: support vector machine; PCA: principal component analysis; GBM: generalized boosting model; LASSO: least absolute shrinkage and selection operator; ROC: receiver operating characteristic; AUC: area under the curve.

Neural network model

The models with the highest AUC, on average, were the neural network and ridge regression model. The observed AUC for the neural network model was 0.811, and the median cross-validated AUC was 0.71 (25%, 0.70; 75%, 0.733), the accuracy was 0.667 (25%, 0.667; 75%, 0.688), the sensitivity was 0.909, and the specificity was 0.167. The cross-validated AUC for ridge regression model was 0.714 (25%, 0.667; 75%, 0.733), with an accuracy of 61% (25%, 0.611; 75%, 0.647). The performance metrics of neural network and ridge regression models are provided in Table 3. Variable importance for the final selected neural network model is reported in Table 4, which displays the five most important variables and their relative importance. Relative importance is scaled such that the most important variable has a value of 100. So, each variable’s importance was measured in relation to the highest performing variable. Relative importance for all variables is presented in Figure 2. The most important variable was age at diagnosis, with mean and MPP for SSF = 0 being the second and third most important, followed by skewness for SSF = 0 and SSF = 4.

Table 3.

Performance metrics for neural network model.

Metric	M	SD	Median	25%	75%
Performance metrics for neural network model
Accuracy	0.694	0.076	0.667	0.667	0.688
AUC	0.696	0.107	0.714	0.700	0.733
Sensitivity	0.925	0.077	0.909	0.900	1.000
Specificity	0.319	0.293	0.167	0.167	0.286
Performance metrics for ridge regression model
Accuracy	0.694	.096	0.611	0.611	0.647
AUC	0.686	0.095	0.714	0.667	0.733
Sensitivity	0.940	0.089	1.000	0.900	1.000
Specificity	0.167	0.289	.000	0.000	0.167

Table 4.

Relative importance for variables in the neural network model.

	Overall
Age at diagnosis	100
Mean.0	60.17689
MPP.0	60.17688
Skewness.0	54.0823
Skewness.4	34.1936

MPP: mean of positive pixels.

Figure 2.

Neural network model variable importance. Relative importance for all variables in the neural network model shows that age at diagnosis was the most important variable, with mean and mean of positive pixels for SSF=0 as the second and third most important, followed by skewness for SSF=0 and SSF=4.

Survival analysis

The Kaplan–Meier plot of the two survival groups based on the neural network model showed that the survival difference was significant (log-rank test, p = 0.004; Figure 3).

Figure 3.

Kaplan–Meier survival plot. Kaplan–Meier survival curve shows significant difference between the two survival groups.

Discussion

In this study, we computed first-order texture features from the whole tumor (enhancing and necrotic component) using all tumor slices (volumetric) and extracted from T1 CE sequence. We evaluated the performance of age and global heterogeneity using multiple classifier models to stratify survival. We found the neural network model to be the best performing model, with age, mean, and MPP at SSF = 0, and skewness at SSF = 0 and SSF = 4 as the most important predicting variables in dichotomizing the survival in patients with GBM. The ridge regression model also showed equivalent performance.

Our study showed that a combination of age and histogram based first-order textures extracted from the whole tumor could predict survival with good diagnostic performance (accuracy of 70%). Choi et al.¹⁹ also showed that a combination of clinical and texture features had higher predictive performance compared to radiomics alone. Similarly, Ingrisch et al.²⁴ found a significant association of whole tumor MRTA with survival in GBM. Similar to our study, Upadhaya et al.²⁷ and Grossman et al.² found the highest accuracy in survival prediction using texture features extracted from T1 CE imaging.

Machine learning–based prediction models have been used previously in assessing GBM survival.³²,³³ In a metanalysis by Sarkiss et al., the overall accuracy of machine learning models in predicting survival based on imaging and clinical data approached 80%.³⁴ However, there is still no consensus on the superiority of any one classifier model out of the multiple available options (random forest, support vector machine, logistic regression, etc.) in building these prediction algorithms. Most of the prior studies (Supplemental Table S1) used one or few classification models in survival prediction (commonly random forest or SVM). In contrast, we found the neural network and ridge regression as the two high performing models, with an inferior performance by SVM and random forest models. Thus, reliance on a single model for performance evaluation may be inadequate and affect clinical decision making. To prevent such bias, we tested 12 different machine learning classification models to explore the performance of different methods of GBM survival prediction. Our approach showed that survival classification using a single a priori selected machine learning model might be insufficient. In addition, to prevent overestimation of accuracy, we also used the approach of cross-validation to split the data into training and testing data subsets. This subdivision of data provides a better estimation of model accuracy.³⁵ Overall, this approach provides more flexibility and avoids bias in selecting a single classifier model type.

The best predictive model in any situation depends on many characteristics, including the number of features, the type of variables in the feature set, the presence of missing data, and the nature of the relationship between predictors and the outcome. In this case, the number of features was relatively small compared to the sample size, all variables in the feature set were continuous, and there was no missingness. In this data set, ridge regression vastly outperformed LASSO, providing evidence that most of the variables have some impact on the response as opposed to a few highly significant predictors and many with effects close to zero. This may also imply that the automatic variable selection done by the random forest model or LASSO does not improve its predictive performance over models that do not have embedded variable selection, such as the neural network. Finally, although ensemble methods can outperform other methods, ensemble classifiers are not guaranteed to be better than a single model and will only be better if the single model is unstable.

Interestingly, a recent study by Molina et al.³⁶ found that a simple prediction model based only on age and morphological MRI features (CE rim width, CE tumor volume, surface irregularity with no inclusion of texture features) outperformed the multiple machine learning models (using similar age and morphological MRI features or models based on texture features) in GBM survival prediction. They also found that the performance of machine learning models dipped significantly when multiple texture features were fed into the model, thereby stressing the known issue of overfitting. In our study, only a limited set of first-order texture features (36 features per patient) was extracted, and feature reduction was performed, thereby reducing overfitting.

MRTA based on multiparametric MRI sequences has been used to predict survival in GBM patients.³³,³⁷,³⁸ However, the computation of texture features from multiple sequences is a time-consuming task. This requires registration and resampling of sequences followed by tumor segmentation. Though most of the task may be automated or semi-automated, the process is still challenging in terms of technical expertise and time and may not be easily performed in the routine clinical setting. Similarly, a deep learning–based radiomics model has been used previously in the genomic and survival prediction of GBM patients.³⁹,⁴⁰ These models do not rely on preselected human input, unlike machine learning. However, deep learning techniques are data hungry and are considered as “black box” due to poor transparency regarding their functioning.⁴¹ Thus, our approach of computing limited first-order texture features from a single T1 CE sequence is a more pragmatic approach for the busy clinical environment.

Our study has several other limitations besides the drawbacks of a retrospective study. We did not evaluate MRTA performance on multiparametric imaging (T2-weighted, FLAIR, or ADC). Few prior studies have noted the high discriminating power of T1 CE derived texture features in predicting GBM survival compared to multiparametric imaging.¹²,²² A study by Liu et al.²² found T1 CE was the highest performing sequence in GBM survival prediction, and its results were comparable to multiparametric imaging. A similar filtration histogram–based first-order MRTA study by Lewis et al.²⁹ also showed the best performance of the T1 CE sequence for GBM genotyping, and thus we included only the T1 CE sequence for our analysis. We could not evaluate other clinical criteria (KPS or genomic status) influencing survival due to them not being available for most patients. However, in a study by Kickingereder et al.,²⁶ the radiomics model was shown to perform better in survival prediction compared to a clinical (age, KPS) or radiology model (perfusion and ADC maps). Our software was limited in providing only the first-order histogram-derived parameters, and thus we could not compare the diagnostic performance of higher-order texture features. Our study group was also slightly inhomogeneous regarding magnetic strength (we included both 3T and 1.5T scans). Magnet strength is known to affect texture features and may influence the results.⁴² However, in clinical practice, the selection of the scanner is difficult to control, and thus our study cohort represents a more pragmatic approach. We do anticipate that the application of the LoG band-pass filter in our study might have minimized the variation in texture parameters by removing noise and image heterogeneity. However, this needs further evaluation. We also did not include texture analysis from peritumoral edema that may provide additional information due to the infiltrative nature of GBM, as described by Choi et al.¹⁹

A strength of our study includes reasonable sample size and extraction of only first-order texture parameters from a single, routinely acquired T1 CE image, which is not a time-consuming task. Features were extracted without any additional processing (normalization, co-registration, etc.). These texture features were found to be predictive of survival times for the whole tumor analysis.

Conclusion

Our study evaluated the effectiveness of filtration-based first-order histogram texture parameters extracted from conventional T1-weighted CE imaging and the performance of multiple machine learning models in order to predict survival in GBM. Our results suggest that the neural network classifier model using first-order texture features and age can distinguish between short- and long-term survivors with good accuracy. Of the 12 machine learning models that were evaluated, neural network and ridge regression models showed the best overall performance.