Prediction of MGMT Status for Glioblastoma Patients Using Radiomics Feature Extraction from ¹⁸F-DOPA-PET Imaging

Abstract

Purpose

Methylation of the O⁶-methylguanine methyltransferase (MGMT) gene promoter is associated with improved treatment response and survival in patients with Glioblastoma (GB), but the necessary pathological specimen can be non-diagnostic. In this study, we assessed whether radiomics features pre-treatment ¹⁸F-DOPA PET imaging could be used to predict pathologic MGMT status.

Methods

This study included 86 patients with newly diagnosed GB, split into three groups (training, validating, predicting). We performed radiomics analysis on ¹⁸F-DOPA PET images by extracting features two tumor-based contours: a “Gold” contour of all abnormal uptake per expert Nuclear Medicine physician, and an “HGG” contour based on a T/N > 20 representing the most aggressive components. Feature selection was performed by comparing the weighted feature importance and filtering with bivariate analysis. Optimization of model parameters was explored using grid search with selected features. The stability of the model with increasing input features was also investigated for model robustness. The model predictions were then applied by comparing the overall survival (OS) probability of the GB patients with unknown MGMT status vs those with known MGMT status.

Results

A radiomics signature was constructed to predict MGMT methylation status. Using features extracted HGG contour alone with a Random Forest model, we achieved 80% ± 10% accuracy for 95% confidence level in predicting MGMT status. The prediction accuracy was not improved with the addition of the Gold contour or with more input features. The model was applied to the patients with unknown MGMT methylation status. The prediction results are consistent with what is expected using OS as a surrogate.

Conclusions

This study suggests that 3 features radiomics modeling of ¹⁸F-DOPA PET imaging can predict the MGMT methylation status with reasonable accuracy. It may provide valuable therapeutic guidance for patients where MGMT testing is inconclusive or non-diagnostic.

INTRODUCTION

Glioblastoma (GB) is the most common primary malignant brain tumor with a median progression free survival of only about 9 months and survival of approximately one year [1]. The standard treatment of GB is surgical resection followed by radiation therapy and chemotherapy.

Studies [2–4] indicate that promotion of the O⁶-methylguanine–DNA methyltransferase (MGMT) DNA-repair gene has been associated with significantly longer survival of GB patients, a median of 22 months. Knowledge of this biomarker can therefore impact the therapeutic regimen. MGMT unmethylated have a poor prognosis and may be better suited to hypofractionated radiotherapy courses or enrollment on clinical trials. MGMT status is an important factor to be considered to balance quality of life, oncologic outcome and toxicity. The current standard of care to obtain MGMT status is with pathologic assessment a biopsied specimen, which presents multiple challenges. First, a larger than diagnostic biopsy specimen is needed, which could be too invasive and not possible for every patient; second, the accuracy of the pathological analysis is highly dependent on biopsy location, yielding indetermined results for about 20%−50% [2, 5] of GB patients; third, laboratory access to this type of pathological assessment is limited and turn-around time may be too long to incorporate into treatment decisions. Therefore, it would be paramount to develop a tool that is capable of predicting MGMT promoter methylation status in a rapid, less invasive manner with reasonable accuracy.

Radiomics can be used as a possible tool to extract pathological or genomic information medical images [6, 7]. It has potential to provide more accurate information because the evaluation is no longer susceptible to sampling error, but rather on the entire tumor, and is not subjected to biopsy restrictions in eloquent brain. Selection of consistent imaging technique is important in radiomics assessment. Conventional T1-contrast-enhancing (T1-CE) and T2/FLAIR MR imaging is currently the standard of care used for GB diagnosis, surgical biopsy or resection planning, radiation therapy treatment planning, and serial follow-up imaging to evaluate progression. However, T1-CE within a brain tumor is highly dependent on the disruption of the blood-brain barrier, and can therefore present without contrast enhancement [8–12]. T2/FLAIR is also used, but lacks the differentiation between aggressive disease and edema [8, 9, 11, 13]. For PET imaging, ¹⁸F-FDG is the most popular and clinically utilized PET tracer, but uptake can be lower than or similar to that of normal gray matter, and can be increased in inflammatory lesions, which limits its application in GB tumor detection [14]. Amino acid tracers such as ¹⁸F-DOPA (3,4-dihydroxy-6-[¹⁸F]fluoro-L-phenylalanine) have shown high tumor-to-background signal and high sensitivity for glioma detection compared with MRI and FDG PET imaging [8, 14–16]. It has been successfully utilized for neurosurgical planning and radiotherapy target delineation [8, 17].

There have been efforts to develop radiomics models to predict pathology or treatment response using MR and PET imaging [18–23], but many of them lack validation to their models by independent test/validation data sets. In this work, by developing and validating a radiomics model, we aim to predict MGMT methylation status of GB patients through ¹⁸F-DOPA-PET imaging, and to use it as complimentary information for biopsy, especially for those patients where MGMT status cannot be determined pathologically. Our study is the first to look at using radiomics feature extraction ¹⁸F-DOPA-PET imaging in newly diagnosed GB patients to predict MGMT promotion status.

MATERIALS/METHODS

Patients Cohorts

The study cohort included patients three institutional prospective studies (NCT01165632, NCT02020720, NCT01991977) with ¹⁸F-DOPA-PET imaging. A total of 86 patients were identified with Grade IV newly diagnosed Glioblastoma, IDH-Mutant, IDH-wildtype, or NOS, between 2010 and 2018. All patients had ¹⁸F-DOPA PET imaging performed before starting radiotherapy. All patients had pathological assessment, including MGMT promoter methylation status whenever sufficient tissue was available.

Among the 86 patients, 69 had conclusive MGMT methylation status, and were randomly split into two groups: 59 patients in Group I for radiomics model training and 10 patients in Group II for model validation. The demographic and pathologic information for the total patient cohort and for each group is summarized in Table 1. The random split provided a similar demographic distribution of the patients in each group. Due to the limited number of patients, the percentage for the training group was intentionally made large to reduce the chance of overfitting, while the validation group size was still sufficient to show the statistical significance of the model predictions. The validation data (Group II) were not exposed to feature engineering or the modelling process, and were kept as an independent data set for validation only. The MGMT methylation status of the remaining 17 patients could not be determined pathologically, and were assigned to Group III, which was used as another independent and indirect test to our model by comparing their OS data as a surrogate to MGMT status.

Table 1:

Patient demographic and clinical data

Patient Cohort		All	Group I (Train)	Group II (Test)	Group III
Patients		86	59	10	17
Gender	Male	48	33	6	9
	Female	38	26	4	8
Age (year)	Minimum	19	19	21	28
	Maximum	86	77	68	86
	Mean	55	56	51	53
	Std	12	11	13	14
MGMT Status	Unmethylated	43	36	7	0
	Methylated	26	23	3	0
	Undetermined	17	0	0	17

The population of the enrolled patients was enriched for unmethylated, higher risk patients. Thus the MGMT status was biased towards unmethylated: among all the patients with conclusive MGMT methylation status, 63% of the patients were unmethylated. The IDH status of the patient cohort was highly biased toward wild-type (92% among all with determined IDH status). Consequently IDH information for modeling was not investigated in this work. It would be ideal to have a completely balanced data cohort, but the higher enrichment of the patients with unmethylated MGMT reflects the true clinical setting of the patients for radiation therapy. The selection of patients in the clinical trials was blind to MGMT status. The patients were referred to radiation oncology by surgeons and medical oncologists. It reflects the fact that radiation oncology often treats the most aggressive and unfavorable cases. With random split, a similar unbalance was maintained in both the training and validation cohorts (Table 1).

PET Image Data Acquisition

PET imaging was performed on a GE Discovery 690XT or a GE Discovery MI PET/CT system. ¹⁸F-DOPA was injected intravenously at a dose of 5 mCi ± 10%. A scout CT scan was acquired to aid in position the head within the field of view. The CT portion of the PET/CT scan, used for attenuation correction, was acquired with the following technique: 120 kVp, 35 mA, 1 second per rotation and a pitch of 0.984. The PET scan was then started 10 min after tracer injection. PET sinograms were reconstructed using a fully 3D iterative reconstruction algorithm with corrections for attenuation, scatter, randoms, deadtime, decay and normalization applied. The PET images were reconstructed into a 300 mm field of view with a pixel size of 1.17 mm and slice thickness of 1.96 mm and 279 mm for the Discovery 690XT and MI systems respectively [15]. The two scanners used in this study have matched spatial resolution, but Discovery MI has higher sensitivity. The PET images were then rigidly registered to and resampled to match the planning CT images with 1 mm slice thickness. The aligned PET images were used for feature extraction.

Delineation of Volumes of Interest

For each patient, two tumor volumes of interest (VOIs) were defined. An expert Nuclear Medicine physician blinded to pathological results contoured the gold-standard (“Gold”) ¹⁸F-DOPA PET-avid region of the tumor for each patient. The second VOI was delineated quantitatively with a specified tumor-to-normal hemispheric (T/N) ratio based on standardized uptake values (SUV). The normal hemisphere SUV uptake was based on a volumetric SUV mean of the normal contralateral brain of the patient. previous work, a T/N ratio larger than 20 can be used to define the highly aggressive portions of the PET-avid tumor region [8]. This second VOI is referred to as the “HGG” (High Grade Glioma) contour. Figure 1 illustrates 3 patients who were imaged with ¹⁸F-DOPA PET and the corresponding Gold (gold) and HGG (green) contours.

Fig. 1: Axial slice of 3 newly diagnosed GB patients.

a) Radiotherapy Planning CT, b) Registered ¹⁸F-DOPA PET, and c) Fused Image of PET and CT. The Gold (gold) contour and the HGG (green) contour are shown.

Radiomics Feature Extraction

The radiomics features were extracted with in-house software, using PyRadiomics [24] and python’s skicit-learn package. Three types of features were calculated the tumor VOIs: shape, tumor intensity and tumor texture, for both Gold and HGG contours, respectively. For each extraction setting (contour, bin width and filter), 26 shape based features and 19 intensity based features were extracted. The texture features contain four classes: gray level co-occurrence matrix (GLCM), gray level dependence matrix (GLDM), gray level run length matrix (GLRLM), and gray level size zone matrix (GLSZM). Respectively, 24, 16, 16 and 14 features were extracted for each texture class for any specific extraction configuration. With 16 different settings, which were permutations of binwidth (2, 4, 8, 16), two filters (original and LoG) and two contours (Gold and HGG), the total number of extracted radiomics imaging features was 1450. For multifocal lesions, the summed volume and summed area were used to calculate the shape features and the distances were calculated as pairwise Euclidean distance with the maximum expansion covering all the lesions. Patient age and tumor laterality were also considered, but all pathologic information was excluded the feature space to ensure that the model input was independent of pathological diagnosis.

Feature Selection

We used a multi-stage method to select features and reduce the dimensionality of the modeling. Because a high correlation was expected for a given feature extracted with different configurations, each feature set different extraction configurations was evaluated separately.

The features were first filtered by bivariate Pearson correlation coefficient, which was calculated for each possible pair of features. For a pair of features with the coefficient larger than the preset threshold, one feature in the pair was arbitrarily excluded selection. Approximately 70% of the features had a pair correlation that exceeded our threshold of 0.9 and were thus excluded.

The investigation of multiple feature interactions and the final feature selection was done with a weighted feature importance algorithm. This algorithm iterated through all the possible combinations of the filtered features with a fixed set size (n= 3, 4, 5) and calculated the summed Random Forest model-based feature importance, weighted by the F1 score, which combines the precision and sensitivity measures, of the model. Figure 2 illustrates the schematic workflow of the algorithm, which explored the multivariate nature of the data structure and tried to find a robust combination of the features with consistent performance.

Fig. 2:

Schematic workflow of feature selection.

Based on the summed feature-importance values, the features were ranked high to low. Then the features were again filtered by the Pearson correlation coefficient. If a feature x had a correlation coefficient larger than 0.5 with a higher ranked feature y, then feature x was excluded. The final n features (n = 3, 4, 5) were selected to ensure that they have the highest importance but low mutual correlations. For the model based with the combination of the Gold and HGG contour, the features both contours were combined and the selection process was the same as sketched in Figure 2, but in the last step at least one feature each contour was selected to make sure that the final group of features was not all a single contour.

Model Construction

Six machine learning models were evaluated with the selected features. The models included Extra Trees, Support Vector Machine (SVM) with linear kernel, SVM with Gaussian kernel, Random Forest (RF), XGBoost, and fully connected Neural Network (NN). The models were chosen due to their popularity and success in machine learning (for example [7], [19] and [25]).

To normalize the features with varying magnitudes and units, the selected feature data were rescaled with a standard scaler $X\prime =(X-\overline{X})/\sigma$, where $\overline{X}$ is the mean value of the feature and σ is the standard deviation. The scaler used for modeling was saved and the same scaler was recalled when the validation data was evaluated.

The parameters of each model were determined by grid search technique, which calculates different combinations of model parameter values (for example “max_depth”, “max_leaf_nodes” etc. for Random Forest), and each combination was tested with 5-fold cross validation. The set of parameters with the best AUC_ROC score was considered as the final choice of the parameters.

Model Validation and Clinical Application Exercise

The constructed models using the training data (Group I) were then tested with the Group II validation data set. By comparing the test accuracy, sensitivity and specificity, the machine learning model with the best accuracy and most consistent performance was chosen. In this stage, no adjustment to the model was attempted. The smallest feature size with best accuracy was preferred.

As an application, the chosen model was applied to the Group III cohort with indetermined MGMT status for prediction. Without the ground truth of the MGMT status of those patients, the OS of the patients with the predicted MGMT status was compared with that of the patients with known corresponding MGMT status. The similarity of the OS curves was used as a surrogate to indirectly confirm the model prediction.

RESULTS

Feature Selection

Based on the algorithm used in this work, the radiomics features selected for various feature set sizes are summarized in Table 2

Table 2:

Features selected for various set sizes. The category of a feature is indicated in the square bracket. In column 4, the tumor contour is indicated in the parentheses.

Size of feature set	Gold Contour	HGG Contour	HGG+Gold Contour
3	[gldm] HighGrayLevelEmphasis, [glcm] Imc2, [gldm] GrayLevelNonUniformity	[first_order] Kurtosis, [gldm] HighGrayLevelEmphasis, [shape] LeastAxisLength	(HGG) [first_order] Kurtosis, (HGG) [gldm] HighGrayLevelEmphasis, (Gold) [glcm] Imc2
4	[gldm] HighGrayLevelEmphasis, [glcm] Imc2, [gldm] GrayLevelNonUniformity, [shape] LeastAxisLength	[first_order] Kurtosis, [gldm] HighGrayLevelEmphasis, [shape] LeastAxisLength, [shape] Elongation	(HGG) [first_order] Kurtosis, (HGG) [gldm] HighGrayLevelEmphasis, (Gold) [glcm] Imc2, (Gold) [gldm] HighGrayLevelEmphasis,
5	[gldm] HighGrayLevelEmphasis, [glcm] Imc2, [gldm] GrayLevelNonUniformity, [shape] LeastAxisLength, [glcm] DifferenceEntropy	[shape] LeastAxisLength, [first_order] Kurtosis, [gldm] HighGrayLevelEmphasis, [shape] Elongation, [glcm] Autocorrelation	(HGG) [first_order] Kurtosis, (HGG) [gldm] HighGrayLevelEmphasis, (HGG) [shape] LeastAxisLength, (Gold) [glcm] Imc2, (Gold) [gldm] HighGrayLevelEmphasis,

Model Training and Validation

A series of models were trained with 5-fold cross validation for the “Gold” and “HGG” tumor segmentation independently and then combined. For all models tested, Random Forest showed higher accuracy and more robust performance for both “Gold” and “HGG” tumor contours. Therefore, the results presented here are the Random Forest classifier only.

Three features were extracted the HGG contour and tested by the model using the independent Group II validation dataset, with 78% prediction accuracy. The sensitivity and specificity were 83% and 67% respectively. Higher prediction capability for unmethylated MGMT patients was expected due to the slightly biased data. The test accuracy was also consistent with the achieved Group I training data accuracy (80%) using the same model. With the features extracted “Gold” contour, 80% validation accuracy was achieved for the Group II dataset. The sensitivity and the specificity with the Random Forest model were 100% and 33%, with an even stronger bias toward the unmethylated MGMT prediction. With 3-feature input, the accuracies for Group I training and Group II validation data were comparable (80% vs 78% respectively).

Including additional features in any radiomics model runs the risk of overfitting. To investigate the model accuracy and robustness, the models with 4-feature and 5-feature inputs were also studied. As shown in Figure 3 (a), features the HGG contours maintained similar validation accuracy, which is an indication of the robustness using the selected features. As shown in Figure 3 (b), including additional features the Gold contours increased the training accuracy, but significantly reduced the testing accuracy. This is a clear indication of overfitting.

Fig. 3:

The accuracy of Random Forest model to train and validation data sets as a function of the number of input features, for features based on (a) HGG tumor contour and (b) Gold tumor contour, respectively.

For the model trained with combined features both the HGG and Gold contours, no improvement in accuracy was observed. The best accuracy for the validation data set was 67%, less than the accuracy obtained either the Gold contour or the HGG contour-based model. The overfitting was also clearly seen for the 5-feature input model (98% accuracy for train data and 44% accuracy for validation data). Based on these findings and because the HGG contour is less subjective, the RF model with 3-feature set based on HGG contour was chosen for predictive modeling.

Clinical Application

The model developed in this work was then applied to predict MGMT status for the patients with indetermined MGMT status (Group III). Due to the strong correlation between treatment outcome and MGMT status, the OS was used as a comparative surrogate to the predictions. Because the Group III patients were enrolled on different trials and treated with various doses to various defined target volumes, the overall survival might differ between trials. Therefore, we chose only the patients a single trial with largest patient enrollment.

In the Group I and II patient cohorts, 63 patients were the same prospective study (NCT01991977). Their OS probabilities were fitted by Kaplan–Meier Estimators and shown in Figure 4. The median OS was 16 months for the patients with unmethylated MGMT and 39 months for the patients with methylated MGMT status.

Fig. 4: Kaplan–Meier Estimates of Overall Survival, grouped by MGMT Promoter Methylation Status.

All the patient data are the same prospective study. “Unmethylated” and “methylated” lines are for the patients with unambiguous MGMT status; “Predicted Unmethylated” line is for the patients without MGMT status pathological analysis but predicted to be unmethylated by the HGG contour based radiomics model. The p values of the curves with unmethylated status, with respect to the curve with methylated status, are calculated by the log rank test.

Our model was applied to the 12 of 17 Group III patients who had indetermined MGMT status and were enrolled on the same trial (NCT01991977). The patients with predicted unmethylated MGMT status by our model have the median survival of 18 months. Their OS curve is also plotted in Figure 4, showing similarity compared with the OS curve with known unmethylated MGMT status Group I and II and clear separation the OS curve with known methylated MGMT status. The patients with predicted methylated status are still alive at the time of this analysis. This agreement between OS suggests indirectly that our model has high predictive accuracy.

DISCUSSION

In this study, we demonstrated that a radiomics model based on ¹⁸F-DOPA PET images can be used to predict MGMT promotor methylation status which is a predictive biomarker for prognosis and an important factor to be considered in the treatment planning to balance life, oncologic outcome and toxicity.

The features used in modeling were selected by a multi-stage importance-ranked algorithm, but the interactions between features were not investigated. Because of the small sample size used in this study, the investigation on the interactions between features is not feasible. In the initial selection of the features based on Pearson Correlation coefficient, the choice of the feature in a highly correlated pair was also arbitrary. Therefore the selection of the best features may not be exclusive or optimal, but the objective of this work is to identify one set of features that can be used to build a robust prediction model for clinical applications.

Overfitting is one of the most significant challenges to radiomics studies due to small sample sizes usually available in medical studies. The importance of model validation can never be overestimated and the ultimate test to the model robustness should rely on independent data. Encouraging studies, which use PET-imaging based radiomics to predict grades, create predictive biomarkers or predict outcomes for GB, have been published and demonstrated excellent predictive accuracies, but often independent validation of models has not been emphasized. For example some studies [22, 26–29] relied on cross validation of the training cohort without using independent test/validation data, which makes it difficult to compare model performance; some studies [28, 29] used large numbers of features compared with the sizes of their patient cohort, which makes the model vulnerable to overfitting. In the current work, the model stability was emphasized by using both an independent validation cohort and a surrogate clinical application each based on an independent cohort. To ensure the stability of the model, the validation dataset was kept completely independent of the modeling process to avoid any contamination to the test validity. In this study, the number of the features chosen for modeling was 3 and we believe that a small parametric set can effectively reduce the chance of overfitting.

This work also finds that the HGG contour, which contains the most aggressive component of the tumors, provides more stable model performance. The results the features of combined HGG and Gold contours suggest that top-ranked features extracted HGG contour and Gold contour are not compatible in statistical distribution for the reasons that are not well understood. The fact that the HGG model works the best may suggest that the most aggressive disease component of the tumor is a more pronounced biological phenotype than its surroundings, which has also been indicated in other work [30]. Why, biologically, the most aggressive “core” part of the tumor is more predictive to the MGMT status is a very interesting topic but is beyond the scope of this study.

The HGG contour-based model was utilized to predict the MGMT status for the patients in whom pathological analysis was not possible (e.g. insufficient tumor sample) or indeterminate. The OS was used as a surrogate check to test the modeling prediction. Both the median OS and the distribution Kaplan–Meier analysis of the predicted patient cohort agree well with those of the patients the same prospective protocol study with known unmethylated MGMT status, and are well distinguished those the patient cohort with known methylated MGMT. The results provide indirect confirmation to the validity of our HGG-contour based model.

Due to the small sample size, the model’s training and validation is still limited. In addition to increasing sample size, future efforts will also be dedicated to incorporating our conventional, perfusion, and diffusion MRI parameters into feature extraction for more complex radiomics modeling. Future radiomics efforts may provide additional prognostic information to predict patient outcomes, patterns of failure, and additional genomic factors beyond what current pathologic markers such as MGMT predict.

CONCLUSION

MGMT gene promoter is one of the critical biomarkers for the predictive prognosis of GB, but currently susceptible to sampling bias, size and location restrictions, and accessibility to analysis. In this work, we constructed a radiomics model based on ¹⁸F-DOPA PET images before radiotherapy obtained on a prospective trial to predict MGMT methylation status. With careful feature engineering, 3 features PET images were chosen as the inputs to the tuned random forest model. About 80% accuracy was achieved for both Group I (training) and Group II (validation) patient cohorts.

Furthermore, the model was utilized to predict the MGMT status for the Group III patients whose MGMT status could not be determined pathologically. The predictions demonstrated agreement with the clinical survival outcomes, which indirectly confirms the validation of the radiomics model developed in this study.