Skip to main content
PMC home page
Nature Communications logoLink to Nature Communications
. 2019 Jul 18;10:3170. doi: 10.1038/s41467-019-11007-0

Multicenter study demonstrates radiomic features derived from magnetic resonance perfusion images identify pseudoprogression in glioblastoma

Nabil Elshafeey 1, Aikaterini Kotrotsou 1,2, Ahmed Hassan 1, Nancy Elshafei 2,3, Islam Hassan 2, Sara Ahmed 2, Srishti Abrol 1, Anand Agarwal 1, Kamel El Salek 1, Samuel Bergamaschi 4, Jay Acharya 4, Fanny E Moron 5, Meng Law 4,6, Gregory N Fuller 7, Jason T Huse 7, Pascal O Zinn 8,9,10,✉, Rivka R Colen 1,2,10,11,✉
PMCID: PMC6639324  PMID: 31320621

Abstract

Pseudoprogression (PsP) is a diagnostic clinical dilemma in cancer. In this study, we retrospectively analyse glioblastoma patients, and using their dynamic susceptibility contrast and dynamic contrast-enhanced perfusion MRI images we build a classifier using radiomic features obtained from both Ktrans and rCBV maps coupled with support vector machines. We achieve an accuracy of 90.82% (area under the curve (AUC) = 89.10%, sensitivity = 91.36%, 67 specificity = 88.24%, p = 0.017) in differentiating between pseudoprogression (PsP) and progressive disease (PD). The diagnostic performances of the models built using radiomic features from Ktrans and rCBV separately were equally high (Ktrans: AUC = 94%, 69 p = 0.012; rCBV: AUC = 89.8%, p = 0.004). Thus, this MR perfusion-based radiomic model demonstrates high accuracy, sensitivity and specificity in discriminating PsP from PD, thus provides a reliable alternative for noninvasive identification of PsP versus PD at the time of clinical/radiologic question. This study also illustrates the successful application of radiomic analysis as an advanced processing step on different MR perfusion maps.

Subject terms: Cancer, Cancer imaging

MRI scans of glioblastoma patients can be misleading and some patients appear to show features of progressive disease although they respond to treatment. Here, the authors use MRI images of progressive disease or pseudoprogression and build a classifier using machine learning to distinguish the two.

Introduction

The current standard of treatment for glioblastoma is the combination of maximal safe resection, radiation, and chemotherapy; this paradigm was shown to prolong the median overall survival to 14.6 months1,2. Immunotherapeutic agents, which utilize the body’s innate immune responses to kill cancerous cells, have demonstrated success in preclinical trials3–5. However, during treatment, the size of the tumor often increases and/or new inflammatory lesions appear. These transient changes typically stabilize or subside without further treatment6, but they are often difficult to distinguish from progressive disease (PD). This PD-mimicking phenomenon is called pseudoprogression (PsP). Patients with PsP have longer overall survival and are less likely to exhibit signs and symptoms of neurological deterioration than those with PD7,8.

Early discrimination of PsP from PD is a clinical challenge. Surgical biopsy is used in current clinical practice as a standard procedure for diagnosis of recurrent or residual disease, but biopsy is not only highly invasive but also limited in accuracy depending on the biopsy site, type of resection, and lesion heterogeneity9. Furthermore, biopsy is not feasible in every glioblastoma patient who has enhancement on post-treatment imaging9.

Conventional magnetic resonance imaging (MRI) and the Response Assessment in Neuro-Oncology (RANO) criteria are used as an alternative to surgical biopsy for distinguishing PsP from PD, but the diagnostic performance of this evaluation varies considerably10. By RANO criteria, patients with clinical deterioration, a new lesion, or an increase in existing tumor size (≥25% increase in the sum of products of perpendicular diameters) on restaging MR scans are defined as having PD, while those with an increase <25% or a decrease of <50% in the sum of products of perpendicular diameters of enhancing lesion are considered to have stable disease, as shown in Supplementary Table 110. Immunotherapy RANO (iRANO) criteria were devised as an update to RANO to monitor patients undergoing immunotherapy11. According to the iRANO criteria, patients with evidence of PD (per RANO) within 6 months of immunotherapy who do not develop considerable neurological decline should continue on the current therapy and undergo follow-up imaging in 3 months for confirmation of PD11. The drawback of this watchful waiting is that patients with PD will continue on an ineffective therapy, incurring the unnecessary risks of potential toxicity and delay in switching to more effective therapy; furthermore, failure to correctly identify PsP may result in premature discontinuation of an effective treatment10,11.

Recently, there have been numerous efforts using MR perfusion to differentiate PD from PsP12–14. MR perfusion, which can assess the vascular properties of the post-treatment enhancing lesions, calculates changes in blood volume, blood flow, and vessel wall permeability. These are important characteristics of tumor vessels15,16. However, lack of standardization in the post-processing steps of MR perfusion studies has led to a discrepancy in the reported cut-off values determining post-treatment changes from PD and corresponding sensitivity and specificity17,18. Therefore, there is a need for an accurate noninvasive quantitative tool to differentiate between PD and pseudoprogression at the time of detection of a brain MRI questionable lesion.

Radiomic analysis is a new automated, high-throughput method that quantifies the tumor phenotype at a microscale level (voxel/pixel level) by using thousands of image-based features obtained by histogram and texture analysis19–21. Unlike surgical biopsy, radiomic analysis assesses the entire three-dimensional tumor inclusive of spatial heterogeneity22,23. Prior studies have employed radiomic analysis on MR perfusion for predicting the underlying glioblastoma molecular phenotype20,24–26 and patient outcomes24,25 as well as distinguishing benign and malignant breast lesions27. However, to date, no multicenter investigations on the use of MR perfusion-based radiomic analysis to distinguish histologically-proven PsP from PD in glioblastoma have been reported.

In this multicenter study, we seek to determine the ability of MR perfusion-based radiomics to discriminate PsP from PD in glioblastoma patients. We evaluated 98 patients with pathologically-proven glioblastoma to identify the MR perfusion-based radiomic signatures of PsP and PD; we then combined these signatures to develop a noninvasive predictive model to robustly differentiate PsP from PD in the clinical setting. With an inclusive and diverse multi-institutional cohort, this pioneering study seeks to distinguish PD from PsP through MR perfusion-based radiomic analysis.

Results

Patient characteristics

We investigated a total of 98 patients from 3 institutions who had histopathologic evidence of PD (n = 76; 77.6%) or PsP (n = 22; 22.4%; Table 1). Using Mann-Whiteney test for non-prametric data and independent sample t test for parametric data, there were no statistical significant difference between the two groups in terms of age, sex, or Ktrans volume (p > 0.05). However, the exctracted VOI of rCBV using 3D slicer were differed significantly between PsP and PD groups (1126.91 mm3 and 1369.44 mm3, respectively; p = 0.010).

Table 1.

Baseline selected demographic and clinical characteristics of patients with Pseudo progression/Glioblastoma grade IV (N = 98)

Characteristic Pseudoprogression N = 22 Glioblastoma grade IV N = 76
Age, years (SD) 53.50 (14.88) 49.37 (12.97)
Sex, male, N (%) 15 (68.2%) 42 (54.7%)
Ktrans volume, mm3 (SD) 3166.99 (4618.17) 4591.56 (6233.76)
rCBV volume, mm3 (SD) 1126.91 (2771.41) 1396.44 (2245.20)
Surgical type:
Total resection, N (%) 19(86.4%) 45(59.2%)
Sub-total resection, N (%) 3(13.6%) 29(38.2%)
Biopsy, N (%) 0(0%) 2(2.6%)
Molecular status:
MGMT
  Methylated, N (%) 3(13.7%) 3(3.9%)
  Unmethylated, N (%) 1(4.5%) 10(13.2%)
  Non tested, N (%) 18(81.8%) 63(82.9%)
IDH
  Positive, N (%) 2(9%) 7(9.2%)
  Negative, N (%) 5(22.7%) 16(21%)
  Non tested, N (%) 15(68.3%) 53(69.8%)
Radio-therapy time, days (SD)** 54.21429(36.59) 56(22.24)
Time after RT to PD/PSP, days(SD)*** 779.92(766.59) 664.9857 (854.85)
Chemotherapy treatment (Temozolomide), N (%) 22 (100%) 76 (100%)
Open in a new tab

rCBV relative cerebral blood volume, SD standard deviation

*, Significant difference

**, 5 PSP and 8 PD with no available data

***, 2 PSP and 5 PD with no available data

Predictive value of perfusion parameters analysis

Perfusion parameter analysis showed that neither Ktrans nor rCBV was able to predict PsP from PD groups, and there was no statistically significant difference between the PsP and PD groups (Mann-Whitney test for non-parametric data; Supplementary Table 4).

Performance of the radiomic model

To determine the capacity of radiomic features to distinguish PsP from PD, we performed an integrative analysis assessing the predictive performance of perfusion radiomic features. The image post-processing workflow is shown in Fig. 1. We first ranked features based on their relevance to the outcome and within-feature redundancy using the MRMR feature selection technique. Subsequently, SVM with linear kernel and C5.0 models were constructed using the features selected by the MRMR analysis. Finally, the diagnostic performance of the models was evaluated using LOOCV for the SVM model and 10-fold cross-validation for the C5.0 model.

Fig. 1.

Fig. 1

Open in a new tab

Image post-processing radiomic workflow. a relative cerebral blood volume (rCBV) and Ktrans maps of perfusion MRI are acquired. b Segmentation of the region of interest using 3D slicer software. c Radiomic feature extraction from the whole tumor volume. d Statistical analysis: radiomic and clinical features are analyzed to determince their diagnostic and predictive values

In the analysis using Ktrans maps, the top 60 features (as ranked by the MRMR feature selection technique) robustly differentiate between PsP and PD. As shown in Fig. 2, selected radiomic features significantly discriminated between PsP and PD using the C5.0 method (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one sided binomial test). Similar results were obtained using the SVM method (AUC 89.8%, sensitivity 92.5%, specificity 88.89%, p-value 0.003744 one sided binomial test) (Fig. 2). We then validated the predictive models using LOOCV (AUC 90%, sensitivity 93%, specificity 89%, p-value 0.004 one sided binomial test, Fig. 2) and 10-fold cross-validation (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one sided binomial test, Fig. 2). Significant features used in the models are shown in Supplementary Table 5.

Fig. 2.

Fig. 2

Open in a new tab

Model building and evaluation using the selected Ktrans features (60 features). a, b ROC curve depicts the predictive model building using C5.0 (P-value 1.512e−11) and SVM methods (P-value 0.003744) respectively. c, d 10-fold cross-validation ROC curve (P-value 1.512e−11) and Leave-One-Out Cross-Validation (LOOCV) ROC curve (P-value 0.004) depicts the performance of the model

In the analysis using rCBV maps, the top 160 features (as identified by the MRMR feature selection technique) achieved the highest predictive accuracy in differentiating between PsP and PD. Significant features used in the models are shown in Supplementary Table 6. Radiomic models built using SVM significantly distinguished PsP and PD (AUC 94%, sensitivity 92%, specificity 100%, p-value 0.012 one-sided binomial test); similar results were obtained using the C5.0 classification (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one-sided binomial test). Validation using LOOCV (AUC 94%, sensitivity 92%, specificity 100%, p-value 0.012 one-sided binomial test, Fig. 3) and 10-fold cross-validation (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one-sided binomial test, Fig. 3) demonstrated to be highly statistically significant.

Fig. 3.

Fig. 3

Open in a new tab

Model building and evaluation using the selected rCBV features (160 features). a, b ROC curve depicts the predictive model building using C5.0 (P-value 1.512e−11) and SVM methods (P-value 0.012) respectively. c, d 10-fold cross-validation ROC curve (P-value 1.512e−11) and Leave-One-Out Cross-Validation (LOOCV) ROC curve (P-value 0.012) depicts the performance of the model

To determine the added value of combining the radiomic features obtained from both Ktrans and rCBV pharmacokinetic maps in predicting PsP from PD, we first ranked all 620 features identified by the MRMR technique and built further predictive models. Using 60 radiomic features (Supplementary Table 7) coupled with SVM, we achieved an accuracy of 90.82% (AUC 89.10%, sensitivity 91.36%, specificity 88.24%, p-value 0.017 one-sided binomial test); similar results were obtained using C5.0 (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one-sided binomial test). Differences identified by LOOCV predictive modeling (AUC 89%, sensitivity 91.4%, specificity 88%, p-value 0.02 one-sided binomial test, Fig. 4) and 10-fold cross-validation (AUC 100%, sensitivity 100%, specificity 100%, p-value 1.512e−11 one-sided binomial test, Fig. 4) were both statistically significant.

Fig. 4.

Fig. 4

Open in a new tab

Model building and evaluation using the selected merged Ktrans and rCBV features (60 features). a, b ROC curve depicts the predictive model building using C5.0 (P-value 1.512e−11) and SVM methods (P-value 0.017), respectively. c, d 10-fold cross-validation ROC curve (P-value 1.512e−11) and Leave-One-Out Cross-Validation (LOOCV) ROC curve (P-value 0.02) depicts the performance of the model

As the image acquisition parameters were distinct between different institutions, we sought to determine the scalability and generalizability of our radiomics analytical technique. Towards this end, we performed sub-analysis on the 63 MDACC patients, we achieved a high accuracy, sensitivity, and specificity to differentiate between PSP and PD using the three different extracted radiomic models obtained from Ktrans, rCBV and combined pharmacokinetic maps as we demonstrated in Supplementary Figs 1–3.

To assess the predictive power of the extracted radiomic features obtained from Ktrans, rCBV, and combined pharmacokinetic maps, we collated an independent prospective dataset of seven pathologically proven MDACC patients (Supplementary Table 2). Using the model built with whole dataset on the 98 patients as well as models built with cross-validation technique (LOOCV and 10-fold CV) for both C5.0 and SVM methods, the prediction results showed that the 7 new prospective patients were correctly classified as PD patients with high probability percentage as mentioned in Supplementary Tables 8–10.

Discussion

In this study, we evaluated the ability of MR perfusion-based radiomics to non-invasively discriminate between PsP and PD in glioblastoma patients at the time of clinical/radiologic question. We demonstrated that, although PsP and PD have similar radiographic appearances, they harbor distinct radiomic information that is hidden within MR perfusion images and can be extracted and used to build a clinically-relevant predictive model that discriminates PsP from PD.

Evaluation of post-treatment changes in glioblastoma patients using conventional MRI remains a challenge for clinicians and radiologist, as post-treatment MR images demonstrate confounding appearances28–30. Advanced MRI techniques, including perfusion MRI, have been proposed as alternatives to current imaging assessment criteria; however, reported cut-off values determining post-treatment changes versus PD vary widely across different studies. In the meta-analysis by Wan et al., they demonstrated substantial discrepancy between the cut-off values (see Wan et al.; Table 1)18, which they attributed to the differences in the acquisition parameters of DSC and DCE among different institutions. Further, the reported accuracy, sensitivity, and specificity of the perfusion studies are not clinically reliable17.

In our patient cohort, ROI-based perfusion parameter analysis using Ktrans and rCBV pharmacokinetic maps alone did not detect any differences between PsP and PD patient groups (Supplementary Table 4). However, our proposed radiomic model achieved high accuracy, sensitivity, and specificity in differentiating between PsP from PD compared to previous methods at the time of the clinical question of PsP versus PD; therefore, radiomics can be used by radiologists and oncologists for clinical decision-making (Figs. 2–4).

The radiomic model used in this study and proposed for ongoing clinical applications was built using imaging data from a multi-institutional cohort of patients for whom histopathologic confirmation of PsP or PD was available. The model demonstrated high discriminatory power, with AUCs greater than 89% (Figs. 2–4). Despite differences in the MR acquisition parameters of DSC and DCE imaging among the institutions (Supplementary Table 3), the radiomics predictive model demonstrated robust performance. Additionally, the discriminatory power was highly robust for the individual pharmacokinetic maps and after fusing the features from both the Ktrans and rCBV maps. These results suggest that our radiomic model is well-calibrated and can be directly applied in any clinical setting without the need to adjust the acquisition protocol or include both DSC and DCE sequences.

Further, the majority of the features included in our models are second-order (Supplementary Tables 5–7); of the key features selected (60 for Ktrans, 160 for rCBV and 60 for fused model), the number of second-order features was 54 (90%) for the Ktrans-based model (Supplementary Table 5), 150 (94%) for the rCBV-based model (Supplementary Table 6), and 54 (90%) for the fused model (Supplementary Table 7). This finding is concordant with the fact that histogram features depict the data distribution within a selected region, and therefore their value is directly affected by the differences in acquisition parameters. Conversely, second-order features capture the inter-relationship between neighboring voxels, and thus are not sensitive to the absolute data value. Hence, by incorporating second-order radiomic features, the model is more robust and ‘immune’ to the MR acquisition parameters.

The robustness of our extracted radiomic features is further demonstrated by the similarity in calculated AUC, sensitivity, and specificity for the two classification algorithms investigated: (i) SVM with linear kernel and (ii) C5.0. A previous study investigated the performance and stability of various feature selection and classifier methods in radiomic applications31. They reported variable accuracy for different combinations of feature selection and classifier methods. Although evaluation of classifier methods and feature selection techniques was beyond the scope of this work, our findings indicate that our extracted radiomic features were stable and performed equally well independent of the classifier method31.

Further, the key features selected for both Ktrans- and rCBV-based models were entropy, sum of squares, and autocorrelation (Supplementary Tables 5–7). These features hold information about the spatial heterogeneity of the studied region and are more sensitive when assessing tumor microenvironment and the presence of tumor cells. The finidngs of the present study confirmed and extended previous work by Zinn et al. who showed that MRI features (Diffsuion and radiomic features) capture spatial heterogeneity25,32,33. Our results show that patients with PD have high entropy and sum of squares and low autocorrelation compared to PsP patients. Thus, selected volumes in PD patients were characterized by non-uniformity and increased complexity compared to PsP patients. This may reflect the heterogeneity seen microscopically34–36.

Limitations of the current study mostly result from the requirement for histopathologic confirmation of PsP or PD. First, the percentage of patients with PsP is low, and our study design did not include patients for whom histopathologic confirmation was not available. Because biopsy is an invasive procedure and clinicians usually prefer a watchful approach over biopsy, it is expected that only a small fraction of patient who have undergone biopsy will have PsP. Second, we performed semi-automated segmentation of the VOI, which is a time-consuming process and also requires an in-depth assessment of inter-rater variability and stability of the extracted radiomic features. A user-friendly tool that performs reliable automatic segmentations is needed, as it would allow for easy translation of the radiomic analysis in the clinical setting. Additionally, being a retrospective study, important information, such as molecular and genetic characteristics, was not available to review. For example, the contribution of MGMT methylation and IDH1/2 mutations, was not investigated in this work, since this information was not available for the majority of the studied population. Finally, the total number of patients included in the study was relatively small and did not allow for a large validation prospective dataset using as a separate cohort. A much larger study with a larger patient cohort that will allow independent validation is needed; further the prospective validation component is currently performed in our institution.

In summary, our study presents a radiomic model based on MR perfusion data for noninvasive, individualized prediction of PsP in glioblastoma patients at the point-of-care (at the time of the clinical question). We demonstrate the high predictive accuracy of radiomics in differentiating PsP from PD. Our radiomic model can be easily integrated into the clinical setting, as it is a post-processing approach that does not require changes in the current imaging protocol and will allow clinicians to make more informed decisions for optimal patient care. After outlining the VOI on DCE and DSC maps, the model will indicate whether the patient has PsP or PD. Our radiomic model will complement the methods now available to clinicians by offering a comprehensive evaluation of the imaging data. We expect that it will dramatically decrease the number of invasive procedures performed to confirm the absence of tumor in patients; Most importantly, our radiomics method can help ensure that patients with PsP will continue on an effective treatment or discontinue an ineffective treatment without the need for delays caused by watchful waiting. Conversely, early identification of PD will allow the prompt transition to a more effective treatment.

Methods

Study population

This retrospective multi-institutional clinical study was conducted in compliance with U.S. Health Insurance Portability and Accountability Act (HIPAA) regulations. All necessary institutional ethics review approvals were obtained by the participating institutions (The University of Texas MD Anderson Cancer Center, Baylor College of Medicine, and University of Southern California Keck School of Medicine).

A total of 651 patients with histopathologic diagnosis of GBM and an enhancing lesion on MRI that may indicate PD or treatment-related changes were received from three institutions; The University of Texas MD Anderson Cancer Center (MDACC) (598 patients), Baylor College of Medicine (BCM) (13 patients), and University of Southern California (USC) Keck School of Medicine (40 patients). A total of 98 patients were included in the final analysis based on the following inclusion criteria: (1) minimum age of 18 years; (2) enhancing lesion on MRI that may indicate PD or treatment-related changes; (3) conventional and advanced MRI, including dynamic susceptibility contrast (DSC) and dynamic contrast-enhanced (DCE) perfusion imaging; (4) pathologic confirmation of PD or PsP within 3 months after detection of a questionable enhancing lesion on MRI (time of question); (5) adequate image quality with no artifact affecting the radiomic analysis. 553 patients did not meet the inclusion criteria and therefore excluded. 8 patients (1 BCM, 7 USC) did not have dynamic susceptibility contrast-enhanced T2*-weighted (DSC) and dynamic contrast-enhanced (DCE) perfusion; 85 patients (82 MDACC, 2 BCM, and 1USC) were excluded due to poor quality of imaging/perfusion data; 460 (453 MDACC, 4 USC, and 3 BCM) were excluded due to absence of histopathological confirmation of PsP versus PD.

The clinical, radiologic, and pathologic data were retrieved from the medical records. Our final study cohort consisted of: 63 (50PD and 13PSP) patients from MDACC, 7 (6PD and 1PSP) patients from BCM, and 28 (21PD and 7PSP) patients from USC. Based on the pathologic diagnosis, 76 (77.6%) patients had PD and 22 (22.4%) patients had treatment-related changes considered to be PsP. Detailed demographic, clinical characteristics and treatment information is presented in Table 1.

For further validation of our results, a prospective study of 7 MDACC patients with pathologically proven PD were included using the same aforementioned inclusion and exclusion criteria. (Supplementary Table 2 showed the demographic, clinical and pathological data)

Histopathologic evaluation

All patients had undergone biopsy or surgical resection, and resected entire tumor/tissue block specimens were histopathologically examined. The pathology status as PD or PSP was decided and reviewed in consensus by two experienced board-certified pathologists (J.T.H. and G.N.F.) blinded to the MRI data. PD was defined as samples with either solely recurrent GBM tissue or mixture of radiation necrosis and recurrent GBM, while PSP was determined by the presence of radiation necrosis only or mixture between radiation necrosis and reactive gliosis.

MRI acquisitions and imaging post processing

For conventional MRI acquisition, images were acquired using a 1.5 or 3.0 Tesla MR Scanner. The MRI protocol for MDACC included an axial T1-weighted sequence (repetition time [TR], 700 ms; echo time [TE], 12 ms; slice thickness, 5 mm; acquisition matrix 352 × 224), an axial fluid attenuation inversion recovery (FLAIR) sequence (TR, 10,000 ms; TE, 140 ms; slice thickness, 5 mm; acquisition matrix, 256 × 256), and an axial post-contrast T1-weighted sequence acquired 5 min after the contrast injection (TR, 750 ms; TE, 13 ms; slice thickness, 5 mm; acquisition matrix, 384 × 256). The MRI protocol for USC included an axial T1-weighted sequence (TR, 600 ms; TE, 8 ms; slice thickness, 5 mm; acquisition matrix, 320 × 192), an axial FLAIR sequence (TR, 8000 ms; TE, 151.2 ms; slice thickness, 5 mm; acquisition matrix, 320 × 224), and an axial post-contrast T1-weighted sequence acquired 5 min after the contrast injection (TR, 615 ms; TE, 17 ms; slice thickness, 5 mm; acquisition matrix, 320 × 192). The MRI protocol for BCM included an axial T1-weighted sequence (TR, 750 ms; TE, 11 ms; slice thickness, 5 mm; acquisition matrix, 384 × 192), an axial FLAIR sequence (TR, 2541 ms; TE, 16.4 ms; slice thickness, 5 mm; acquisition matrix, 384 × 192), and an axial post-contrast T1-weighted sequence acquired 5 min after the contrast injection (TR, 750 ms; TE, 11 ms; slice thickness, 5 mm; acquisition matrix, 384 × 192). while for advanced MRI acquisition: The parameters of both DCE and DSC for each institution are shown in Supplementary Table 3. Pre-contrast T1 mapping images were acquired with six flip angles (2, 5, 10, 15, 20, and 25 degrees). Images were acquired before and after injection of passively targeted gadolinium-diethylenetriamine penta-acetic acid (0.1 mmol per kg).

The advanced imaging analysis and pharmacokinetic map calculation were performed using Nordic ICE (NordicNeuroLab, Bergen, Norway). Images were segmented using 3D Slicer (version 4.3.1, https://www.slicer.org), an open-source software program widely used for image visualization and segmentation37,38. The image post-processing workflow is shown in Fig. 1.

DCE data were analyzed by using Nordic ICE, using the dual compartment modified Tofts and Kermode pharmacokinetic model as described elsewhere39,40. We manually selected the arterial input function in the middle cerebral artery located on the ipsilateral side, then the pharmacokinetic Ktrans parameter (reflects local blood flow, endothelial permeability, endothelial surface area, proportional blood volume within a given voxel) was obtained41. For DSC analysis, the arterial input function was manually selected in the middle cerebral artery located on the ipsilateral side. Using the integrated DSC module, which incorporates a relative cerebral blood volume (rCBV) leakage-correction algorithm and manual noise thresholding, we obtained the amount of blood in a given volume of tissue expressed as mL per 100 mL of tissue41.

Perfusion parametric maps analysis

We performed a perfusion parameter analysis to determine whether perfusion parameters can differentiate between PD and PsP. Both capillary permeability (Ktrans) and leakage-corrected rCBV maps were rigidly registered based on mutual information with the post-contrast T1WI using Nordic ICE; the resulting registrations were visually inspected to ensure adequate alignment. Subsequently, circular regions-of-interest (ROIs) with an area of 10.55 mm2 were manually delineated on the area with the highest parameter value for each parameter map, ensuring that necrotic areas and large blood vessels were excluded. The mean values were extracted from the ROIs for each parameter map. To reduce sampling error, four ROIs were selected and the average of the mean values was obtained42. This method has been demonstrated to provide the optimal interobserver and intra-observer reproducibility43.

Quantitative radiomic analysis and model building

Post-contrast T1WI were coregistered with Ktrans and rCBV maps to ensure appropriate tumor delineation; cystic, necrotic regions and intralesional macrovessels were excluded. The coregistered images for both Ktrans and leakage-corrected rCBV maps were semi-automatically segmented by an experienced user (N.E., 4 years experience) and manually reviewed slice-by-slice by an experienced board-certified radiologist (R.R.C., 9 years experience), both of whom were blinded to histopathologic assessment. Three-dimensional volumes of interest (VOIs) were subsequently extracted for further analysis. A total of 310 radiomic features were extracted from each pharmaocokinetic parametric map, consisting of 10 histogram-based features44 and 300 s-order Haralick features45–47. Accounting for the Ktrans and leakage-corrected rCBV maps analyzed in this study, a total of 620 radiomic features were obtained per patient26,48,49. Radiomic analysis was performed using our in-house developed software implemented in MATLAB (version 2017b; MathWorks Inc; Natick, MA).

Histogram-based features describe the Ktrans and rCBV values distribution within the VOI; the features obtained were minimum, maximum, mean, standard deviation, skewness, kurtosis, and the percentiles 1%, 5%, 95%, and 99%. The second-order Haralick features were computed from the gray-level co-occurrence matrix (GLCM) after requantization of the image gray levels; the gray levels implemented in this analysis were 8, 16, 32, 64, and 256. The second-order Haralick features describe mathematical relationships between co-occurring voxels separated by a given distance in a specific direction. For the purposes of this analysis, we investigated a distance of 1 voxel and 4 in-plane directions. Twenty GLCM-based features were obtained per gray level: autocorrelation, contrast, correlation, cluster shade, cluster prominence, dissimilarity, energy, entropy, homogeneity, maximum probability, variance, sum average, sum variance, sum entropy, difference variance, difference entropy, information measure of correlation 1, information measure of correlation 2, inverse difference moment, and normalized inverse difference moment. Rotation invariant measures of the features were obtained by calculating the average, range, and angular variance of the features across the four in-plane directions. Taking into account the number of gray levels and the number of rotation invariant measures, a total of 300 were calculated per map. Fig. 1 shows the pipeline used to get the radiomic perfusion model.

To investigate the predictive performance of extracted radiomic features in discriminating between PD and PsP (outcome), we first identified the radiomic features (feature set) that are associated with outcome. To minimize irrelevant and redundant radiomic features, we used the Maximum Relevance Minimum Redundancy (MRMR) feature selection technique; MRMR ranks the features based on maximizing the relevance and avoiding excess redundancy50. The MRMR approach was implemented using the mRMRe R package.

Different sizes of feature sets were investigated to ensure selection of the fetaureset size with best prediction accuracy. We started with the MRMR-ranked 310 features and evaluated different featureset sizes ranging from 310 to 20 in steps of 10.

MRMR-ranked radiomic features were used for classification and model building. Two different supervised learning algorithms were used in this analysis for classification and predictive model building: Support Vector Machine (SVM) and decision tree algorithm C5.0. In the C5.0 algorithm, a decision tree is used while modeling the classification process51. SVM is a supervised machine learning algorithm that is used for solving classification problems by transforming the feature space to a higher dimension space so that a separating hyperplane maximizes the distance between the two classifiers52. To increase model accuracy in our analysis, we tested the classifier in SVM with three different kernels (linear, polynomial, and radial basis)53.

We used leave-one-out cross-validation (LOOCV) and k-fold cross-validation (k = 10) to evaluate the accuracy, sensitivity, and specificity of the models54. Receiver Operating Characteristic (ROC) curves were plotted; and the area under the curve (AUC), accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value, and p-value were reported for each cross-validation and prediction output.

Statistical analysis

Statistical analyses was performed using R software, version 3.4.3 (https://www.r-project.org)55. The statistical significance was two-sided in this study, with significance level <0.05. Prior to univariate analysis, the fitness of numeric dataset to normal distribution was determined by the Kolmogorov– Smirnov test. Differences in non-parametric data, such as demographic and clinical characteristics and individual perfusion parameters (rCBV and Ktrans), between the PD and PsP groups were assessed using the Mann-Whitney U test.

Supplementary information

Supplementary Information (928.7KB, pdf)

Acknowledgements

This work was supported by the John S. Dunn Sr. Distinguished Chair in Diagnostic Imaging fund, MD Anderson Cancer Center startup funding, Multi-Investigator Imaging (MI2) Research Award. Radiological Society of North America scholar grant (RSCH11506). The Cancer Prevention & Research Institute of Texas (RP160150) and by the U.S. National Cancer Institute through the MD Anderson Cancer Center Support Grant (P30CA016672) (RRC).

Author contribution

N. Elshafeey. and R.R.C conceived the study; S. Ahmed., S. Abrol., A.H., I.H., A.A., K.S., J.A. and S.B. collected patient clinical data; N. Elshafeey. and A.K. designed and performed the imaging analysis; N. Elshafeey., N. Elshafei., and A.K. performed the statistical analysis; J.T.H. and G.N.F. pathology evaluation and review; N. Elshafeey. and A.H. prepared the figures and tables; F.M and M.L. provided patient data from collaborating institutes; N. Elshafeey., A.K., P.O.Z., F.M., M.L. and M.L. provided support for the interpretation of the results; N. Elshafeey. and A.K. drafted the manuscript; N. Elshafeey., A.K., F.M. and R.R.C critically revised the article for important intellectual content. All authors read and approved the final version of the article.

Data availability

The data is available within the Article or the Supplementary Information. The imaging data that support the findings of this study are available from the corresponding author upon request.

Code availability

For the MR perfusion analysis, we used Nordic ICE, a commercially available processing platform (https://www.nordicneurolab.com/products/nordicICE.html). For volumetric analysis we used 3D Slicer (version 4.3.1, https://www.slicer.org/), an open-source software program widely used for image visualization and segmentation. The pipeline for the radiomic analysis has been introduced in Zinn et al.20. For the biostatistical analysis, we used R software (version 3.4.0, R Foundation for Statistical Computing, Vienna, Austria); package MRMRe (version 2.0.7) and C5 decision tree (version 0.1.2) and SVM (version 1.7–0.1)50–52.

Competing interests

Meng Law receives Honorarium and Research Grant Support from Bracco Diagnostics. The remaining authors declare no competing interests.

Footnotes

Peer review information: Nature Communications thanks the anonymous reviewers for their contribution to the peer review of this work.

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Contributor Information

Pascal O. Zinn, Email: pascal.zinn@gmail.com

Rivka R. Colen, Email: rrcolen@gmail.com

Supplementary information

Supplementary Information accompanies this paper at 10.1038/s41467-019-11007-0.

References

  • 1.Holland EC. Glioblastoma multiforme: The terminator. Proc. Natl Acad. Sci. USA. 2000;97:6242–6244. doi: 10.1073/pnas.97.12.6242. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 2.Ostrom QT, et al. CBTRUS Statistical report: primary brain and other central nervous system tumors diagnosed in the United States in 2009–2013. Neuro-Oncol. 2016;18:v1–v75. doi: 10.1093/neuonc/now207. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 3.Wainwright DA, et al. Durable therapeutic efficacy utilizing combinatorial blockade against IDO, CTLA-4, and PD-L1 in mice with brain tumors. Clin. Cancer Res. : Off. J. Am. Assoc. Cancer Res. 2014;20:5290–5301. doi: 10.1158/1078-0432.CCR-14-0514. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 4.Antonios JP, et al. Detection of immune responses after immunotherapy in glioblastoma using PET and MRI. Proc. Natl Acad. Sci. USA. 2017;114:10220–10225. doi: 10.1073/pnas.1706689114. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 5.Zeng J, et al. Anti-PD-1 blockade and stereotactic radiation produce long-term survival in mice with intracranial gliomas. Int. J. Radiat. Oncol., Biol., Phys. 2013;86:343–349. doi: 10.1016/j.ijrobp.2012.12.025. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 6.Lin AL, et al. Molecular and histologic characteristics of pseudoprogression in diffuse gliomas. J. neuro-Oncol. 2016;130:529–533. doi: 10.1007/s11060-016-2247-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Chamberlain MC, Glantz MJ, Chalmers L, Van Horn A, Sloan AE. Early necrosis following concurrent Temodar and radiotherapy in patients with glioblastoma. J. Neuro-Oncol. 2007;82:81–83. doi: 10.1007/s11060-006-9241-y. [DOI] [PubMed] [Google Scholar]
  • 8.de Wit MCY, de Bruin HG, Eijkenboom W, Sillevis Smitt PAE, van den Bent MJ. Immediate post-radiotherapy changes in malignant glioma can mimic tumor progression. Neurology. 2004;63:535–537. doi: 10.1212/01.WNL.0000133398.11870.9A. [DOI] [PubMed] [Google Scholar]
  • 9.Hygino da Cruz LC, Rodriguez I, Domingues RC, Gasparetto EL, Sorensen AG. Pseudoprogression and pseudoresponse: imaging challenges in the assessment of posttreatment glioma. Am. J. Neuroradiol. 2011;32:1978–1985. doi: 10.3174/ajnr.A2397. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 10.Wen PY, et al. Updated response assessment criteria for high-grade gliomas: response assessment in neuro-oncology working group. J. Clin. Oncol. 2010;28:1963–1972. doi: 10.1200/JCO.2009.26.3541. [DOI] [PubMed] [Google Scholar]
  • 11.Okada Hideho, Weller Michael, Huang Raymond, Finocchiaro Gaetano, Gilbert Mark R, Wick Wolfgang, Ellingson Benjamin M, Hashimoto Naoya, Pollack Ian F, Brandes Alba A, Franceschi Enrico, Herold-Mende Christel, Nayak Lakshmi, Panigrahy Ashok, Pope Whitney B, Prins Robert, Sampson John H, Wen Patrick Y, Reardon David A. Immunotherapy response assessment in neuro-oncology: a report of the RANO working group. The Lancet Oncology. 2015;16(15):e534–e542. doi: 10.1016/S1470-2045(15)00088-1. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 12.Young RJ, et al. MRI perfusion in determining pseudoprogression in patients with glioblastoma. Clin. imaging. 2013;37:41–49. doi: 10.1016/j.clinimag.2012.02.016. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 13.Prager A.J., Martinez N., Beal K., Omuro A., Zhang Z., Young R.J. Diffusion and Perfusion MRI to Differentiate Treatment-Related Changes Including Pseudoprogression from Recurrent Tumors in High-Grade Gliomas with Histopathologic Evidence. American Journal of Neuroradiology. 2015;36(5):877–885. doi: 10.3174/ajnr.A4218. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 14.Ye J, et al. Differentiation between recurrent gliomas and radiation necrosis using arterial spin labeling perfusion imaging. Exp. Ther. Med. 2016;11:2432–2436. doi: 10.3892/etm.2016.3225. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 15.Essig M, et al. Perfusion MRI: the five most frequently asked technical questions. Ajr. Am. J. Roentgenol. 2013;200:24–34. doi: 10.2214/AJR.12.9543. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Law M. Advanced imaging techniques in brain tumors. Cancer Imaging. 2009;9:S4–S9. doi: 10.1102/1470-7330.2009.9002. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Huang, R. Y., Neagu, M. R., Reardon, D. A. & Wen, P. Y. Pitfalls in the neuroimaging of glioblastoma in the era of antiangiogenic and immuno/targeted therapy—detecting illusive disease, defining response. Fron. Neurol6, 33 (2015). [DOI] [PMC free article] [PubMed]
  • 18.Wan B, et al. The diagnostic performance of perfusion MRI for differentiating glioma recurrence from pseudoprogression: A meta-analysis. Medicine. 2017;96:e6333. doi: 10.1097/MD.0000000000006333. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 19.Abrol S, Kotrotsou A, Salem A, Zinn PO, Colen RR. Radiomic phenotyping in brain cancer to unravel hidden information in medical images. Top. Magn. Reson. imaging : TMRI. 2017;26:43–53. doi: 10.1097/RMR.0000000000000117. [DOI] [PubMed] [Google Scholar]
  • 20.Zinn Pascal O., Singh Sanjay K., Kotrotsou Aikaterini, Hassan Islam, Thomas Ginu, Luedi Markus M., Elakkad Ahmed, Elshafeey Nabil, Idris Tagwa, Mosley Jennifer, Gumin Joy, Fuller Gregory N., de Groot John F., Baladandayuthapani Veera, Sulman Erik P., Kumar Ashok J., Sawaya Raymond, Lang Frederick F., Piwnica-Worms David, Colen Rivka R. A Coclinical Radiogenomic Validation Study: Conserved Magnetic Resonance Radiomic Appearance of Periostin-Expressing Glioblastoma in Patients and Xenograft Models. Clinical Cancer Research. 2018;24(24):6288–6299. doi: 10.1158/1078-0432.CCR-17-3420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 21.Kotrotsou A, Zinn PO, Colen RR. Radiomics in brain tumors: an emerging technique for characterization of tumor environment. Magn. Reson. Imaging Clin. North Am. 2016;24:719–729. doi: 10.1016/j.mric.2016.06.006. [DOI] [PubMed] [Google Scholar]
  • 22.Castellano G, Bonilha L, Li LM, Cendes F. Texture analysis of medical images. Clin. Radiol. 2004;59:1061–1069. doi: 10.1016/j.crad.2004.07.008. [DOI] [PubMed] [Google Scholar]
  • 23.Davnall F, et al. Assessment of tumor heterogeneity: an emerging imaging tool for clinical practice? Insights into Imaging. 2012;3:573–589. doi: 10.1007/s13244-012-0196-6. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 24.Kong DS, et al. Quantitative radiomic profiling of glioblastoma represents transcriptomic expression. Oncotarget. 2018;9:6336–6345. doi: 10.18632/oncotarget.23975. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 25.Rao A, et al. A combinatorial radiographic phenotype may stratify patient survival and be associated with invasion and proliferation characteristics in glioblastoma. J. .Neurosurg. 2016;124:1008. doi: 10.3171/2015.4.JNS142732. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 26.Zinn PO, et al. Distinct radiomic phenotypes define glioblastoma TP53-PTEN-EGFR mutational landscape. Neurosurgery. 2017;64:203–210. doi: 10.1093/neuros/nyx316. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 27.Parekh, V. S. & Jacobs, M. A. Integrated radiomic framework for breast cancer and tumor biology using advanced machine learning and multiparametric MRI. NPJ Breast Cancer3, 10.1038/s41523-017-0045-3 (2017). [DOI] [PMC free article] [PubMed]
  • 28.Shah R, et al. Radiation necrosis in the brain: imaging features and differentiation from tumor recurrence. Radiographics. 2012;32:1343–1359. doi: 10.1148/rg.325125002. [DOI] [PubMed] [Google Scholar]
  • 29.Kumar AJ, et al. Malignant gliomas: MR imaging spectrum of radiation therapy- and chemotherapy-induced necrosis of the brain after treatment. Radiology. 2000;217:377–384. doi: 10.1148/radiology.217.2.r00nv36377. [DOI] [PubMed] [Google Scholar]
  • 30.Ulmer S, et al. Clinical and radiographic features of peritumoral infarction following resection of glioblastoma. Neurology. 2006;67:1668–1670. doi: 10.1212/01.wnl.0000242894.21705.3c. [DOI] [PubMed] [Google Scholar]
  • 31.Parmar C, Grossmann P, Bussink J, Lambin P, Aerts HJWL. Machine learning methods for quantitative radiomic. Biomark. Sci. Rep. 2015;5:13087. doi: 10.1038/srep13087. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 32.Zinn PO, et al. Diffusion weighted magnetic resonance imaging radiophenotypes and associated molecular pathways in glioblastoma. Neurosurgery. 2016;63(Suppl 1):127–135. doi: 10.1227/NEU.0000000000001302. [DOI] [PubMed] [Google Scholar]
  • 33.Zinn Pascal O., Singh Sanjay K., Kotrotsou Aikaterini, Hassan Islam, Thomas Ginu, Luedi Markus M., Elakkad Ahmed, Elshafeey Nabil, Idris Tagwa, Mosley Jennifer, Gumin Joy, Fuller Gregory N., de Groot John F., Baladandayuthapani Veera, Sulman Erik P., Kumar Ashok J., Sawaya Raymond, Lang Frederick F., Piwnica-Worms David, Colen Rivka R. A Coclinical Radiogenomic Validation Study: Conserved Magnetic Resonance Radiomic Appearance of Periostin-Expressing Glioblastoma in Patients and Xenograft Models. Clinical Cancer Research. 2018;24(24):6288–6299. doi: 10.1158/1078-0432.CCR-17-3420. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 34.Qin J-b, et al. Grading of gliomas by using radiomic features on multiple magnetic resonance imaging (MRI) sequences. Med. Sci. Monit. 2017;23:2168–2178. doi: 10.12659/MSM.901270. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 35.Hall-Beyer M. Practical guidelines for choosing GLCM textures to use in landscape classification tasks over a range of moderate spatial scales. Int. J. Remote Sens. 2017;38:1312–1338. doi: 10.1080/01431161.2016.1278314. [DOI] [Google Scholar]
  • 36.Tiwari P, et al. Computer-extracted texture features to distinguish cerebral radionecrosis from recurrent brain tumors on multiparametric MRI: a feasibility study. Am. J. Neuroradiol. 2016;37:2231. doi: 10.3174/ajnr.A4931. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 37.Zinn PO, et al. Radiogenomic mapping of edema/cellular invasion MRI-phenotypes in glioblastoma multiforme. PloS one. 2011;6:e25451. doi: 10.1371/journal.pone.0025451. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 38.Zinn PO, et al. A novel volume-age-KPS (VAK) glioblastoma classification identifies a prognostic cognate microRNA-gene signature. PloS one. 2012;7:e41522. doi: 10.1371/journal.pone.0041522. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 39.Tofts PS. Modeling tracer kinetics in dynamic Gd-DTPA MR imaging. J. Magn. Reson Imaging. 1997;7:91–101. doi: 10.1002/jmri.1880070113. [DOI] [PubMed] [Google Scholar]
  • 40.Tofts PS, et al. Estimating kinetic parameters from dynamic contrast-enhanced T(1)-weighted MRI of a diffusable tracer: standardized quantities and symbols. J. Magn. Reson Imaging. 1999;10:223–232. doi: 10.1002/(SICI)1522-2586(199909)10:3&#x0003c;223::AID-JMRI2&#x0003e;3.0.CO;2-S. [DOI] [PubMed] [Google Scholar]
  • 41.Zhang J, et al. Clinical applications of contrast-enhanced perfusion MRI techniques in gliomas: recent advances and current challenges. Contrast Media Mol. Imaging. 2017;2017:7064120. doi: 10.1155/2017/7064120. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 42.Law M, et al. Glioma grading: sensitivity, specificity, and predictive values of perfusion MR imaging and proton MR spectroscopic imaging compared with conventional MR imaging. Am. J. Neuroradiol. 2003;24:1989–1998. [PMC free article] [PubMed] [Google Scholar]
  • 43.Wetzel SG, et al. Relative cerebral blood volume measurements in intracranial mass lesions: interobserver and intraobserver reproducibility study. Radiology. 2002;224:797–803. doi: 10.1148/radiol.2243011014. [DOI] [PubMed] [Google Scholar]
  • 44.Papoulis Athanasios, S. U. P. Probability, random variables, and stochastic processes. McGraw-Hill 4 (1991/2).
  • 45.Clausi DA. An analysis of co-occurrence texture statistics as a function of grey level quantization. Can. J. Remote Sens. 2002;28:45–62. doi: 10.5589/m02-004. [DOI] [Google Scholar]
  • 46.Haralick RM, Shanmugam K, Dinstein I. Textural features for image classification. IEEE Trans. Syst., Man, Cybern. SMC. 1973;3:610–621. doi: 10.1109/TSMC.1973.4309314. [DOI] [Google Scholar]
  • 47.Soh LK, Tsatsoulis C. Texture analysis of SAR sea ice imagery using gray level co-occurrence matrices. IEEE Trans. Geosci. Remote Sens. 1999;37:780–795. doi: 10.1109/36.752194. [DOI] [Google Scholar]
  • 48.Zinn PO, et al. 139 Clinically applicable and biologically validated mri radiomic test method predicts glioblastoma genomic landscape and survival. Neurosurgery. 2016;63:156–157. doi: 10.1227/01.neu.0000489709.98960.e1. [DOI] [Google Scholar]
  • 49.Colen Rivka R., Fujii Takeo, Bilen Mehmet Asim, Kotrotsou Aikaterini, Abrol Srishti, Hess Kenneth R., Hajjar Joud, Suarez-Almazor Maria E., Alshawa Anas, Hong David S., Giniebra-Camejo Dunia, Stephen Bettzy, Subbiah Vivek, Sheshadri Ajay, Mendoza Tito, Fu Siqing, Sharma Padmanee, Meric-Bernstam Funda, Naing Aung. Radiomics to predict immunotherapy-induced pneumonitis: proof of concept. Investigational New Drugs. 2017;36(4):601–607. doi: 10.1007/s10637-017-0524-2. [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 50.Hanchuan P, Fuhui L, Ding C. Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 2005;27:1226–1238. doi: 10.1109/TPAMI.2005.159. [DOI] [PubMed] [Google Scholar]
  • 51.Frank E, Wang Y, Inglis S, Holmes G, Witten IH. Using model trees for classification. Mach. Learn. 1998;32:63–76. doi: 10.1023/A:1007421302149. [DOI] [Google Scholar]
  • 52.Vojt et al. in Proceedings of the 28th International Conference on International Conference on Machine Learning 665-672 (Omnipress, Bellevue, Washington, USA, 2011).
  • 53.Platt J. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv. large margin Classif. 1999;10:61–74. [Google Scholar]
  • 54.Refaeilzadeh, P., Tang, L. & Liu, H. in Encyclopedia of Database Systems (eds Ling Liu & M. Tamer ÖZsu) 532–538 (Springer US, 2009).
  • 55.Team, R. C. R: A Language and Environment for Statistical Computing. doi:citeulike-article-id:12956656 (2013).

Associated Data

This section collects any data citations, data availability statements, or supplementary materials included in this article.

Supplementary Materials

Supplementary Information (928.7KB, pdf)

Data Availability Statement

The data is available within the Article or the Supplementary Information. The imaging data that support the findings of this study are available from the corresponding author upon request.

For the MR perfusion analysis, we used Nordic ICE, a commercially available processing platform (https://www.nordicneurolab.com/products/nordicICE.html). For volumetric analysis we used 3D Slicer (version 4.3.1, https://www.slicer.org/), an open-source software program widely used for image visualization and segmentation. The pipeline for the radiomic analysis has been introduced in Zinn et al.20. For the biostatistical analysis, we used R software (version 3.4.0, R Foundation for Statistical Computing, Vienna, Austria); package MRMRe (version 2.0.7) and C5 decision tree (version 0.1.2) and SVM (version 1.7–0.1)50–52.

Articles from Nature Communications are provided here courtesy of Nature Publishing Group

RESOURCES