Reliability of tumor segmentation in glioblastoma: Impact on the robustness of MRI‐radiomic features

Abstract

Purpose

The use of radiomic features as biomarkers of treatment response and outcome or as correlates to genomic variations requires that the computed features are robust and reproducible. Segmentation, a crucial step in radiomic analysis, is a major source of variability in the computed radiomic features. Therefore, we studied the impact of tumor segmentation variability on the robustness of MRI radiomic features.

Method

Fluid‐attenuated inversion recovery (FLAIR) and contrast‐enhanced T1‐weighted (T1WI_CE) MRI of 90 patients diagnosed with glioblastoma were segmented using a semiautomatic algorithm and an interactive segmentation with two different raters. We analyzed the robustness of 108 radiomic features from five categories (intensity histogram, gray‐level co‐occurrence matrix, gray‐level size‐zone matrix (GLSZM), edge maps, and shape) using intra‐class correlation coefficient (ICC) and Bland and Altman analysis.

Results

Our results show that both segmentation methods are reliable with ICC ≥ 0.96 and standard deviation (SD) of mean differences between the two raters (SD_diffs) ≤ 30%. Features computed from the histogram and co‐occurrence matrices were found to be the most robust (ICC ≥ 0.8 and SD_diffs ≤ 30% for most features in these groups). Features from GLSZM were shown to have mixed robustness. Edge, shape, and GLSZM features were the most impacted by the choice of segmentation method with the interactive method resulting in more robust features than the semiautomatic method. Finally, features computed from T1WI_CE and FLAIR images were found to have similar robustness when computed with the interactive segmentation method.

Conclusion

Semiautomatic and interactive segmentation methods using two raters are both reliable. The interactive method produced more robust features than the semiautomatic method. We also found that the robustness of radiomic features varied by categories. Therefore, this study could help motivate segmentation methods and feature selection in MRI radiomic studies.

1. Introduction

Radiomic analysis consists of the high‐throughput mining of quantitative features from standard‐of‐care medical imaging¹ and is becoming a powerful tool for treatment planning, outcome prediction, and personalized therapy.^{2, 3, 4} Radiomic models are often built using features obtained from baseline or follow‐up computed tomography (CT) or positron emission tomography (PET) images.^{1, 4, 5, 6} Recently, there has been an increased interest in performing radiomic studies using MR images.^{7, 8, 9, 10, 11, 12, 13} Using radiomics with MRI is favorable for some tumor types, such as brain tumors, for which MRI is the standard‐of‐care imaging modality. Also, MRI provides unique characteristics that differ from PET or CT features, providing additional information for prognostication and outcome prediction. In brain, MR images such as contrast‐enhanced T1‐weighted images (T1WI_CE) and fluid‐attenuated inversion recovery (FLAIR) images can be used to identify different structures such as edema, tumor infiltration, and necrosis. Radiomic models constructed from these images have been shown to predict overall survival,^{11, 14} identify distinct molecular subtypes of glioblastoma (GBM),¹⁵ and identify patients likely to most benefit from antiangiogenic therapy,¹⁶ or used as a guide for intensification of postoperative radiation therapy.¹⁷ MRI diffusion tensor imaging (DTI) provides noninvasive and reproducible biomarkers that can help with the classification of low‐ and high‐grade glioma, metastasis, and meningioma, and consequently assist with decision‐making for the choice of treatment.^{18, 19, 20} Multiparametric MRI has also shown an ability to determine the grade, spatial extent, and heterogeneity of brain tumors and consequently help to improve the management of patients with GBM.²¹

Radiomic analyses that involve the extraction of features may be impacted by variability from two main categories: (a) controllable variability that stems directly from the computational methods used, such as histogram normalization, the quantization level, and three‐dimensional (3D) versus two‐dimensional (2D) analysis; and (b) uncontrollable (or less controllable) variability that originates from image acquisition, reconstruction, and tumor segmentation processes. The responsible use of any radiomic model should: (a) describe how the radiomic features were extracted, including all the preprocessing steps performed and (b) acknowledge the variability of the selected features due to uncontrollable sources.

Although the robustness of radiomic features with respect to acquisition, reconstruction, or segmentation methods and test–retest reproducibility have been well described in PET and CT, these results cannot be directly extrapolated or transferred to MR images.²² To the best of our knowledge, there is only one study that has investigated the interobserver agreement of one radiomic feature (entropy from the co‐occurrence matrix) in diffusion weighted MRI.²³

In this study, we investigated the reliability of the segmentation of glioblastoma tumors using two segmentation methods (semiautomatic and interactive with manual corrections) obtained with two raters (medical physicists with knowledge in radiology) and analyzed the robustness of classically used MRI radiomic features based on segmentation reliability. Consequently, this study should help to identify the set of radiomic features that are generally robust and guide the selection of appropriate segmentation techniques for radiomic analyses.

2. Materials and methods

2.A. Patients

From the 262 patients of The Cancer Genome Atlas Glioblastoma Multiforme (TCGA‐GBM) collection (n = 262) of The Cancer Imaging Archive (TCIA),²⁴ 136 were excluded due to motion artifacts, huge surgical cavities, or absence of FLAIR/T1WI_CE images. Ninety patients were randomly selected prior to analysis from the remaining 126 patients. Inclusion criteria consisted of patients with both T1WI_CE and FLAIR images, and without motion artifacts, or surgical cavities. We analyzed both T1WI_CE and FLAIR images as the former is commonly used to describe enhancing tumor extent, while the latter has been shown to describe nonenhancing tumor extent despite confounding brain edema.^{25, 26, 27} Image characteristics, including image resolution, are summarized in Table 1.

Table 1.

Image characteristics.

Image characteristics	n = 90 (%)
Resolution (x,y) mm
[0.4–0.7]	29 (32%)
[0.7–1]	61 (68%)
Slice thickness
3 mm	18 (20%)
5 mm	64 (71%)
other	8 (9%)

2.B. Image segmentation

The first method implemented a semiautomatic segmentation on the FLAIR images based on the work of Veeraraghavan and Miller 2011.²⁸ Starting from user‐drawn brush strokes on 2–3 slices to identify the tumor and background regions, the algorithm learns a model using a support vector machine (SVM) classifier. This method adds to the competitive region‐growing technique known as the “GrowCut” method that is popular in 3D Slicer²⁹—the main difference being that this method uses a SVM classifier trained on the user’s brush stroke inputs to further refine the GrowCut segmentations.

The second method implemented an interactive segmentation with manual corrections (I_MC). This method adds to the competitive region‐growing GrowCut with SVM refinement. To further improve the segmentation, this method incorporates an interactive mode where regions that are difficult to segment are automatically identified by the algorithm and presented as queries to the user to indicate whether those regions correspond to tumor or background. Following user inputs, the SVM is retrained until there are no more regions for querying or the user terminates the algorithm. Postprocessing using standard morphological open–close (by 3 pixels to capture the center and the immediately adjacent pixels) and hole filling are performed to derive smooth segmentation contours in 3D. This method includes algorithm prompting and subsequent user corrections that are lacking in the first method. After the algorithm terminates, the user checks and manually edits the segmentation. Examples of results given by these two segmentation methods are presented in Fig. 1.

Figure 1.

Tumor segmentation for two different patients. (a) User‐drawn brush strokes used to initiate the two segmentation methods; (b) Semi‐automatic segmentation; (c) Interactive segmentation with manual corrections (I_MC). The two left columns and two right columns are the results obtained with rater 1 and rater 2, respectively. [Color figure can be viewed at wileyonlinelibrary.com]

2.C. Radiomic feature extraction

The images analyzed in this study presented large in‐plane (0.4–1 mm) and between‐plane (3–5 mm) differences in resolution. For this reason, following the segmentation and prior to radiomic feature extraction, images were resampled with linear interpolation to obtain isotropic voxels of 1 mm³ allowing for the extraction of radiomic features in 3D. In addition, the MR intensities in all images were binned to Q = 128 values using:

$$V_Q\left(x\right)=round\left(\frac{\left(Q-1\right).\left(x-V_{min}\right)}{V_{max}-V_{min}}+1\right)$$ (1)

with V_min and V_max as the minimum and maximum intensity values in the segmented tumor volumes. These binned images were used to compute texture features [Grey Level Co‐occurrence Matrix (GLCM) and Grey Level Size Zone Matrix (GLSZM)], while histogram and shape features were computed from the original images. No additional image preprocessing was performed.

Additionally, Gabor (with four orientations: θ = 0°, 30°, 45°, 90°, and two frequencies: f = 2, 2√2),³⁰ Sobel³¹ and Laplacian of Gaussian (LoG)³² filtering were applied to extract edge maps from the images.

A total of 108 features were extracted from the FLAIR and T1WI_CE images: (a) 20 features from the intensity histogram; (b) 26 from the GLCM using 13 directional offsets (3D) and a distance of 1 between the voxels. One co‐occurrence matrix was produced by combining the contributions from all offsets into one and used to calculate the aforementioned features; (c) 15 from the GLSZM using one matrix describing zones in the 3D tumor volume; (d) seven shape features; and (e) four intensity histogram features (mean, standard deviation, skewness, and kurtosis) from ten different edge maps obtained with the filtered FLAIR and T1WI_CE images.

All investigated features were obtained using the CERR software,³³ which is compliant with the Image Biomarkers Standardization Initiative (IBSI).³⁴ The choice of the investigated features was made using the most common radiomic feature categories and based on features that we used in a previous GBM study.¹¹

2.D. Statistical analysis

The interobserver reliability of the two segmentation methods (semiautomatic and I_MC) was measured using Bland and Altman analysis³⁵ and the interobserver agreement was quantified by calculating the intraclass correlation coefficient (ICC) using a two‐way random effects model and the mean differences of the two raters.³⁶ The reliability limits were defined as the mean differences ± 1.96 standard deviation (SD) of the differences. The 95% confidence intervals (CIs) were reported for these limits, the mean differences, and the ICCs. The Shapiro–Wilk test³⁷ was used to evaluate the normality of the distribution of differences for the two raters. The effect of voxel size on reliability measurements was investigated using a Mann–Whitney U test.³⁸ For this test, the patient cohort was split into two groups based on low or high resolution [x,y voxel resolution (0.4–0.7) or (0.7–1) mm², respectively—see Table 1].

The same statistical tests were used to analyze the robustness of the MRI radiomic features due to the reliability of the segmentation method used. All these tests were performed with the Rstudio software (v1.1.453) and P‐values < 0.05 were considered statistically significant. Bland and Altman results were reported with the absolute difference value and with the mean percentage difference according to the following formula:

$$\left({\phi }_{r1}-{\phi }_{r2}\right)/\frac{{\phi }_{r1}+{\phi }_{r2}}{2}\ast 100$$ (2)

where ϕ _r1 and ϕ _r2 are the values of a feature obtained with the segmentation from raters 1 and 2, respectively. The robustness of the MRI radiomic features was evaluated using a six‐level scale (going from 0 to 5) and based on a combination of ICCs and SD of the mean percentage differences between the two raters, named SD_diffs in the manuscript. On this scale, we set level 0 to correspond to features with the highest (or very high) robustness with an SD_diffs ≤ 10% and an ICC ≥ 90% and level 5 to correspond to features with the lowest (or very low) robustness with an SD_diffs > 100% and an ICC < 70%. This scale is described in the legend of Fig. 2. Our six‐level scale consists of very low (level 5), low (4), limited (3), medium (2), high (1), and very high (0) robustness.

Figure 2.

Robustness of the radiomic features obtained on FLAIR and T1WI_CE images using semiautomatic segmentation with raters 1 and 2 (columns ‘SA’) and using interactive segmentation with manual corrections with raters 1 and 2 (columns ‘I_MC’). FLAIR, Fluid‐attenuated inversion recovery [Color figure can be viewed at wileyonlinelibrary.com]

From this scale, we computed a category robustness ϒ_Ω to summarize the robustness by feature category across the segmentation methods and the image sequences (FLAIR, TIWI_CE):

$${\mathrm{{\rm Y}}}_{\mathrm{\Omega }}=1-\frac{{\sum }_{i\in \mathrm{\Omega }}{S}_{{\phi }_i}}{5\ast Card\left(\mathrm{\Omega }\right)}$$ (3)

where $S_{{\phi }_i}$ is the level value on the scale described above for the robustness of feature i and Ω is the feature category (or all the features together). This formula gives values between 0 and 1, with a higher value corresponding to a higher robustness.

3. Results

3.A. Segmentation reliability

Table 2 contains a summary of the reliability results for the tumor volume obtained by the two raters for the two segmentation methods. The highest differences were found for the semiautomatic method with a mean difference of 8.2 ± 30.4% and an ICC of 0.96. The I_MC method gave the best reliability with a mean difference of 1.5 ± 21.5% and an ICC of 0.99.

Table 2.

Reliability results of volume segmentation using the semiautomatic and interactive with manual corrections (I_MC) methods between the two raters.

Method	Differencesa	LRL [95% CI]	Mean ± SD [95% CI]	URL [95% CI]	ICC [95% CI]
Semiautomatic	cm3	−40.4 [−48.7 to −32.1]	4.5 ± 22.9 [−0.3 to 9.3]	49.4 [41.1 to 57.7]	0.96 [0.94, 0.98]
	percentage	−51.5 [−62.5 to −40.4]	8.2 ± 30.4 [1.8 to 14.6]	67.8 [56.8 to 78.9]
IMC	cm3	−27.0 [−32.1 to −22.0]	0.3 ± 14.0 [−2.6 to 3.3]	27.7 [22.6 to 32.8]	0.99 [0.98, 0.99]
	percentage	−40.6 [−48.4 to −32.8]	1.5 ± 21.5 [−3.0 to 6.0]	43.7 [35.9 to 51.5]

ICC, intraclass correlation coefficient; IMC, interactive with manual corrections; LRL, lower reliability limit; URL, upper reliability limit.

^aDifferences have a non‐normal distribution according to the Shapiro–Wilk test.

The tumor volumes generated by both semiautomatic and I_MC methods using the two rater inputs were highly similar for patients with voxel resolutions (x,y) < or ≥ 0.7 mm² the Mann–Whitney U test gave P‐values 0.28 and 0.24 for the semiautomatic and I_MC methods, respectively (see Fig. 3).

Figure 3.

Bland and Altman plots for (a)—semiautomatic, (b)—interactive with manual corrections segmentation (I_MC). Results report the mean difference between the two raters in cm³ with 95% CI and the lower and upper reliability limits with 95% CI. CI, confidence interval. [Color figure can be viewed at wileyonlinelibrary.com]

3.B. Robustness of the MRI radiomic features against the segmentation method

Table 3 summarizes the results of the robustness of MRI radiomic feature categories for both segmentation methods. Figure 2 shows the results of the robustness of individual MRI radiomic features for both segmentation methods.

Table 3.

Robustness of each feature category across segmentation techniques (SA = semiautomatic; I_MC = interactive with manual corrections) and image sequences (FLAIR and T1WI_CE) according to Eq. (3).

	FLAIR		T1WICE
Feature category	SA	IMC	SA	IMC
Histogram	0.80	0.83	0.72	0.81
Shape	0.74	0.89	0.74	0.89
GLCM	0.90	0.92	0.81	0.88
GLSZM	0.72	0.76	0.61	0.77
Edge features	0.67	0.76	0.71	0.81
All	0.76	0.82	0.72	0.83

FLAIR, Fluid‐attenuated inversion recovery; GLCM, Grey level co‐occurrence matrix; GLSZM, Grey level size zone matrix.

3.B.1. FLAIR images

The histogram feature category extracted from the volume of interest (VOI) had a robustness of ϒ _histogram = 0.80 and ϒ _histogram  = 0.83 for the semiautomatic and I_MC methods, respectively. Individual histogram features had high to very high robustness except for skewness and kurtosis (very low robustness for both methods), minimum (low robustness for both methods), and energy (medium robustness for the semiautomatic method and high robustness for the I_MC method).

The GLCM feature category had a robustness of ϒ_GLCM = 0.90 and ϒ_GLCM = 0.92 for the semiautomatic and I_MC methods, respectively. Individual GLCM features had high to very high robustness for both methods with the exception of cluster shade (very low robustness for both methods) and Haralick correlation (medium robustness for the semiautomatic method and high robustness for the I_MC method).For most of the individual GLCM features, robustness was higher for the I_MC method compared with the semiautomatic method.

The edge feature category had a robustness of ϒ_edge = 0.67 and ϒ_edge = 0.76 for the semiautomatic and I_MC segmentation methods, respectively. Mean and standard deviation features from the edge maps showed high to very high robustness irrespective of the segmentation method. Skewness computed from the edge maps showed very low to low robustness except for those computed from Sobel and Laplacian of Gaussian (LoG) edge maps. Kurtosis showed medium robustness for the semiautomatic method and high robustness for I_MC method.

The GLSZM feature category had a robustness of and ϒ_GLSZM = 0.72 and ϒ_GLSZM = 0.76 for the semiautomatic and I_MC segmentation methods, respectively. Individual GLSZM features exhibited mixed robustness ranging from low to very high robustness. The most robust features for this category were: small zone emphasis, gray‐level nonuniformity normalized, size zone nonuniformity normalized, zone percentage, small zone high gray‐level emphasis and gray‐level variance. These features were generally less robust than features of a different feature category regardless of the segmentation method.

In addition to the summarized results from Table 3 and Fig. 2, the full results including the lower and upper reliability limits together with the 95% CI of these limits, mean differences and ICCs, and absolute value of the differences are reported in Tables S1 and S2.

3.B.2. T1WI_CEimages

The histogram feature category had a robustness of ϒ _histogram = 0.72 and ϒ _histogram  = 0.81 for the semiautomatic and I_MC methods, respectively. Irrespective to the segmentation method, skewness and kurtosis histogram features showed very low robustness while minimum showed limited robustness. All the other features showed medium to very high robustness and for many of these features, robustness increased by one level between the semiautomatic and I_MC segmentation methods.

The GLCM feature category for T1WI_CE images had a robustness of ϒ_GLSZM = 0.81 and ϒ_GLSZM = 0.88 for the semiautomatic and I_MC methods, respectively. Individual GLCM features also showed similar robustness to that of GLCM features computed from FLAIR images, where all features except cluster shade, cluster prominence, Haralick correlation, and energy showed high to very high robustness.

The edge feature category had a robustness of ϒ_edge = 0.71 and ϒ_edge = 0.81 for the semiautomatic and I_MC methods, respectively. All individual edge features had high to very high robustness with the exception of skewness computed from the Gabor edge maps (very low to low robustness).

The GLSZM feature category had a robustness of ϒ_GLSZM = 0.61 and ϒ_GLSZM = 0.77 for the semiautomatic and I_MC methods, respectively. Grey Level Size Zone Matrix features exhibited mixed robustness ranging from low to very high robustness. The most robust features for this category were small zone emphasis, gray‐level nonuniformity normalized, and size‐zone nonuniformity normalized.

In addition to the summarized results from Table 3 and Fig. 2, the full results including the lower and upper reliability limits together with the 95% CI of these limits, mean differences, and ICCs, and absolute value of the differences are reported in Tables S3 and S4.

3.B.3. Shape features

The shape feature category had a robustness of ϒ_shape = 0.74 and ϒ_shape = 0.89 for the semiautomatic and I_MC methods, respectively. The highest robustness was found for spherical disproportion and sphericity with I_MC segmentation (Fig. 2, Table 3). For additional details, the full results are reported in Tables S5 and S6.

3.B.4. Overall results

Overall, features had a category robustness ϒ _overall ranging between 0.72 to 0.76 for the semiautomatic method and 0.82 to 0.83 for the I_MC method. On FLAIR images, category robustness was ϒ _overall  = 0.76 and ϒ _overall  = 0.82 for the semiautomatic and I_MC segmentation methods, respectively. When using T1WI_CE, the category robustness for the features improved from ϒ _overall  = 0.72 for the semiautomatic to ϒ _overall  = 0.83 for the I_MC segmentation method.

4. Discussion

In this study, we investigated the inter‐rater robustness of MRI‐radiomic features. We also studied the reliability of segmented tumor volumes when using two raters and obtained using semiautomatic and I_MC segmentation to achieve a close approximation of the tumor extent. Prior studies have shown that semiautomatic methods lead to more reproducible segmentations and radiomic features than those computed using manual delineations.^{39, 40, 41} Consequently, we looked at the utility of adding machine learning to a semiautomatic, competitive region‐growing or GrowCut segmentation with SVM learning in two different modes. In one mode, called the semiautomatic method, the algorithm simply uses the user inputs as brush strokes and derives a segmentation. In the second mode, called the interactive mode with manual corrections (I_MC), the algorithm queries back the user based on the uncertainty derived from the machine learning method to appropriately adjust its segmentation. In other words, the algorithm queries are directed only toward areas that are hard to segment by the algorithm with the goal of reducing time‐consuming slice‐by‐slice user verification. Recent works on segmentation of brain tumors allow automatic detection of tumors and achieve closer‐to‐expert manual segmentations than those obtained with the semiautomatic method used here.^{42, 43, 44, 45} Clearly, more advanced and fully automated deep learning methods^{43, 46, 47} could lead to more accurate tumor segmentations and speed up the workflow for automated radiomic analysis. Results from the BRATS challenge can be used for selecting the best deep learning methods.⁴⁸ However, one major drawback of these methods is their interpretability. Furthermore, deep learning methods would still require manual correction even if it were so for fewer cases and slices than the interactive method. This is simply because any machine learning method is optimized for its training set that is based on one or utmost a few experts. As a consequence, it is generally impossible to get perfectly accurate segmentations on a completely different testing set. For these two reasons, we used the interactive method with manual corrections to obtain a reasonable representation of radiomic feature robustness across raters.

The semiautomatic algorithm has the advantage that once initiated through user‐drawn brush strokes, no additional inputs are necessary, and it demonstrated a high segmentation reliability between the two raters with an ICC of 0.96. However, this method led to less accurate segmentations in some tumors adjacent to the skull due to segmentation leakage into small portions of the skull. This can adversely impact the reliability of the MRI features obtained from these volumes and result in a misleading interpretation of the utility of these features as biomarkers of treatment response or outcome. On the other hand, I_MC segmentation allowed for the correction of these obvious mistakes in the segmentation. This method showed a high reliability with an ICC of up to 0.99.

Although our results showed that the volumes generated between the two raters were not normally distributed given their skewed distributions, the original work by Bland and Altman validates the application of this method to our data.³⁵

The segmentation was generated from FLAIR images and the segmented volumes were then utilized on T1WI_CE. We chose to segment on FLAIR because we found that the inclusion of the FLAIR hyperintense tumoral and peritumoral lesion (usually larger than the contrast enhanced lesion) helps the radiomic features quantify the strength of tumor edges on T1WI_CE.¹¹ Moreover, texture matrices used for the extraction of radiomic features were computed using the 3DVOI as is commonly done with PET/CT images.

In our study, we identified that the I_MC segmentation method led to more robust features compared with the semiautomatic method. We believe that the increased robustness using the I_MC method may result from their reliance on accurate tumor segmentation especially close to the boundaries of the tumor, which are discernable to the human eye and can be corrected.

Similar to the results obtained on the tumor volume, most of the considered features were not normally distributed and the reliability limits and 95% CIs reported should be used with caution. Images were resampled to obtain isotropic voxels of 1 mm³ in order to compute radiomic features in 3D, which reduced the impact of wide differences between in‐plane and between‐plane resolutions. Such resampling techniques have been shown to be useful for outcome studies, especially when using multi‐institutional patient scans that do not share the same resolution.

The 6‐point scale used in this manuscript was made in order to summarize the results on easy‐to‐read table and figures, but thresholds may not be adapted for all situations (annexes contain the full raw results). The feature robustness metrics considered in this study gives an indication of the degree by which features may be impacted by the segmentation method of choice. Nonetheless, the robustness values themselves should not be interpreted as a means for selecting features. Instead, low robustness indicates that the threshold levels used to distinguish outcomes using a particular measure should be stringent on the number of outliers allowed to distinguish classes. For instance, skewness of the Gabor edge maps was found to have a very low robustness in this study. Yet in a previous study, high negative skewness of Gabor edges was identified as a biomarker to differentiate patients with GBM by survival,¹¹ and with a threshold of −0.49 it misclassified only <3% of the patients in this study, using I_MC segmentation. Similarly, high robustness should not be used as a criterion to select features. Some of the features included in this study, such as the ones derived from Sobel and LoG edge maps, were found to be robust to inter‐rater segmentation variations but to our knowledge have never been described as useful predictors of outcome or therapy response in GBM patients. In other words, robustness can be a motivation for feature exclusion but not for feature inclusion. Inclusion must be realized using feature selection techniques such as LASSO⁴⁹ and/or knowledge of the pathology. In GBMs, studies often describe histogram, Gabor edge, GLSZM, and GLCM features from T1WI_CE and FLAIR images as predictors of patient survival,^{11, 50, 51} MGMT methylation status^{52, 53} or chemoradiation response.⁵⁴

This study suffers from some limitations: First, despite isotropic resampling, the lack of 3D MR acquisitions can limit the relevance of the radiomic features computed in 3D. Secondly, results presented here are valid for GBM and even if the images we analyzed show variabilities in terms of sizes and compactness (volume: median = 76 cm³, interquartile range = 83 cm³; sphericity: median = 0.75, interquartile range = 0.16), other cancer models with smaller tumor volumes or lower compactness may lead to less robust features. As a result, the reported values for radiomic features should be similar to other cancer models with similar tumor volume and compactness since the segmentation methods showed comparable reliability.

5. Conclusion

This study reports applied semiautomatic and interactive segmentation with manual corrections (I_MC) methods to compute MRI‐radiomic features. Our results show that both methods produce reliable volumes from inter‐rater inputs and reasonably robust radiomic measures. In particular, the GLCM textures were robust, and a subset of GLSZM features was robust through interactive segmentation.

Conflicts of interest

The authors have no relevant conflicts of interest to disclose.

Supporting information

Table S1: Reliability results of the radiomics features obtained on FLAIR images using semi‐automatic segmentation with raters 1 and 2.

Table S2: Reliability results of the radiomics features obtained on FLAIR images using interactive + manual corrections segmentation (I_MC) with raters 1 and 2.

Table S3: Reliability results of the radiomics features obtained on T1WI_CE images using semi‐automatic segmentation with raters 1 and 2.

Table S4: Reliability results of the radiomics features obtained on T1WI_CE images using interactive + manual corrections segmentation (I_MC) with rater 1 and 2.

Table S5: Reliability results of the shape features obtained using using semi‐automatic segmentation with raters 1 and 2.

Table S6: Reliability results of the shape features obtained using interactive + manual corrections segmentation (I_MC) with rater 1 and 2.

2.E. Data availability

The segmentations generated and analyzed during this study are available from the corresponding author upon reasonable request.