Automated volumetry of meningiomas in contrast-enhanced T1-Weighted MRI using deep learning

Abstract

Background: Meningiomas are among the most common intracranial tumors. In these tumors, volumetric assessment is not only important for planning therapeutic intervention but also for follow-up examination.However, a highly accurate automated volumetric method for meningiomas using single-modality magnetic resonance imaging (MRI) has not yet been reported. Here, we aimed to develop a deep learning-based automated volumetry method for meningiomas in MRI and investigate its accuracy and potential clinical applications. Methods: For deep learning, we used MRI images of patients with meningioma who were referred to Osaka University Hospital between January 2007 and October 2020. Imaging data of eligible patients were divided into three non-overlapping groups: training, validation, and testing. The model was trained and tested using the leave-oneout cross-validation method. Dice index (DI) and root mean squared percentage error (RMSPE) were measured to evaluate the model accuracy. Result: A total of 178 patients (64.6 ± 12.3 years [standard deviation]; 147 women) were evaluated. Comparison of the deep learning model and manual segmentation revealed a mean DI of 0.923 ± 0.051 for tumor lesions. For total tumor volume, RMSPE was 9.5 ± 1.2%, and Mann–Whitney U test did not show a significant difference between manual and algorithm-based measurement of the tumor volume (p = 0.96). Conclusion: The automatic tumor volumetry algorithm developed in this study provides a potential volume-based imaging biomarker for tumor evaluation in the field of neuroradiological imaging, which will contribute to the optimization and personalization of treatment for central nervous system tumors in the near future

1. Introduction

Meningiomas are among the most common intracranial tumors. These tumors account for approximately 30% of primary intracranial tumors and are often discovered incidentally.^1,2 Radiological and autopsy studies have reported an incidence of approximately 1.2 cases of incidental meningiomas per 100,000 people per year.^3,4 The World Health Organization (WHO) classifies meningiomas as benign (grade I), atypical (grade II), or malignant (grade III), with most of them being diagnosed as Grade I.⁵ The WHO grades of meningiomas diagnosed based on histopathologic findings, are related to imaging features and grade II or III meningiomas grow at a comparatively faster rate.6, 7, 8 Enlarged meningiomas have been reported in 70% of cases, and increased meningioma volume is known to be associated with its symptoms.9, 10, 11, 12 The primary treatment strategy for asymptomatic meningioma is observation, wherein some patients require regular follow-up with imaging studies to help determine the timing of therapeutic intervention.¹³

Manual segmentation by physicians is commonly used to measure tumor volume; however, differences between inter- and intra-observers may make it difficult to achieve a precise follow-up and study.^14,15 To the best of our knowledge, no previous studies have reported an automated volumetric method for meningiomas using single-modality magnetic resonance imaging (MRI) with high accuracy.

Therefore, in this study, we used deep learning techniques with contrast-enhanced T1 (CET1)-weighted MRI to develop and evaluate a supportive algorithm for automated volumetry of meningiomas.

2. Materials and methods

This retrospective study was approved by the institutional review board of our institution. Patient informed consent was waived because this study consisted of a retrospective analysis of existing records.

2.1. Datasets

A total of 497 patients who were referred to Osaka University Hospital for meningioma treatment and underwent MRI between January 2007 and October 2020 were included. MRI Digital Imaging and Communications in Medicine data from different scanners, including data from a referral center, were used. Among the 255 patients with a completely available MRI dataset before treatment, those with axial CET1-weighted MRI sequence were further included. If a patient underwent multiple MRI scans, we only considered the final images. Furthermore, 77 patients were excluded from further assessment for the following reasons (Fig. 1): tumor with a large cyst, previous treatment, image size other than 256 × 256 or 512 × 512 pixels, recurrent or multiple meningiomas, and severe MRI artifacts. Patients with inappropriate volumetry on axial imaging, such as remarkably small meningiomas located at high convexity, were excluded.

Fig. 1.

Flowchart depicting the patient selection process.

All scans were performed for clinical indications. MRI acquisitions were performed using diverse scanners from our institution and the referring institutions. The examinations were mainly performed using MRI units (Achieva, Philips Medical Systems, Best, the Netherlands and SIGNA, GE Healthcare, Milwaukee, WI, USA). Each patient underwent imaging with a CET1-weighted MRI sequence after the administration of gadolinium.

2.2. Algorithm development methods

All meningioma segmentations were conducted manually by two experienced neurosurgeons (R.H. and S.Y.) using Horos (www.horosproject.org; Horos Project) for macOS and Onis 2.5 Professional (www.onis-viewer.com; DigitalCore). The segmentation data were converted into a binarized image, which was used as the ground truth. The input MRI data of the deep learning network included only slices that contained the tumor. Images of size 256 × 256 pixels were converted to 512 × 512 pixels.

Convolutional neural networks are often used for deep-learning-based segmentation. We performed transfer learning by fine-tuning a DeepLabV3 model that was pretrained by ResNet-101 to fully automate the process of meningioma segmentation. The networks were created in PyTorch using a Windows desktop with a 32-GB RAM and NVIDIA RTX 2080. For training, we used the Adam optimizer with a batch size of four. The learning rate was initialized to exp (−2). Because tumors are 3D objects, three neighboring 2D slices were stacked as the inputs after the input images were converted to grayscale. Thus, the 2D network could detect a small range of 3D contexts each time. Image augmentation was employed to improve the generalizability of the network. Training was terminated if the validation loss did not improve after 20 epochs. The eligible data were divided into training (90%) and test (10%) datasets. The model was trained and tested using leave-one-out 10-fold cross-validation (Supplementary Fig. 1).

2.3. Statistical analysis

For quantitative analysis, the Dice index (DI) and root mean squared percentage error (RMSPE) of the tumor volume were measured. We defined pixels with tumors as true and those without tumors as false.

The DI is a proven statistical validation metric used to evaluate the performance and spatial overlap between two sets of segmentations of the same anatomy. The DI is represented as a percentage using the below formula, with 100% being a perfect voxel-wise match between results.

$$DI=\frac{2\times TP}{\left(TP+FP\right)+\left(TP+FN\right)}$$

DI, Dice index; TP, true positive; FP, false positive; FN, false negative.

RMSPE is defined by the below formula and is the error between the manually measured volume of each tumor and the volume calculated using artificial intelligence. Continuous variables are expressed as means and standard deviations. The Mann–Whitney U test was used to compare the two groups. JMP Pro 16.0.0 (SAS Institute, Cary, NC, USA) was used for statistical analyses.

$$RMSPE=\sqrt{\frac{1}{n}\sum _{i=1}{\left(\frac{Vi-\overline{Vi}}{Vi}\right)}^2}$$

RMSPE, root mean squared percentage error; Vi, manually measured volume; $\overline{\text{Vi}}$, automatically measured volume

3. Results

The training dataset included 178 patients (147 females and 31 males) with meningiomas. The characteristics of all patients are listed in Table 1. The mean age of the patients was 64.6 ± 12.3 years. Tumor locations included the convexity (n = 66), parasagittal (n = 34), falx (n = 26), cerebellopontine angle (n = 16), and other locations (n = 36). Peritumoral brain edema was observed in 65 patients.

Table 1.

Patient and tumor characteristics.

		CASES (n = 178)
SEX	Female	147 (82.6%)
	Male	31 (17.4%)
AGE (YEARS)	Mean ± SD	64.6 ± 12.3
	Range	23–89
LOCATION	Convexity	66 (37.1%)
	Parasagittal	34 (19.1%)
	Falx	26 (14.6%)
	CP-angle	16 (9.0%)
	Other	36 (20.2%)
TUMOR VOLUME (CM3)	Mean ± SD	20.5 ± 26.0
	Range	0.9–138.8
PERITUMORAL BRAIN EDEMA	Yes	65 (36.5%)
	No	113 (63.5%)

3.1. Segmentation and volumetry performance of the network

Among all patients, manual segmentation volume of tumors averaged 20.5 ± 26.0 cm³ (range 0.9–138.8 cm³). The accuracy and consistency of the developed algorithm were further evaluated using the DI to assess the 2D accuracy of the individual images. The mean DI for the tumor region was 0.923 ± 0.051. The actual tumor volume was significantly correlated with the predicted volume (R: 0.9877, p < 0.001; Pearson correlation analysis). The tumor volume showed a weak correlation with the DI (R: 0.21283, p: 0.0043411; Pearson correlation analysis) (Fig. 2). Furthermore, we compared the DI in two groups of tumors: those below and above the median volume of all tumors in this study. This analysis revealed a significantly lower DI for tumors with a volume below the median compared with those above the median (DI = 0.91 ± 0.051 and 0.94 ± 0.049; p < 0.0001; Wilcoxon rank sum test).

Fig. 2.

(A)Plot of actual vs. predicted tumor volume. Tumor volume was significantly correlated with the predicted volume (R: 0.9877, p: 9.9084e-144; Pearson correlation analysis). (B) Plot of tumor volume vs. Dice index (DI). Tumor volume showed a weak correlation with the DI (R: 0.21283, p: 0.0043411; Pearson correlation analysis).

Representative cases are shown in Fig. 3. Cases with a clear contrast effect, but with the tumor in contact with the skull base (Fig. 3A) and cases with a large tumor and no large surrounding vessels to be contrasted (Fig. 3B), but with irregularly shaped tumor, were automatically segmented with a high DI. In contrast, cases with small tumors and poor contrast effects (Fig. 3C) were poorly segmented with a low DI.

Fig. 3.

Representative cases are shown. The left column shows a contrast-enhanced T1 image with the outline of a detected tumor. The right column image is a magnified image of the tumor and its surrounding area. The blue contour shows manual segmentation, while the red contour represents automated segmentation using the algorithm.A (case no. 30) is a case of left frontal base meningioma with a Dice index (DI) of 0.975, which is representative of highly accurate segmentation of the region bordering the superior sagittal sinus and falx cerebri.B (case no. 212) is a case of right convexity meningioma. Although the tumor was irregularly shaped, it was segmented with high accuracy (DI: 0.973).C (case no. 109) is a case of right parasagittal meningioma with a DI of 0.688 and poor segmentation. The tumor was small, and the contrast effect with the surrounding parenchyma was poor.

3.2. Comparison of automated and manual volumetry

The volume RMSPE was measured to assess the 3D and clinical utility of individual cases. The mean RMSPE for tumor volume was 9.5 ± 1.2%, and the Mann–Whitney U test did not show a significant difference between the tumor volume measured manually and that measured with our algorithm (p = 0.96).

4. Discussion

In this study, we developed a deep learning-based method for the automated volumetry of meningiomas. The DI and RMSPE showed that the automatically segmented tumor regions correlated well with manual segmentation, which was highly accurate compared to other previous studies based on brain lesion segmentation using deep learning.^16,17

Artificial intelligence has been demonstrated to be reaching a point where it can be used reliably in clinical applications.15, 16, 17 Glioma is a clinically important brain tumor due to its nature and has been a research target for automated tumor segmentation.^16,17 Meningioma segmentation has also been a research focus, with several automated approaches being reported since tracking of tumor volume plays an important role in determining therapeutic interventions for meningiomas.^18,19 Recent studies have reported the accuracy of meningioma tumor volume measurement using deep learning methods with multiple MRI sequences, including T1-weighted image (T1WI) and T2WI as well as CET1 and fluid-attenuated inversion recovery (FLAIR) images.^20,21 The segmentation accuracy of CE tumor regions has reached a DI of up to 0.91 ± 0.08.²¹ In order to obtain a high accuracy, only CET1-weighted images were used in our study. The reasons for the high accuracy of this algorithm are that a large number of examples were used, transfer learning was well adapted, and noise was added during augmentation to minimize the overfitting of the algorithm and generalizability.

Automatic detection is highly reproducible and objective as it avoids observer differences in the definition of tumor volume.^14,15 Moreover, conventional diameter measurements tend to underestimate tumor growth, whereas volume measurements based on deep learning can reliably detect tumor growth by volume, thereby supporting decision-making for treatment strategies and providing a precise understanding of the disease state.¹⁴ Thus, the reliable automatic volume measurement shown in this study can replace the time-consuming manual segmentation and deal with large amounts of data.

The deep learning method developed in this study is also clinically useful because it can be implemented using a single MRI modality. Furthermore, MR images from different scanners, including those from a referral center, were used to prove that the method could provide accurate volume measurements. Because meningiomas have less structural complexity than gliomas and other types of tumors, the segmentation accuracy should be higher. However, segmentation of meningiomas may be rendered difficult due to the following tumor features¹: significant edema around the tumor,² infiltration into the skull base or bone invasion,³ wide dural tail sign,⁴ large cysts,⁵ necrosis, and⁶ CE vascular structures adjacent to the tumor. Therefore, more data are needed on the accurate segmentation of tumors with these characteristics.

The current study has several limitations. First, this was a single-center, retrospective study. In the future, its validity will be necessary to be evaluated under the imaging conditions at other facilities. Second, the accuracy of detecting the presence of tumors was not evaluated because the training was performed only on slices where the tumor was present. In addition, to increase specificity, training on a mixture of datasets with and without meningiomas is required. However, in clinical follow-up situations, assessing the increase in meningioma volume may be more important than detecting the presence of meningioma. Third, CET1-weighted images were used in this study to develop a method that minimizes the variability inherent in manual volume measurements. In clinical practice, excellent agreement between meningioma size and growth is obtained from T1 3D-gadolinium and 2D-T2WI, suggesting that the use of non-contrast images may be sufficient for the follow-up of untreated meningiomas.²² However, our dataset did not include non-contrast T1 and T2 annotation data, limiting our ability to extend our findings to these imaging modalities. Therefore, we are planning a study to create a similar algorithm using T2WI or FLAIR images.

5. Conclusions

Our study results suggest that automatic volume calculation using deep learning can be easily used for follow-up and treatment planning of meningiomas with high reproducibility and accuracy. Additionally, the automatic tumor volumetry algorithm developed in this study provides a potential volume-based imaging biomarker for tumor evaluation in the field of neuroradiological imaging, which will contribute to the optimization and personalization of treatment for central nervous system tumors in the near future.

Funding

This investigation was supported by the Japan Society for the Promotion of Science (JSPS) KAKENHI (grant number 22K07279).

CRediT authorship contribution statement

Takamitsu Iwata: Writing – original draft. Ryuichi Hirayama: Writing – review & editing. Shuhei Yamada: Data curation, Funding acquisition. Noriyuki Kijima: Data curation. Yoshiko Okita: Data curation. Naoki Kagawa: Data curation. Haruhiko Kishima: Supervision.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Abbreviations

CET1

Contrast-enhanced T1

DI

Dice index

FLAIR

fluid-attenuated inversion recovery

MRI

Magnetic resonance imaging

RMSPE

Root mean squared percentage error

T1WI

T1-weighted image

WHO

World Health Organization

Appendix A. Supplementary data

The following is/are the supplementary data to this article.

Supplementary fig 1

A diagram illustrating the process of 10-fold cross-validation was utilized in this study. The dataset was divided into ten subsets, with nine of these subsets being used for training, and one subset for test. The mean Dice index was calculated based on the ten partitioned test results, and this value was utilized as the accuracy of the model, serving as a performance indicator for the cross-validation framework.