Improved localization and segmentation of spinal bone metastases in MRI with nnUNet radiomics

Highlights

• •The nnUNet model demonstrates effectiveness in automatically locating and segmenting spinal bone metastases in MRI images.
• •The study reports high Dice coefficients for both training and test sets, indicating strong agreement between the nnUNet model segmentation and manual delineation by physicians.
• •The nnUNet model not only effectively segments bone metastases but also demonstrates capability in accurately segmenting small lesions.

Abstract

Objective

Variability exists in the subjective delineation of tumor areas in MRI scans of patients with spinal bone metastases. This research aims to investigate the efficacy of the nnUNet radiomics model for automatic segmentation and identification of spinal bone metastases.

Methods

A cohort of 118 patients diagnosed with spinal bone metastases at our institution between January 2020 and December 2023 was enrolled. They were randomly divided into a training set (n = 78) and a test set (n = 40). The nnUNet radiomics segmentation model was developed, employing manual delineations of tumor areas by physicians as the reference standard. Both methods were utilized to compute tumor area measurements, and the segmentation performance and consistency of the nnUNet model were assessed.

Results

The nnUNet model demonstrated effective localization and segmentation of metastases, including smaller lesions. The Dice coefficients for the training and test sets were 0.926 and 0.824, respectively. Within the test set, the Dice coefficients for lumbar and thoracic vertebrae were 0.838 and 0.785, respectively. Strong linear correlation was observed between the nnUNet model segmentation and physician-delineated tumor areas in 40 patients (R² = 0.998, P < 0.001).

Conclusions

The nnUNet model exhibits efficacy in automatically localizing and segmenting spinal bone metastases in MRI scans.

1. Introduction

Spinal metastatic tumors represent a subset of bone metastases, constituting a prevalent pathology necessitating surgical intervention [1]. The spinal system in the human body houses a significant reservoir of red bone marrow and a dense network of capillaries, facilitating robust blood circulation that promotes the proliferation of tumor emboli [2], [3]. Consequently, the bone, encompassing the spinal system, serves as a common metastatic destination for various malignant tumors, encompassing both carcinomas and sarcomas [4]. Spinal bone metastases represent a predominant proportion, exceeding 70 %, of all cases diagnosed with bone metastases [5]. Between 5 % to 10 % of individuals afflicted with cancer will experience the onset of spinal bone metastases, posing a grave threat to the patient’s overall well-being and survival [6]. Symptoms commonly associated with spinal bone metastases include pain, pathological fractures, hypercalcemia, and nerve compression, with pain being the most prevalent and characteristic manifestation [7]. Early and accurate diagnosis of spinal metastatic tumor disease is paramount in clinical practice for effective treatment strategies [8], [9].

The advancement of medical expertise and the refinement of clinical treatment methodologies have significantly enhanced both the diagnostic and therapeutic efficacy in managing spinal bone metastases [10], [11]. Nevertheless, radiologists currently rely on manual identification methods to detect spinal bone metastases in MRI images, a process contingent upon subjective expertise and professional experience [12]. This reliance can lead to increased workload, physical fatigue, and susceptibility to environmental variations, potentially resulting in disparate interpretations of the same image data. These factors heighten the risk of missed diagnoses and misinterpretations in subsequent treatment decisions [13], [14]. Therefore, there is a critical need for the development of a reliable quantitative tool capable of automatically segmenting spinal bone metastases.

Quantitative noninvasive methodologies, exemplified by radiomics, have garnered significant interest within the medical domain [15], [16]. Nonetheless, the necessity for manual segmentation by physicians persists in predicting medical image targets, thereby constraining the clinical utility of these approaches [14], [17]. Accordingly, this study employs a deep learning model to automate the segmentation of spinal bone metastases, offering a promising avenue for achieving clinically effective segmentation of this pathology.

2. Materials and methods

2.1. General information of subject

For this investigation, a cohort of 118 patients diagnosed with spinal bone metastases was assembled from our hospital, spanning from January 2020 to December 2023. This cohort comprised 73 male and 45 female patients, with ages ranging from 37 to 90 years and a mean age of (61.5 ± 4.9) years.

2.2. Imaging method

Prior to surgical intervention, all patients underwent MRI scans using a 3.0 T MRI scanner (Siemens Magnetom Trio, Erlangen, Germany). The imaging protocol included conventional sagittal plane T1-weighted and T2-weighted sequences, fat inhibition T2-weighted sequences, cross-sectional T1-weighted and T2-weighted sequences, and coronal plane T1-weighted sequences. The localization of spinal bone metastases for each patient was meticulously delineated by a seasoned physician using the medical software ITK-SNAP (version 3.6.0) as the reference standard (Fig. 1).

Fig. 1.

Delineation of the affected vertebra.

2.3. Preprocessing of data image

Augmenting the dataset size can enhance the generalization and segmentation efficacy of deep neural networks. Consequently, this paper incorporates various pre-processing techniques, including rotation and scaling, addition of Gaussian noise, Gaussian blur, brightness adjustment, contrast enhancement, low pixel simulation, gamma correction, and image manipulation, to augment the dataset volume.

2.4. Construction of segmentation model

In this investigation, the model is trained using nnUNet, an adaptive variant of the UNet framework. The nnUNet network architecture for spinal bone metastases segmentation is based on U-Net and consists of an encoder section for image analysis and feature extraction, paired with a corresponding decoder section for generating segmented block graphs. The nnUNet model enhances the traditional U-Net structure by integrating additional losses at each decoder layer, excluding the bottom two layers during decoding. This adaptation facilitates deeper injection of gradient information into the network [18].

For our training dataset, preprocessing is conducted, and the nnUNet network model autonomously generates the requisite hyperparameters for model training based on the preprocessed data and the available video memory size. The patch size is set to 32 × 256 × 224, with a corresponding batch size of 2. The network structure, derived from nnUNet multi-organ image segmentation, is illustrated in Fig. 2. Within the entire network architecture, each encoder and decoder module consists of two Conv blocks, employing the Leaky Rectified Linear Unit (LReLU) activation function, and a convolutional kernel size of 3 × 3 × 3. A total of 320 high-level feature maps are obtained through downsampling, followed by upsampling to restore the original size, culminating in the output of the segment map via softmax activation [19].

Fig. 2.

Network structure diagram based on nnUNet radiomics image segmentation.

2.5. Loss function

The choice of loss function significantly impacts the learning dynamics and convergence of the network, playing a crucial role in its overall performance. Among the commonly used loss functions in segmentation tasks, dice loss is particularly prominent. It measures the overlap between the predicted segmentation map and the ground truth, effectively addressing the challenge posed by the significant imbalance between positive and negative samples, especially in tasks involving small foreground areas.

In tackling the learning of segmentation for challenging samples, focal loss stands out as a valuable tool. This loss function enhances the network's focus on difficult samples by dynamically adjusting the weight reduction rate for simpler samples. Additionally, TopK loss strategically directs the network's attention towards a selected subset of challenging samples during training. This approach enables the model to effectively concentrate on complex and unbalanced samples, thereby optimizing its learning process.

To address the issue of image imbalance, this study amalgamates the merits of dice loss, focal loss, and TopK loss, introducing the novel Hybrid Multi-task Enhanced Penalization (HMEP) loss and Easy-Penalized (EP) loss. These innovations empower the network to thoroughly investigate the foreground region, while prioritizing the scrutiny of challenging and imbalanced samples, thus augmenting the network's learning capability. The HMEP loss is formulated as a composite loss function, integrating dice loss, focal loss, and TopK loss:

$$L_{\mathit{HMEP}}=L_{\mathit{dice}}+L_{\mathit{focal}}+L_{\mathit{Topk}}$$ (1)

EP loss is a composite loss function of dice loss and focal loss:

$$L_{\mathit{EP}}=L_{\mathit{dice}}+L_{\mathit{focal}}$$ (2)

L_dice, L_focal, and L_TopK are dice losses, focal losses and TopK losses, respectively:

$$L_{\mathit{dice}}=1-\frac{2{\sum }_{i=1}^N{\sum }_{c=1}^C{g}_i^c{s}_i^c}{{\sum }_{i=1}^N{\sum }_{c=1}^C{g}_i^{c^2}+{\sum }_{i=1}^N{\sum }_{c=1}^C{s}_i^{c^2}}$$ (3)

N is the total number of voxels. C is the number of categories.

$$L_{\mathit{focal}}=-\frac{1}{N}\sum _{c=1}^C\sum _{i=1}^N{\left(1-S_i^c\right)}^{\gamma }{g}_i^c\log S_i^c$$ (4)

The log base e by default. c is the correct classification of pixel i. $g_i^c$ is the true binary indicator of the class label c of voxel i. $S_i^c$ is the corresponding prediction split probability. γ is the weight factor.

$$L_{\mathit{Topk}}=-\frac{1}{N}\sum _{c=1}^C\sum _{i\in k}{g}_i^c\log S_i^c$$ (5)

Setting K to 10 % of the worst-performing pixels.

2.6. Training of segmentation models

The dataset provided here is divided into a training set of 78 instances and a separate test set of 40 instances. During the phases of network training and evaluation, distinct neural architectures are trained independently using a 50 % cross-validation strategy. Subsequently, the optimal network configuration is determined through evaluation on the test set. During testing, multi-model integration inference is executed on the identified optimal network, resulting in the generation of an automated segmentation prediction image. The delineation process for spinal bone metastases, as outlined in this investigation and based on nnUNet’s radiomics, is visually depicted in Fig. 3. Initially, the input data undergo preprocessing before the training and evaluation of the network model.

Fig. 3.

Automatic segmentation process of spinal metastatic tumor segmentation model.

2.7. Evaluation of segmentation model

In evaluating the segmentation model to achieve a quantitative analysis of the segmentation efficacy, this paper employs the Dice coefficient, a widely utilized metric for image segmentation evaluation. The Dice coefficient serves as a measure of similarity and overlap between two samples, calculated using the following method:

$$\mathrm{D}\mathrm{i}\mathrm{c}\mathrm{e}=\frac{2\left|X\cap Y\right|}{\left|X\right|+\left|Y\right|}$$ (6)

X is the true value of the label; Y is the predicted value of the model. The coefficient of the numerator is 2 because the denominator is double-counting the common elements between X and Y.

3. Results

3.1. Comparison of segmentation effect

Fig. 4 illustrates the MRI segmentation outcomes produced by the nnUNet model for patients with spinal bone metastases. The nnUNet model accurately identifies and segments metastatic tumors in spinal MRI images, showing expertise in delineating even small lesions. The generated results closely resemble those delineated manually.

Fig. 4.

Segmentation results of spinal bone metastases. A: Original image. B: Manually sketch the result. C: nnUNet model segmentation results. D: Tumor enlargement area. Red for manual sketch, yellow for model segmentation. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

3.2. Evaluation of segmentation performance

Fig. 5 presents the results of the segmentation performance evaluation for spinal bone metastases on MRI images. The Dice coefficients achieved for the training set and the test set are 0.926 and 0.824, respectively, indicating the nnUNet's strong segmentation performance. Within the test set, the Dice coefficient for the lumbar spine reached 0.838, which surpassed the segmentation performance observed for the thoracic spine (0.785).

Fig. 5.

Evaluation of segmentation performance of nnUNet model.

3.3. Segmentation consistency detection

To assess the variance between the area of spinal bone metastases derived from the mesh segmentation model and the manually calculated area by physicians, data from 40 patients in the test set were analyzed for consistency. A scatter plot illustrating the area values of spinal bone metastases against the predicted values is depicted in Fig. 6. Notably, a strong linear correlation is observed between the manually delineated actual values and the segmentation values predicted by the network model (y = 1.0012x + 0.7097, R² = 0.998, P < 0.001).

Fig. 6.

Consistency test between model segmentation and manual segmentation.

4. Discussion

The spine stands as the most prevalent site for metastasis of malignant tumors, with an incidence ranging from 30 % to 50 % [4]. Metastatic tumors induce pain and functional impairments, significantly impacting the quality of life for affected patients [20]. As metastatic tumors progress, they compromise the mechanical integrity of the vertebral body, precipitating secondary vertebral fractures, nerve root injury, and spinal cord compression [7], [21]. As of now, the challenge of automatically localizing and segmenting spinal bone metastases in MRI images remains unresolved [22]. In clinical practice, the localization and segmentation of spinal bone metastases rely on the subjective manual segmentation performed by imaging physicians. However, this process imposes a heavy workload and demands a significant amount of time from the physicians [14], [23]. Hence, investigating a computer-aided method to assist radiologists in automatically and accurately localizing and segmenting lung cancer spinal bone metastases bears substantial clinical significance.

As computer performance advances and deep learning rapidly evolves, convolutional neural networks have become ubiquitous in medical image segmentation and detection, yielding remarkable achievements [24]. In recent years, nnUNet has emerged as a leading tool in medical image segmentation, widely adopted and demonstrating significant efficacy [25]. Built upon the U-Net architecture, nnUNet extends this framework by integrating additional losses into each decoder layer, thereby enhancing the capabilities of the traditional U-Net structure [26]. This modification enables deeper injection of gradient information into the network, thereby facilitating the training of all layers within the network architecture [18]. The nnUNet imaging genomics model stands out in tumor segmentation tasks owing to several key advantages. These include its flexible network structure, automated hyperparameter optimization, comprehensive support for data augmentation and post-processing, and remarkable performance capabilities. Alqaoud et al. [27] applied nnUNet for segmenting breast tumors, leveraging preoperative multi-modal MRI and intraoperative ultrasound to improve surgical precision and patient outcomes. Krishnan et al. [28] showcased nnUNet's ability to automatically quantify kidney and cyst volumes from MRI images, offering potential for prognostic assessment and therapeutic monitoring in autosomal dominant polycystic kidney disease.

In this study, MRI images of patients with spinal bone metastases were automatically analyzed using the nnUNet model. The findings showed that nnUNet successfully localized and segmented metastatic tumors on spinal MRI images, including small lesions, achieving results comparable to manual delineation. Liu et al. [29] conducted research on bone imaging and bone transfer utilizing the nnUNet network, which shares similarities with our work. Their study validated the accuracy of the network model.

To conduct a quantitative analysis of segmentation efficacy, this study utilizes the Dice coefficient, a widely recognized metric for evaluating image segmentation. The Dice coefficients obtained for the training set and the test set are 0.926 and 0.824, respectively, indicating commendable segmentation performance by nnUNet. Within the test set, the Dice coefficient for the lumbar spine was measured at 0.838, which surpassed the segmentation performance observed for the thoracic spine (0.785). Previous findings have indicated that the highest proportion of patients exhibit single lesions in the lumbar and thoracic regions. Consequently, it has been inferred that vertebral metastatic tumors tend to invade the thoracolumbar region [30]. The findings of this study further underscore the value of the network model for MRI image recognition of spinal metastatic tumor patients in future clinical settings. Moreover, to assess the disparity between the area of spinal bone metastases obtained by the mesh segmentation model and that calculated manually by doctors, the consistency of patient sketched data was rigorously tested. The results revealed a robust linear correlation between manually sketched real values and network model segmentation values. In clinical practice, while manual localization and segmentation of bone metastases are crucial, they can be subject to subjective differences in segmentation. Therefore, this study holds significant clinical value in enabling the segmentation of tumor location regions across the entire MRI image.

The study is subject to certain limitations. Firstly, the dataset size in this study is limited. Future efforts will involve the inclusion of multi-center image data to enhance the robustness of the findings. Secondly, the patient image data included in this study comprises only MRI, while CT data will be incorporated in future investigations for a comprehensive comparative analysis. Lastly, the utilization of various neural network architectures that incorporate advanced feature extraction methods [31], optimization strategies [32], [33], or integrated with computational analysis approaches [34], [35] can greatly improve the nnUNet model's performance in the automatic detection and segmentation of spinal bone metastases in MRI scans.

5. Conclusion

This paper investigated the automatic localization and segmentation of spinal bone metastases in MRI using the nnUNet model. The results demonstrate that this method exhibits excellent segmentation efficiency, offering imaging doctors a quantitative and non-invasive tool for image segmentation.

CRediT authorship contribution statement

Yong Xu: Methodology, Investigation. Chengjie Meng: Resources, Data curation. Dan Chen: Supervision. Yongsheng Cao: Formal analysis, Data curation. Xin Wang: Validation. Peng Ji: Writing – review & editing, Project administration.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.