CT prostate segmentation based on synthetic MRI-aided deep attention fully convolution network

Abstract

Purpose:

Accurate segmentation of the prostate on computed tomography (CT) for treatment planning is challenging due to CT’s poor soft tissue contrast. Magnetic resonance imaging (MRI) has been used to aid prostate delineation, but its final accuracy is limited by MRI-CT registration errors. We developed a deep attention-based segmentation strategy on CT-based synthetic MRI (sMRI) to deal with the CT prostate delineation challenge without MRI acquisition.

Methods and materials:

We developed a prostate segmentation strategy which employs an sMRI-aided deep attention network to accurately segment the prostate on CT. Our method consists of three major steps. First, a cycle generative adversarial network was used to estimate an sMRI from CT images. Second, a deep attention fully convolution network was trained based on sMRI and the prostate contours deformed from MRIs. Attention models were introduced to pay more attention to prostate boundary. The prostate contour for a query patient was obtained by feeding the patient’s CT images into the trained sMRI generation model and segmentation model.

Results:

The segmentation technique was validated with a clinical study of 49 patients by leave-one-out experiments and validated with an additional 50 patients by hold-out test. The Dice similarity coefficient, Hausdorff distance, and mean surface distance indices between our segmented and deformed MRI-defined prostate manual contours were 0.92 ± 0.09, 4.38 ± 4.66, and 0.62 ± 0.89 mm, respectively, with leave-one-out experiments, and were 0.91 ± 0.07, 4.57 ± 3.03, and 0.62 ± 0.65 mm, respectively, with hold-out test.

Conclusions:

We have proposed a novel CT-only prostate segmentation strategy using CT-based sMRI, and validated its accuracy against the prostate contours that were manually drawn on MRI images and deformed to CT images. This technique could provide accurate prostate volume for treatment planning without requiring MRI acquisition, greatly facilitating the routine clinical workflow.

1. INTRODUCTION

Prostate cancer is the most common cancer and the second leading cause of cancer death among men in the United States.¹ Depending on the risk of recurrence and extent of disease, around 30% of prostate cancer patients receive radiation therapy.² External beam radiotherapy is a noninvasive modality which provides localized dose delivery, and has been widely adopted for the definitive treatment of prostate cancer. Prostate delineation is a prerequisite for treatment planning.^3,4 In clinical practice, the prostate is contoured manually. Due to the low soft tissue contrast of CT images, prostate delineation suffers from inter- and intra-observer variability, and the delineation accuracy is highly dependent on the clinician’s expertise and experience.⁵ Studies shows that CT-defined prostate volume often overestimates the prostate by over 30%.^6,7 Computer-aided prostate segmentation methods have been developed to improve the accuracy of organ delineation. The current CT prostate segmentation techniques can be briefly summarized as the following:⁸

1. Contour- and shape-based methods segment the prostate based on boundary information, which may be affected by unreliable edge information in CT images.⁹ Prior shape information is incorporated to provide better results.¹⁰

2. Region-based methods use predominant intensity distributions of the prostate region to segment the CT contour. These methods are further categorized into atlas¹¹ and level set methods.¹²

3. Clustering-based methods extract a set of feature vectors with the goal of identifying groups or clusters of similar objects based on the feature vectors. Proximity measures are used to group data into clusters of similar types.¹³

4. Machine learning-based methods regard prostate as a learning-based target.¹⁴ These methods use a set of CT images with manual contour as a priori information to build a classifier to assign a new arrival CT voxel to prostate and non-prostate regions. Supervised and unsupervised techniques are used to build the classifier to divide the feature space into different regions.

Supervised machine learning has demonstrated enormous potential in medical image segmentation. According to the training model used, these methods can be grouped into support vector machines (SVM)-based,¹⁵ random forest (RF)-based,¹⁶ and deep learning-based methods.^17,18 The SVM-based and RF-based methods use handcraft information, such as texture features, to train a SVM or RF classifier. However, handcraft features, such as histogram of oriented gradient (HoG), local binary pattern (LBP), and Haar wavelet features in SVM- and Haar-like features in RF-based methods, may not accurately represent the CT image. To address this issue, feature selection has been incorporated to identify the most informative features who have the discriminative power to differentiate prostate from non-prostate.¹⁵ Deep features, which are directly assigned with semantic and structural information of the prostate region, are used to train a deep learning-based classifier. Liu et al. used deep neural networks (DNN) to learn the deep features for prostate segmentation.¹⁷ DNN can automatically learn the deep features adaptive to the data, which differs from the handcrafted features. Kazemifar et al. used a 2D U-Net structure, a DNN variant, to delineate the prostate, bladder, and rectum in pelvic CT images.¹⁹ However, this method was based on 2D inputs, which lacks 3D spatial information. Balagopal et al. presented a fully automated workflow for male pelvic CT image segmentation using deep learning. The architecture consists of a 2D U-Net localization network followed by a 3D U-Net segmentation network for volumetric segmentation of prostate, bladder, rectum, and femoral heads.²⁰

Nevertheless, the accuracy of these methods is limited by the inherent low soft tissue contrast of CT images, which usually yields overestimation of prostate volume. MRI has been used to aid prostate delineation due to its superior soft tissue contrast, but the corresponding prostate contour needs to be registered to CT images for dose calculation. Therefore, its accuracy is subject to MR-CT registration errors. Moreover, MRI may not be always available due to insurance issues or lack of device accessibility.

This study aims to develop an accurate prostate segmentation strategy on CT images with CT-based sMRI to deal with prostate segmentation challenges without requiring MRI acquisition. The contributions of the paper are as follows:

1. Compared to CT images, the superior soft tissue contrast of sMRI improves the prostate segmentation accuracy and alleviates the issue of prostate volume overestimation when using CT images alone.

2. A deep attention fully convolution network (DAFCN) was introduced to enhance the features’ discriminative ability to differentiate the prostate and non-prostate regions. The attention models were introduced to select the most relevant features to differentiate prostate and non-prostate regions.²¹

The paper is organized as follows: We first provide an overview of the proposed CT prostate segmentation framework in Section 2, followed by detailed description of the sMRI generation using a cycle-consistent generative adversarial network (CycleGAN),²² and prostate segmentation using DAFCN.²¹ We evaluated the proposed method through a comparison with a state-of-the-art U-Net segmentation method by leave-one-out experiments,²⁰ and test the robustness of the proposed method by hold-out test. We demonstrate that prostate segmentation is enhanced with the integration of sMRI, and compared prostate contours generated on sMRI and CT using both U-Net and proposed DAFCN. Finally, along with an extended discussion, we conclude the presentation of our novel sMRI-aided CT prostate segmentation framework.

2. MATERIALS AND METHODS

2.A. Overview

We propose a new prostate segmentation strategy to accurately segment the prostate on CT images. Figure 1 outlines the schematic flow chart of the proposed method. The segmentation consists of three major steps. First, a CycleGAN was used to estimate sMRI from CT images. By introducing an inverse CT-to-MRI mapping, CycleGAN can enforce the mapping from CT to MRI to be close to a one-to-one mapping.²³ Second, a DAFCN was trained on sMRI and corresponding prostate contours that were delineated on real MRI and deformed to CT images. A deep attention model was integrated into the DAFCN to retrieve the most relevant features that can represent prostate and non-prostate boundaries. Deep supervision was also incorporated into this DAFCN to enhance the features’ discriminative ability.^24,25 Finally, the segmented prostate contour of a test patient case was obtained by first feeding the CT image into the trained CycleGAN to generate the sMRI, and by then fed the sMRI into the trained DAFCN to generate the segmentation. The individual components of the algorithm are outlined in further detail in the following sections. The details of used network architectures are shown in Fig. 2.

FIG. 1.

The schematic flow diagram of the proposed method. The first row shows the synthetic magnetic resonance imaging generation using CycleGAN, and the training procedure of segmentation model. The second row shows the segmentation stage for a new patient’s computed tomography image.

FIG. 2.

The architectures of proposed networks.

2.B. Image acquisition

In this retrospective study, we collected the CT and MRI images of 49 patients who had received pelvic radiation therapy in our institution. MRIs were acquired using a Siemens standard T2-weighted MRI scanner with 3D T2-SPACE sequence and 1.0 × 1.0 × 2.0 mm³ voxel size (TR/TE: 1000/123 ms, flip angle: 95°). MRIs were acquired at 1.5 T. Built-in distortion corrections were applied during reconstruction. CTs were captured with a Siemens CT scanner with 1.0 × 1.0 × 2.0 mm³ voxel size with 120 kVp and 299 mAs. The prostate contours manually delineated by physician on MRI images were also collected for our study. For each patient, all training MR and CT images were deformably registered with an intra-subject strategy using a commercial software, Velocity AI 3.2.1 (Varian Medical Systems, Palo Alto, CA).

2.C. Synthetic MRI generation

The deformed MR image was used as the learning-based target of the planning CT image for our proposed sMRI-aided strategy. Because local mismatches between MR and CT images remain even after deformable registration, as the images have fundamentally different properties, training a CT-to-MRI transformation model is difficult. To cope with this challenge, inspired by a recent 2D CycleGAN study,²² we introduced a 3D CycleGAN in our sMRI generation algorithm because of its ability to mimic target data distribution by incorporating an inverse MRI-to-CT mapping.²⁶ In addition, the patient anatomy can vary significantly among individuals. In order to accurately predict each voxel in the anatomic region (air, bone, and soft tissue), inspired by densely connected CNN,²⁷ we introduced several dense blocks to capture multiscale information (including low-frequency and high-frequency) by extracting features from previous and following hidden layers.

As shown in generator architecture of Fig. 2, after two convolution layers with stride size of 2 to reduce the feature map sizes, the feature map goes through nine dense blocks, and then two deconvolution layers and a tanh function to perform end-to-end mapping, that is, equal-sized input and output. The tanh function works as a nonlinear activation function and makes it easy for the model to generalize or adapt to a variety of data and to map structures with similar CT intensities to different structures on MRI, such as prostate and bladder.²⁸ The dense block is implemented by six convolution layers. A first layer is applied to the input to create k feature maps, which are concatenated to the input. A second layer is then applied to create another k feature maps, which are again concatenated to the previous feature maps. The operation is repeated five times. Then, the output of these layers goes through the last layer to shorten the feature maps to k.²⁹

The discriminator is used to judge the realism of generated synthetic images against the original images. As shown in the discriminator architecture part, the discriminator is a typical classification-based FCN, which consists of multiple convolution layers. The number of convolution layers depends on input patch size, since the input patch is 64 × 64 × 64, the number of convolution layers is nine, where eight layers are with batch normalization and with stride size of 2, the last one layer is without batch normalization but with sigmoid function. The discriminator outputs a reduced size patch with element 1 denoting real and element 0 denoting fake.

Figure 3 shows axial views of the CT image, sMRI, and deformed prostate manual contour at two axial levels for a patient. These images well demonstrate that sMRI enhanced the prostate soft tissue contrast as compared to CT. To better illustrate the contrast enhancement of sMRI, Figs. 3(a4) and 3(b4) compare the profiles of the dashed yellow lines in subfigures (a1) and (b1) for CT image and sMRI. To provide a meaningful comparison, we use a $\frac{x-\text{min}\left(X\right)}{\text{max}\left(X\right)-\text{min}\left(X\right)}$ normalization to scale voxel intensities on the dash line to [0, 1], where x denotes a voxel’s intensity on dash line, X denotes the all voxels’ intensity appeared on dash line. In addition, the plot profile of the binary mask of the prostate manual contour on the dash line was used to determine the boundary of the prostate. To make this boundary clear, we set 0 as non-prostate and 1.1 as prostate for the binary mask. Thus, the boundary of the prostate is the jump discontinuity on the plot profile of a binary mask. As is shown in the subfigures (a4) and (b4), sMRI provides superior prostate contrast to CT images.

FIG. 3.

Visual results of generated synthetic magnetic resonance imaging (sMRI). (a1) and (b1) show the original computed tomography (CT) image at two axial levels, (a2) and (b2) show the generated sMRI, (a3) and (b3) show the deformed manual contour, (a4) and (b4) show the plot profiles of CT, sMRI, and manual contour of the yellow dashed line in (a1) and (b1), respectively.

2.D. Deep attention fully convolution network

The DAFCN was trained on the patients’ pelvic sMRIs, with the binary mask of corresponding manual prostate contour used as learning-based target. These manual prostate contours were delineated by physicians on MR, and deformed to CT images, which were verified and approved for treatment planning. A 2.5D DAFCN, which takes five consistent sMRI transverse slices as 5-channel input, was used to perform an end-to-end segmentation. As shown in Fig. 2, the DAFCN network consists of three compression and decompression paths, each of compression and decompression paths are connected by a bridge path. The compression path is constructed by three convolution layers, followed by a max-pooling layer. In each convolution layer, feature representations can be extracted via 2D convolutions followed by the parametric rectified linear unit (PReLU). The decompression path is constructed by a deconvolution layer to enhance the resolution and followed by two convolutional layers. To output equal-sized feature maps, 2 × 2 stride size is adopted for deconvolutions. The bridge path concatenates the feature maps from the current compression path and two previous decompression paths. DAFCN with such concatenation encourages each path to obtain both high-frequency information (such as textural information) and low-frequency information (such as structural information) to represent the image patch. At the end of DAFCN, probability maps of contours are generated with soft-max operators. A threshold was used to binarize the probability maps to binary masks of contours, which we denote as segmented contours.

Rather than directly using the feature maps from compression path, attention gates (AGs) are incorporated to focus on the prostate boundary structures. Previous work demonstrated that by integrating AGs into a standard U-Net model,²¹ the most relevant semantic contextual information can be captured without a requirement to enlarge the receptive field, which is highly beneficial for organ localization. As is shown in Fig. 2, the feature maps extracted from the coarse scale is used in gating to disambiguate irrelevant and noisy responses in skip connections. This is performed immediately prior to the concatenation operation to merge only relevant activations. Additionally, AGs filter the neuron activations during both the forward pass and the backward pass. Gradients originating from background regions are downweighted during the backward pass. This allows model parameters in shallower layers to be updated mostly based on spatial regions that are relevant to a given task. By using AGs, the most salient features from compression path are highlighted and are passed through the bridge path. The detailed implementation of AGs is introduced in recent works.^21,30 We also use deep supervision to force the intermediate feature maps to be semantically discriminative at each image scale.³¹ This helps to ensure that AGs, at different scales, have an ability to influence the responses to a large range of prostate content.

To demonstrate the efficacy of deep attention, we compared the results of our proposed DAFCN algorithm against the algorithm without using deep attention, that is, deeply supervised FCN (DSFCN). Figure 4 shows 3D scatter plots of the first three principal components of patch samples in the feature maps of each decompression path outputs. We randomly selected 2000 samples from the prostate region as well as the non-prostate region around the prostate boundary, as shown in the subfigure (a4). The scatter plots of a DSFCN in the subfigure (b1–-b4) illustrate an overlap between the samples from the prostate and the non-prostate regions; thus, these two regions cannot be easily separated. Whereas with DAFCN, as shown in the subfigure (c1–c4), the prostate and non-prostate samples can be readily separated, demonstrating the significant enhanced discerning capability with the addition of a deep attention strategy.

FIG. 4.

An illustrative example of the benefit of our deep attention fully convolution network (DAFCN) compared with DSFCN without attention gate, (a1) shows the original CT image in transverse plane. (a2) shows corresponding manual contour, (a3) shows the generated synthetic magnetic resonance imaging (sMRI), (a4) shows the sample patches’ central positions drawn from sMRI, where the samples belonging to the prostate are highlighted by green circles, and the samples belonging to the non-prostate are highlighted by red asterisks. (b1–b4) show the scatter plots of the first three principal components of corresponding patch samples in feature maps at each decompression path outputs by using a DSFCN, respectively. (c1–c4) show the scatter plots of first three principal components of corresponding patch samples in the feature maps at each decompression path outputs by using our DAFCN, respectively. The position of the viewer in (b1–b4) and (c1–c4) is azimuth = 10° and elevation = 30°.

2.E. Implementation details

The computation complexity is higher than state-of-the-art U-Net algorithms due to having another CycleGAN-based sMRI generation model used. At training stage, the MRI and CT images were fed into the network in 64 × 64 × 64 patches, with an overlap between any two neighboring patches of 48 × 48 × 48. This overlap ensures that a continuous whole-image output can be obtained and allows for increased training data for the network. The learning rate for Adam optimizer was set to 2e-4, and the model was trained and tested on two NVIDIA Tesla V100 with 32 GB of memory for each GPU, a batch size of 4. 10 GB CPU memory and 58 GB GPU memory were used for each batch optimization. It takes about 36 min per 2000 iterations during training. The training was stopped after 150 000 iterations. Model training takes about 45 h. For testing, sMRI patches were generated by feeding CT patches into the trained CycleGAN model. Pixel values were averaged when overlapping exists. The sMRI generation for one patient takes about 23 min. Tensor-flow was used to implement both CycleGAN and DAFCN network architecture. During segmentation, the input patch size of our proposed DAFCN was set to 512 × 512 × 5, binary cross entropy was used as loss function, and the learning rate for Adam optimizer was set to 1e-3. The training was stopped after 200 epochs. For each epoch, the batch size was set to 20. The training of DAFCN model takes about 1.7 h, whereas the U-Net model takes more training time (about 2.4 h) because it first trains a 2D U-Net for localization and then trains a 3D U-Net for segmentation.²⁰ However, after training, our proposed algorithm has similar segmentation times compared to U-Net.²⁰ A single prostate segmentation can be completed in 1–2 s.

2.F. Quantitative measurements

We used leave-one-out experiments to evaluate the proposed segmentation algorithm on 49 patients’ data. That is, for each experiment, one patient was used as test data and the remaining 48 patients were used as training data. We excluded test patient from training our deep learning-based segmentation model. After training, this excluded patient’s CT image was used for the segmentation test. We repeated the experiment 49 times to let each patient used as test data exactly once. Our segmentation results were compared with the contours that were manually delineated on MRI and deformed to CT images. We repeated on all ways to cut the original data on a validation set of one patient data and a training set. In addition, a hold-out test with additional 50 patients’ data was performed to test the robustness of the proposed method. We trained the proposed sMRI-aided DAFCN model from our previous 49 patients’ data and test the model by these 50 patients’ data.

We calculated the dice similarity coefficient (DSC), precision score, recall score, Hausdorff distance (HD), mean surface distance (MSD), and the residual mean square distance (RMSD) to evaluate the accuracy of our segmentation method. The calculation of these metrics is introduced in recent studies.^25,32–34 The DSC, precision, and recall scores are used to quantify volume similarity between two contours. The HD, MSD, and RMSD metrics are used to quantify boundary similarity between two surfaces. Generally speaking, more accurate segmentation results are associated with lower HD, MSD, and RMSD scores and higher DSC, precision, and recall scores.

To illustrate the significant improvement of our proposed step-by-step enhancement, a paired two-tailed t-test was used between the results of any two comparison methods. Corrected P-value by using Bonferroni correction method was also used to compare the significance of the improvement.

3. RESULTS

3.A. Efficacy of sMRI-aided strategy

To evaluate the influence of sMRI-aided strategy on the segmentation, we quantitatively compared the results obtained with the proposed DAFCN tested separately on CT and sMRI images. As is shown in Table I, compared to CT segmentation, our sMRI-aided segmentation demonstrates superior performances on DSC, sensitivity, HD, MSD, and RMSD metrics.

TABLE I.

Numerical comparison by using a deep attention fully convolution network on computed tomography (CT) and synthetic magnetic resonance imaging (sMRI) data.

	DSC	Sensitivity	Specificity	HD	MSD	RMSD
CT	0.86 ± 0.08	0.80 ± 0.10	0.94 ± 0.04	7.36 ± 7.38	1.11 ± 1.19	1.79 ± 2.02
sMRI	0.92 ± 0.09	0.92 ± 0.12	0.93 ± 0.06	4.38 ± 4.66	0.62 ± 0.89	1.07 ± 1.42
P-value (sMRI vs CT)	<0.001	<0.001	0.256	<0.001	<0.001	<0.001
Corr P-value (sMRI vs CT)	8.43E-13(1)	4.01E-16(l)	1(0)	1.41E-3	2.00E-5	7.24E-4

HD, Hausdorff distance; MSD, mean surface distance, RMSD, residual mean square distance.

3.B. Contribution of deep attention

We also compared the numerical results of DAFCN with those of DSFCN on sMRI data in Table II. As is shown, DAFCN achieved significant improvements on all metrics over DSFCN.

TABLE II.

Numerical comparison by using deep attention fully convolution network (DAFCN) and DSFCN on synthetic magnetic resonance imaging (sMRI) data.

	DSC	Sensitivity	Specificity	HD	MSD	RMSD
DSFCN	0.88 ± 0.07	0.87 ± 0.10	0.90 ± 0.05	6.43 ± 4.21	0.85 ± 0.66	1.38 ± 0.97
DAFCN	0.92 ± 0.09	0.92 ± 0.12	0.93 ± 0.06	4.38 ± 4.66	0.62 ± 0.89	1.07 ± 1.42
P-value (DAFCN vs DSFCN)	<0.001	<0.001	<0.001	<0.001	<0.001	0.004
Corr P-value (DAFCN vs DSFCN)	1.31E-9(1)	1.39E-7(1)	8.89E-7(1)	1.71E-4(1)	2.18E-3(1)	2.40E-2(l)

HD, Hausdorff distance; MSD, mean surface distance, RMSD, residual mean square distance.

3.C. Comparison with U-Net method

To further validate our proposed method, we compared its performance against a state-of-the-art U-Net method.²⁰ This method consists of a 2D localization U-Net followed by a 3D segmentation U-Net for volumetric segmentation of the prostate.²⁰ In addition, to evaluate the performance of sMRI-aided and deep attention strategies separately, the comparison of U-Net to our proposed DAFCN was tested on both CT and sMRI images. In other words, we compared four strategies: U-Net on CT (CT U-Net), U-Net on sMRI (sMRI U-Net), DAFCN on CT (CT DAFCN), and our proposed method, that is, DAFCN on sMRI (sMRI DAFCN).

Figure 5 compares the segmentation results of these four strategies. As shown in subfigure (b1–b5) and (d1–d5), the prostate contour generated with the proposed method (sMRI DAFCN) shows the best resemblance to the manual contour.

FIG. 5.

Comparison of the proposed method with state-of-the-art U-Net model. (a1) and (c1) show the computed tomography (CT) image. (a2) and (c2) show the magnetic resonance imaging (MRI). (a3) and (c3) show the synthetic MRI (sMRI). (a4) and (c4) show the zoomed-in region of CT of yellow rectangle shown in (a1) and (c1). (a5) and (c5) show the zoomed-in region of sMRI of that yellow rectangle. (b1–b5) and (d1–d5) show the zoomed-in binary mask of manual contour, segmented contour obtained by CT U-Net, sMRI U-Net, CT deep attention fully convolution network (DAFCN), and sMRI DAFCN, respectively.

Table III compares the numerical results obtained by these four methods. As is shown in this table, the proposed sMRI DAFCN method outperformed other three methods on DSC, sensitivity, HD, MSD, and RMSD metrics, and generates comparable specificity on prostate contours.

TABLE III.

Quantitative metrics comparison of our proposed algorithm versus U-Net on both computed tomography (CT) and synthetic magnetic resonance imaging (sMRI) by leave-one-out experiments.

	DSC	Sensitivity	Specificity	HD	MSD	RMSD
CT U-Net	0.84 ± 0.07	0.77 ± 0.11	0.94 ± 0.05	7.85 ± 4.27	1.18 ± 0.71	1.87 ± 1.05
sMRI U-Net	0.88 ± 0.07	0.89 ± 0.10	0.87 ± 0.05	5.89 ± 3.28	0.85 ± 0.60	1.34 ± 0.87
CT DAFCN	0.86 ± 0.08	0.80 ± 0.10	0.94 ± 0.04	7.36 ± 7.38	1.11 ± 1.19	1.79 ± 2.02
sMRI DAFCN	0.92 ± 0.09	0.92 ± 0.12	0.93 ± 0.06	4.38 ± 4.66	0.62 ± 0.89	1.07 ± 1.42
P-value (our vs U-Net CT)	<0.001	<0.001	0.234	<0.001	<0.001	<0.001
Corr P-value (our vs U-Net CT)	3.58E-11(1)	2.34E-13(1)	1(0)	8.42E-7(1)	3.64E-7(1)	1.20E-6(l)
P-value (our vs U-Net sMRI)	<0.001	<0.001	<0.001	<0.001	<0.001	0.012
Corr P-value (our vs U-Net sMRI)	1.15E-9(1)	1.06E-3(l)	3.05E-11(1)	1.96E-04(l)	4.29E-03(l)	7.08E-2(l)
P-value (our vs DAFCN CT)	<0.001	<0.001	0.256	<0.001	<0.001	<0.001
Corr P-value (our vs DAFCN CT)	8.46E-13(1)	4.01E-16(l)	1(0)	1.40E-03(l)	2.00E-05(l)	7.26E-4(1)

HD, Hausdorff distance; MSD, mean surface distance, RMSD, residual mean square distance.

To test the robustness of our proposed method, hold-out test with additional 50 patients’ data was performed. As is shown from Table IV, the performance of the proposed method (sMRI DAFCN) on the additional patients’ data is consistent with the performance of that on leave-one-out experiments.

TABLE IV.

Quantitative metrics of our proposed algorithm on hold-out test.

	DSC	Sensitivity	Specificity	HD	MSD	RMSD
Proposed	0.91 ± 0.07	0.91 ± 0.08	0.93 ± 0.04	4.57 ± 3.03	0.62 ± 0.65	1.11 ± 1.47

HD, Hausdorff distance; MSD, mean surface distance, RMSD, residual mean square distance.

4. DISCUSSION

We proposed a prostate segmentation method which incorporates both an sMRI-aided strategy and a deep attention strategy into an DAFCN architecture to automatically segment the prostate using CT images only. The proposed method makes use of the superior soft tissue contrast of sMRI, bypasses MR acquisition and also CT-MR registration with its potential registration errors, and has the potential to generate accurate and consistent prostate segments using CT images alone.

Recently, Liu et al. reported a DNN-based CT prostate segmentation and mean DSC between their DNN-based prostate segmentation and physician-generated contours for test data in three groups were 0.85 ± 0.06, 0.85 ± 0.04, and 0.88 ± 0.03, respectively.¹⁷ The HD was 7.0 ± 3.5 mm, 7.3 ± 2.0 mm, and 6.3 ± 2.0 mm for three groups.¹⁷ Our results showed that the mean DSC and HD of 44 3D TRUS images were 0.92 ± 0.09 and 4.38 ± 4.66 mm, respectively. Although the mean metrics of our method is better than their results, the standard deviation of our method is larger than what Liu et al. reported. This larger standard deviation may be caused by the relatively smaller size of database used in our study. We will validate the proposed method on more patients’ data in future.

There are several limitations of the proposed method. First, our ground truth of CT prostate volume was obtained from the manual contours, which were delineated on MRI images by physicians and deformed to CT images. These manual contours may have been affected by systematic errors, deformation uncertainty, and random errors. Our proposed method may mitigate random errors. However, the systematic errors (e.g., physician’s contouring style) and deformation uncertainty (e.g., small local MRI-CT mismatch) can still affect our final segmented results. In addition, we need deformable registered MRI-CT to train the CycleGAN-based sMRI generation model. In this sense, the registration accuracy of the data also affects the delineation accuracy ultimately. Thus, accurate deformable registration would be expected to be a limitation to our proposed methods.

Second, the number of patients is relatively small. Since we used a small set of data from one institution/imaging system, the method has not been shown that it is generalizable. Multi-institutional study will be done in the future. Recently, unpaired CycleGAN was introduced to estimate synthetic CT images from MRIs by training on unpaired dataset.³⁵ This method could be used to take care of the deformation correction. In the future, by adding more sufficient data, we will focus on training an unpaired CycleGAN network and our deep attention segmentation network on more sufficient data to handle the deformation accuracy limitation.

Third, in this study, we only involved the segmented organ of prostate. Organs at risk (OARs), such as rectum and bladder, are also generally well delineated from MRIs. This method could also be used for other organs from CT images. The male CT pelvic images multi-organ segmentation will be our future work. In addition, it is hard to identify the boundaries of the prostate tumor in the CT scan, whereas in the T2-weighted MRI, the tumor structure within the prostate is apparent. Detecting and segmenting prostate tumors from CT images by using the proposed sMRI-aided strategy would be our future work.

This study aims to propose an accurate prostate segmentation method in the context of radiation treatment planning and demonstrate its feasibility. In order to evaluate its clinical utility, a dosimetric study on treatment planning based on our segmentations is worth further investigation. Moreover, the biological effect stem from the dosimetric benefit directly affects the tumor control and toxicity. Studies about the biological effect of radiation dose and volume, such as QUANTEC,³⁶ are extensive. These studies present different models and parameters to describe the relationship between received dose and tissue injury, which would be helpful in estimating the resulted biological effect in future study.

5. CONCLUSIONS

In summary, we developed an accurate prostate segmentation strategy on CT images with CT-based sMRI to deal with the challenge of prostate delineation. This technique could provide accurate prostate volumes for treatment planning without requiring MRI acquisition, greatly facilitating the routine clinical workflow.