3D Prostate Segmentation of Ultrasound Images Combining Longitudinal Image Registration and Machine Learning

Abstract

We developed a three-dimensional (3D) segmentation method for transrectal ultrasound (TRUS) images, which is based on longitudinal image registration and machine learning. Using longitudinal images of each individual patient, we register previously acquired images to the new images of the same subject. Three orthogonal Gabor filter banks were used to extract texture features from each registered image. Patient-specific Gabor features from the registered images are used to train kernel support vector machines (KSVMs) and then to segment the newly acquired prostate image. The segmentation method was tested in TRUS data from five patients. The average surface distance between our and manual segmentation is 1.18 ± 0.31 mm, indicating that our automatic segmentation method based on longitudinal image registration is feasible for segmenting the prostate in TRUS images.

1. INTRODUCTION

Prostate cancer is the second leading cause of cancer death in men in the United States [1]. Transrectal ultrasound (TRUS)-guided biopsy is the clinical standard for definitive diagnosis of prostate cancer. While two-dimensional (2D) TRUS-guided biopsy is routinely performed, however, 2D ultrasound images do not provide three-dimensional (3D) location information of the biopsy sample. Consequently, the physician must mentally estimate the 3D location of the biopsy needle based on limited 2D information, thus leading to suboptimal biopsy targeting. Three-dimensional transrectal ultrasound images have been used to guide prostate biopsy [2, 3] or therapy [4, 5]. Another challenge for TRUS-guided biopsy is that ultrasound imaging cannot reliably detect prostate cancer and thus cannot be used to guide targeted biopsy. Other imaging modalities such as positron emission tomography (PET) has been used to detect prostate cancer. Various PET imaging agents have been developed for prostate cancer detection and staging, these include anti-1-amino-3-¹⁸F-fluorocyclobutane-1-carboxylic acid (¹⁸F-FACBC) [6, 7], ¹¹C-choline [8, 9], ¹⁸F-fluorocholine [10], ¹¹C-acetate [11], ¹¹C-methionine [12], and other PET agents. By combining PET/CT with 3D ultrasound images, multimodality image-guided targeted biopsy may be able to improve the detection of prostate cancer as recently reported by us [3].

In order to build a 3D model from ultrasound images for TRUS-guided biopsy, segmentation of the prostate is required. Segmentation of the prostate in ultrasound images can be difficult because of the shadows from the bladder and because of a low contrast between the prostate and non-prostate tissue. Many methods were proposed to automatically segment the prostate in ultrasound images [13–31]. A semiautomatic method by warping an ellipse to fit the prostate on TRUS images was previously presented by Badiei et al [32]. A 2D semiautomatic discrete dynamic contour model was used to segment the prostate [33]. Level-set based methods [27, 34] also were used to detect the prostate surface from 3D ultrasound images. Gabor support vector machine (G-SVM) and statistical shape model were used to extract the prostate boundary [35, 36]. Accurate segmentation of the prostate plays a key role in biopsy needle placement and treatment planning [26, 37]. In this study, we developed a 3D segmentation method for prostate ultrasound images, which is based on longitudinal image registration and machine learning.

2. METHODS

Our segmentation method consists of three major components: (1) Longitudinal image registration, (2) Hierarchical representation of image features using Gabor filter banks, and (3) Training of kernel support vector machine (KSVM). Figure 1 shows a schematic flow chart of our method. The steps are briefly described as follows.

Figure 1.

Schematic flow chart of the proposed algorithm for 3D segmentation of the prostate.

1. Register the previously acquired TRUS image, e.g. pre-biopsy image, to the newly acquired TRUS image, e.g. intra-biopsy image. In our group, a variety of rigid and nonrigid registration methods have been validated for prostate image registration [38–45]. In this study, we developed a deformable registration method [46] for this particular application, which is used to obtain the spatial deformation field between the intra-biopsy image and pre-biopsy image. The same transformation is applied to the segmented prostate surface in the pre-biopsy data. Use a histogram matching algorithm, the intensity of pre-biopsy TRUS image is matched to the intensity of the intra-biopsy TRUS image.

2. Apply three Gabor filters in three orthogonal planes to extract prostate features from the registered image as well as from the newly acquired TRUS image. Every TRUS image is divided into n 3D sub-regions overlapped with each other. Every 3D sub-region has three sub-volumes of Gabor features in the three orthogonal planes.

3. Train and apply the kernel support vector machines (KSVM) for segmentation. Once the KSVMs are trained with the three sub-volumes of each sub-region in the pre-biopsy images, they are applied to the corresponding sub-volumes of the same sub-region in the newly acquired TRUS image in order to segment the prostate.

Gabor filter banks are used to capture image features in multiple scales and multiple orientations [47]. To save computation time and also reasonably approximate the complete set of 3D Gabor features, we use three banks of 2D Gabor filters located in the three orthogonal planes (Figure 2). The three selected filter banks are orthogonal or parallel to the axis of the ultrasound probe. Below, 2D Gabor filter banks in the transverse plane are described as follows [30]:

$$t(x,y)=\frac{1}{2\pi {\sigma }_x{\sigma }_y}exp[-\frac{1}{2}(\frac{x^2}{{\sigma }_x^2}+\frac{y^2}{{\sigma }_y^2})+2\pi \sqrt{-1}(u_{0}x+v_{0}y)]$$ (1)

Figure 2.

The frequency spectra of three orthogonal banks of 2D Gabor filters located in three orthogonal planes. The ellipse contour denotes the half-peak magnitude of the filter responses. The axes x, y, and z denote the three orthogonal directions, respectively.

Where (u₀, v₀) is the spatial frequency of the filter in the frequency domain, and σ_x and σ_y are the standard deviation of the Gaussian function. Similarly, the 2D Gabor filter banks in the other two orthogonal planes can be described as g(y, z) and h(x, z), respectively. When applying the above Gabor filter banks to an individual voxel τ, three sets of texture features are obtained, which correspond to the real and imaginary parts of the three orthogonal filter banks:

$$\{T_{\lambda ,\theta }^{\mathit{real}}(\tau )\},\{T_{\lambda ,\theta }^{\mathit{imag}}(\tau )\},\{G_{\lambda ,\theta }^{\mathit{real}}(\tau )\},\{G_{\lambda ,\theta }^{\mathit{imag}}(\tau )\},\{H_{\lambda ,\theta }^{\mathit{real}}(\tau )\},\{H_{\lambda ,\theta }^{\mathit{imag}}(\tau )\}$$ (2)

Where λ and θ are the number of the scales and orientations, respectively. Figure 3 shows the extracted Gabor features from an example TRUS image. Notably, although certain Gabor filters can be modeled as orientation and scale tunable edge detectors or smoothing filters, the micro-features extracted by these filters in a given region often jointly characterize the underlying texture information.

Figure 3.

Gabor features extracted by three orthogonal Gabor filter banks. Left: From top (a₁) to bottom (a₅), the transverse image, the Gabor features of the two real components ( $H_{4,1}^{\mathit{real}}$ and $H_{4,4}^{\mathit{real}}$) and the two imaginary ( $H_{5,2}^{\mathit{imag}}$ and $H_{5,3}^{\mathit{imag}}$) components, respectively. Middle and right: The TRUS images and the Gabor features in the coronal and saggital orientations, respectively.

Kernal support vector machine (KSVM) is a statistical classification method proposed by Vapnik [48]. Given m samples, KSVM can create a nonlinear decision plane that optimally separates the positive samples from negative samples based on the statistical learning theory. We use a number of KSVMs, each of which is applied to a 3D subregion in the TRUS image. Particularly, the entire TRUS image is uniformly divided into sub-regions that are designed to be overlapped with each other. Each KSVM is trained by Gabor texture feature vectors extracted from prostate tissue and non-prostate tissue, respectively. During the training stage, the texture priors of the prostate and non-prostate samples are implicitly characterized. The pre-trained KSVM is then used to label tissue at each voxel according to its corresponding Gabor texture feature vector. The KSVM outputs a real value that is nonlinearly mapped to [0, 1], indicating the likelihood of the voxel belonging to the prostate tissue.

3. RESULTS

We apply the segmentation method to five patients’ TRUS images (matrix: 224×224×175, 0.380×0.380×0.390 mm³/voxel) that were acquired pre- and post-biopsy. In our implementation, 36 KSVMs are attached to 36 sub-regions. The number of orientation and scales are 6 and 3 for the Gabor filter, respectively. In our experiment, we used pre-biopsy images to segment post-biopsy images. Five corresponding post-biopsy images were used for evaluation. The automatic segmentation results were evaluated by the manual segmentation results. Dice overlap ratio was used as the metric to evaluate the performance of the segmentation [49–51].

Figure 4 shows the nonrigid registration result between the newly acquired TRUS image (post-biopsy) and previously acquired image (pre-biopsy). As the 3D images at different time points have different sizes of the prostate and their TRUS images may be acquired at slightly different position and orientation, the nonrigid registration (translations, rotations, scaling, and deformation) is able to normalize the image. Figure 5 shows the segmentation results for one patient. The results from the automatic segmentation methods are close to the manual segmentation.

Figure 4.

Visual assessment of 3D TRUS registration. a₁, b₁ and c₁ are post-biopsy images in three directions. a₂, b₂ and c₂ are pre-biopsy images, and a₃, b₃ and c₃ are the fusion images between pre- and post-biopsy images. a₄, b₄ and c₄ are the 3D registered pre-biopsy images; and a₅, b₅ and c₅ are the fusion images between registered pre- and post-biopsy images.

Figure 5.

Comparison between the proposed method and manual segmentation. Images from left to right are in three orientations of the same TRUS image volume. The line in yellow is the manual segmentation result. The dashed line in red is the segmentation result of the proposed method.

Table 1 provides the quantitative evaluation results of the segmentation method for the five patients. The average surface distances range from 0.88 mm to 1.63 mm with a mean distance of 1.18 mm. The root mean squares (RMS) vary from 1.14 mm to 1.93 mm with a mean of 1.43 mm. The maximum distance (MAX) is less than 5.00 mm for the five patients. The Dice ratios range from 87.0% to 90.2% with a mean of 88.1%.

Table 1.

Comparison between our and manual segmentations.

Patients ID	Surface Distance (mm)		Max	Dice (%)
	Average	RMS
28	1.63	1.93	4.72	87.0
23	1.37	1.54	3.03	87.3
32	0.97	1.30	4.39	89.1
17	0.88	1.14	3.42	90.2
25	1.03	1.28	3.92	87.1
Mean ± STD	1.18±0.31	1.43±0.31	3.89±0.70	88.1±1.44

4. CONCLUSIN

This paper presented an automatic, 3D segmentation method for prostate TRUS images based on longitudinal image registration. Using longitudinal images for each individual subject, we registered previously acquired images to newly acquired images of the same subject. Patient-specific Gabor features from the images at different time points were used to train kernel support vector machines (KSVMs) and then to segment the newly acquired prostate image. Validation on patient data shows that our approach is able to accurately and robustly segment the prostate from TRUS images. As the patient-specific pre-biopsy TRUS image is used to train the kernel support vector machines for automatic segmentation, the robustness and accuracy of the algorithm is improved. The reliability of the segmentation method does not depend on any initialization step. We are integrating the proposed segmentation method into our 3D ultrasound-guided biopsy system.