Principal Curve Based Semi-Automatic Segmentation of Organs in 3D-CT

Abstract

Radiation therapy plays an important and effective role in the treatment of cancer. A main goal in radiation therapy is to deliver high radiation doses to the perceived tumors while minimizing radiation to surrounding normal tissues. Manual delineation of tumors and organs-at-risk(OARs) on three-dimensional computed tomography (3D-CT) is both a time-consuming and labor intensive task, and there maybe variability between manual delineations by different radiation oncologists. In this paper, we present a semi-supervised method to segment the contours of organs represented by piecewise linear segments connected with a small number of points given the user’s input in one or more slices as an approximate initialization. This method detects ridge samples from the kernel interpolation of the edge map and approximates the shape of organs using piecewise linear segments among those sample points based on the principal curve score. Results are provided in two 3D-CT scans. Evaluation of the efficacy of our semi-automatic segmentation method are based on the overlapping ratio between the manually delineated contours and the semi-automatic segmented contours represented by a small number of points. The preserved points can be as low as 10 percent of the initial manual points, and the Dice Coefficients are approximately 0.93 for lung segmentation.

I. INTRODUCTION

Automatic and semi-automatic segmentation of all regions of interest (ROIs) have been an intense research topic. A wide variety of medical image segmentation techniques have been developed. Most are based on the gray intensity of each pixel, such as thresholding [8], region growing, split and merge [4], edge detection [1]. However, the accuracy of the segmentation is critically dependent on the quality of the image. If the image is noisy or has low resolution or overlapping gray-level range between different organs, gray levels alone may not be sufficient to segment the ROIs accurately. Therefore, these approaches are often combined with other segmentation algorithms.

Deformable models are curves or surfaces deformed under the influence of external and internal energy within the image, and are widely used in the segmentation in biomedical applications. Huang [5] proposed a semi-automatic CT segmentation in tumors and organs using optic flow to obtain the deformation matrix. A few points were manually drawn on a CT slice and fourier interpolated to form the initial contours. Initial contours were then deformed into the boundary of objects in the adjacent CT slices based on the deformation matrix until tumors and organs segmented in all slices. In general, deformable models for image segmentation are slow and computationally expensive due to the time necessary to conduct parameter optimization.

The non-parametric technique, mean shift algorithm [2], has been developed for clustering and segmentation problems. Recently, using mean shift update in the constrained normal subspace to find the locally defined principal curve/surface is proposed by Erdogmus and Ozertem [3], [6]. Principal curves can represent an object boundary by finding the ridges of the underlying distribution. In our previous work, a kernel density estimate (KDE)-based principal surface algorithm was proposed for volumetric segmentation and contour propagation of tumors or organs between 3D phases of a four-dimensional computed tomography (4D-CT) dataset without performing full blown 3D-3D deformation registration [9].

In this paper, we propose a semi-automatic method to segment organs using the principal curve algorithm. This method keeps the users interactive in the segmentation procedure by incorporating a few slices delineated by the users in the reference slices, and the initial contours from the users are improved during automatic segmentation process. The segmented contours are formed by a finite number of joint fragments connected with a small number of points. The boundary of organs can be accurately made up of a finite number of linear segments connected with only a small number of points, at the lowest 10 percent of the original manual contour points preserved. Therefore, the computational complexity and workload for delineating contours are reduced.

II. METHODOLOGY

Fig. 1 illustrates the flowchart of our method. Labeled contour points from one or more given slices are projected to the principal curve of the kernel interpolation of the edge distribution. If there exists sparse areas in the output of the principal curve projection, points are added in these areas to do the projection repeatedly until a satisfactory number of samples are obtained from the boundary. Those projected principal curve points on the given slices are then propagated to slices above and below the given slices. A principal curve score for each pair of projected points on all the slices is computed. This score indicates the similarity and consistency of the projected principal curve samples. Using one of the projected points as the starting point, the next point connecting to the starting point is the furthest one in Euclidean distance with the principal curve score above a threshold. In this case, the boundary of objects can be approximated using piecewise linear segments connected with a small number of down sampled projected points while with the shape accuracy preserved.

Fig. 1.

Flowchart of the method

A. Principal Curve Projection

Locally defined principal curves and surfaces presented by Erdogmus and Ozertem [3], [6], [7] are obtained utilizing local first and second order derivatives of the at least twice differentiable underlying density function. Here, the underlying density function is the kernel interpolation of the edge distribution over space, and points of the principal curve are on the ridge of this edge distribution.

Let I be the image, ${\{{\mathbf{p}}_i\}}_{i=1}^N$ be the pixel locations of the image, where p_i ∈ ℝⁿ, I(p_i) is the intensity at that location, and E(p_i) is the edge image. The edge maps can be obtained via calculating the magnitude of the gradient field of the image. E(p)_i = ‖∇I(p_i)‖. The kernel interpolation of the edge map is given as $f(\mathbf{p})={\sum }_{i=1}^{N}w({\mathbf{p}}_i)k_{{\mathbf{\Sigma }}_i}(\mathbf{p}-{\mathbf{p}}_i)$ The weights w_i are the normalization factor of the edge map, $w_i=\frac{\mathbf{E}({\mathbf{p}}_i)}{{\sum }_{i=1}^N\mathbf{E}({\mathbf{p}}_i)}$ Here we use a fixed-width isotropic Gaussian kernel for simplicity. Σ_i is the covariance of the Gaussian kernel $k_{{\mathbf{\Sigma }}_i}(\mathbf{p})=C_{{\mathbf{\Sigma }}_i}{e}^{-\frac{1}{2}(\mathbf{p}-{\mathbf{p}}_i)T}{\mathbf{\Sigma }}_i^{-1}(\mathbf{p}-{\mathbf{p}}_i)$. The gradient and the Hessian of the kernel interpolation are $\mathbf{g}(\mathbf{p})=-{\sum }_{i=1}^{N}w({\mathbf{p}}_i)c_i{\mathbf{u}}_i,\mathbf{H}(\mathbf{p})={\sum }_{i=1}^{N}w({\mathbf{p}}_i)c_i({\mathbf{u}}_i{\mathbf{u}}_i^T-{\mathbf{\Sigma }}_i^{-1})․c_i=k_{{\mathbf{\Sigma }}_i}(\mathbf{p}-{\mathbf{p}}_i),{\mathbf{u}}_i={\mathbf{\Sigma }}_i^{-1}(\mathbf{p}-{\mathbf{p}}_i)$. The inverse of local covariance of the Gaussian density is defined as C(p) = Σ⁻¹(p) = −H_logf(p) (P) = −f (p)⁻¹H(p) + f (p)⁻²g(p)g^T (p). Principal curve projections follow the linear trajectories formed by the eigenvectors of the Gaussian’s covariance inverse matrix. ((λ₁(p), v₁(p)), …, (λ_n(p), v_n(p)) are the eigenvalue - eigenvector pairs of C(p), where the eigenvalues are sorted such that λ₁(p) < λ₂(p) < … < λ_n(p) and λ_i ≠ 0.

A point is on the d dimensional principal curve iff the local gradient is in the span of d eigenvectors of the local covariance inverse and the corresponding (n − d) eigenvalues are positive. If the corresponding (n − d) eigenvalues are negative, the point is on the d dimensional minor curve. The eigen decomposition of ${\mathbf{C}}_{\perp }(\mathbf{p})={\mathbf{V}}_{\perp }{\mathbf{\Gamma }}_{\perp }{\mathbf{V}}_{\perp }^T$, where V_⊥(p) = [v_d+1(p)…v_n(p)] is the (n − d) largest eigenvectors of C(p), and Γ_⊥ = diag(λ_d+1(p), …, λ_n(p)). Then, x is iteratively forced to converge to the principal curve in the constrained space ${\mathbf{V}}_{\perp }{\mathbf{V}}_{\perp }^T\mathbf{m}(\mathbf{p})$ through the subspace mean-shift update, where $\mathbf{m}(\mathbf{x})={({\sum }_{i=1}^N{k}_{{\mathbf{\Sigma }}_i}(\mathbf{x}-{\mathbf{x}}_i)\mathbf{C}(\mathbf{p}))}^{-1}{\sum }_{i=1}^N{k}_{{\mathbf{\Sigma }}_i}(\mathbf{x}-{\mathbf{x}}_i)\mathbf{C}(\mathbf{p}){\mathbf{x}}_i$. If the gradient is orthogonal to the subspace spanned by the selected (n − d) eigenvectors when projecting the data from n to d dimensions, the mean-shift iterations stop. The stopping measure is defined as $\gamma (\mathbf{p})=\frac{\mathbf{g}{(\mathbf{p})}^T{\mathbf{C}}_{\perp }(\mathbf{p})\mathbf{g}(\mathbf{p})}{\Vert \mathbf{C}(\mathbf{p})\mathbf{g}(\mathbf{p})\Vert \Vert \mathbf{g}(\mathbf{p})\Vert }$. If the stopping measure reaches 0, the point is on the principal curve. γ(p) is positive around the principal curve regions, since all the eigenvalues of C_⊥(p) are positive, and negative around the minor curve. Due to the normalization term, γ(p) is bounded between [−1, 1].

B. Principal Curve Score

The outputs of the principal curve projection are the sampling points on the ridge of kernel interpolation of the edge distribution. In order to down sample the points and connect these down sampled points to represent the shape of organs, we define a pairwise principal curve score to check whether or not two points belong to the same ridge. Nearby samples in a neighborhood can then be separated based on their underlying ridges for the purpose of downsampling.

If ℘(a, b) is the pairwise principal curve score between points a and b, then the line integral of the scalar valued function, γ(․), from a to b evaluated on the curve l(t) and the arc length of the curve L, is $\wp (a,b)=\frac{{\int }_0^1\gamma (\mathbf{l}(t)){[{\mathbf{l̇}}^T(t)\mathbf{l̇}(t)]}^{\frac{1}{2}}dt}{L(\mathbf{a},\mathbf{b})}$. Here we parameterize l(t) = a+t(b−a) as a line with l(0) = a, l(1) = b and l̇(t) = (b − a), and $L={\int }_0^1{[{\mathbf{l̇}}^T(t)\mathbf{l̇}(t)]}^{\frac{1}{2}}dt$. If two points are on the same curve, then the principal curve score between these two points is relatively low. Conversely, if two points are on different curves, then the score is relatively high.

Since γ(p) will attain positive values in a convex region around the principal curve and negative values around the minor curves, the principal curve score is only calculated between two points where the connection of these two points lies inside a local convex region around the ridge such that $\mathrm{\wp ̄}(\mathbf{a},\mathbf{b})=\{\begin{array}{cc}\wp (\mathbf{a},\mathbf{b})\hfill & \text{if}\forall t\in [0,1]{\lambda }_{d+1,\dots ,n}(\mathbf{l}(t))>0\hfill \\ \infty \hfill & \text{otherwise}\hfill \end{array}$

C. Sampling on Principal Curve

Once the local connectivities between two projected principal curve points are all obtained, principal curve points can be down sampled to approximate the shape with piecewise linear lines by varying the threshold, thr. The deviation from the original curve is defined as $\overline{{\wp }^{*}}(\mathbf{a},\mathbf{b})=\mathit{\text{max}}(\mathrm{\wp ̄}(\mathbf{a},\mathbf{b}),\mathrm{\wp ̄}(\mathbf{b},\mathbf{a}))$. Given the threshold, thr, the pre-defined condition, $\overline{{\wp }^{*}}(\mathbf{a},\mathbf{b})<thr$, the projected principal curve points, p₁, p₂, …, p_N, and a starting reference point p_ref, where ref is the index, Table 1 shows the procedure of down sampling points on the principal curves.

TABLE I.

Down sampling on the principal curves

Select a starting reference point pref Find the furthest point pref+ on the same ridge as pref in the consecutively increasing index side of ref with $\overline{{\wp }^{*}}({\mathbf{p}}_{\mathit{\text{ref}}},{\mathbf{p}}_{\mathit{\text{ref}}+})<thr$, and the furthest point pref− on the same ridge as pref in the consecutively decreasing index side of ref with $\overline{{\wp }^{*}}({\mathbf{p}}_{\mathit{\text{ref}}},{\mathbf{p}}_{\mathit{\text{ref}}-})<thr$. Starting from pref+, repeat step 2 in the consecutively increasing index side until the largest index point satisfying the pre-defined condition is achieved Starting from pref−, repeat step 2 in the consecutively decreasing index side until the smallest index point satisfying the pre-defined condition is achieved

III. EXPERIMENTS

We test the proposed algorithm on 2 patient lung 3D-CT scans for lung segmentation and compare with the manually delineated contours. All ROIs are manually delineated by physicians as reference for evaluation. The segmentation performance are evaluated qualitatively and quantitatively.

A. Principal Curve Projection for Lung Segmentation

Fig. 2 shows the results of the principal curve projections on one slice. The blue lines are the edges, the red lines are the projected principal curves, and the yellow lines are the manual contours. The projected principal curve is faithfully able to recapitulate manually drawn lung boundaries.

Fig. 2.

The principal curve projection to the lung boundary: edge points (blue), the projected principal curve (red), the manual contours (yellow)

The initial contours may not converge to the low edge density regions or the boundary concavities if the kernel width is too small or the distance of the initial curve is far from the ridges. Therefore, if the Euclidean distance between adjacent points is above a threshold, then additional points are generated between these two points to be projected to the ridges again. This interpolation procedure is repeated iteratively until the distance is below the threshold. Fig. 3 shows the effect of up sampling points on the ridges if the output of the principal curve projection is not satisfied. The yellow lines are the manually delineated contours, the blue lines are the edge, the green lines are the output of the principal curve projection, and the red lines are the up sampled points along the boundary concavities. This greatly improves the conformity to the object boundary and the overlapping ratio between the manual contours and our segmented contours.

Fig. 3.

Up sampling points to the ridges of the object boundaries: edge points (blue), principal curve projection (green), up sampled points (red)

B. Propagation of Contours

Since there are no significant shape and position changes of organs between adjacent slices, the output of the automatic segmentation from one slice provides an ideal initialization for the propagation of the contours to the adjacent slices. If repeatedly propagating contours between neighboring slices through the entire slices of a 3D-CT scan, a complete set of contours will be segmented. Fig. 4 shows the propagation of contours between slices. The slice in the middle is the reference slice, and the contours are manually delineated, yellow lines. The manual contours are used as an initialization to segment lung contours in the reference slice. The output of the automatic segmentation, red lines in the reference slice is used as an initialization for the propagation of contours to the slices above and below. The images at the left side are the slices below the reference slice, and the images at the right side are the slices above the reference slice. White lines are the output of the automatic segmentation from the previous slice and also the initial contours for the current slice.

Fig. 4.

Propagation of contours between slices: manual contours (yellow), the projected principal curves (red), initial contours and the output of the segmentation from the previous slice (white)

C. Down sampling of the Principal Curves

Fig.5 shows the results of down sampled points representing the lung shape. More points are preserved at high curvature regions and few points are retained at the smooth areas. The promising results demonstrate the feasibility of representing the shape using only a small number of pre-served points.

Fig. 5.

down sampling points on the ridges of the object boundaries: edge points (blue), principal curve projection (red), manual contours (yellow), down sampled points (green circle) with black lines connecting them

Fig. 6 visualizes the segmentation results by those preserved points in transversal, sagittal, and coronal views, as well as 3D reconstructed lung surfaces in ITK-SNAP. This results show the accuracy and robustness of using a small number of points to represent the shape model.

Fig. 6.

Segmentation results shown in red shades viewed in Transversal, sagittal, coronal views and 3D reconstructed lung surfaces

D. Quantitative Evaluation

We use the Dice coefficient to quantify the overlap between the manually drawn contours and those determined by our automatically segmented contours. $d=2\frac{|A\cap B|}{|A|+|B|}$. A and B indicate the volume of objects. The output of our algorithm is the 2D coordinates of the object boundaries. A binary 3D volume mask based on the 2D coordinates of the object boundaries from all the slices should first be created in order to calculate the Dice coefficients.

Fig. 7 displays the dice coefficient between the contours represented by the down sampled principal curve points and the manual contours and the fraction of preserved points to the original principal curve points by varying the values of compression parameters with different patients. The Dice coefficients are around 0.94 and 0.93 for patient 1 and 2 respectively. The preserved number of points can be as low as 10 percent of the manual contours points.

Fig. 7.

Dice coefficient and the fraction of preserved points by varying thr values with different patients

IV. CONCLUSIONS AND FUTURE WORK

In this paper, we propose a semi-supervised segmentation algorithm to represent contours of organs with a small number of points given a few slices by the clinicians. The algorithm identifies principal curve samples from the ridge of the kernel interpolation of the edge distribution and then down samples the ridge samples by checking whether or not the samples belong to the same ridge based on the pairwise principal curve score. This method is not computationally expensive and time-consuming and does not require extensive parameter optimization. This method speeds up manual delineation and allows clinicians to intervene in the initialization and have control over the solutions during the process. The proposed method is tested on 2 patient 3D-CT datasets. Quantitative and qualitative experimental results demonstrate that our semi-automatic segmentation method produces acceptable segmentation accuracy. The algorithm generally performs well on the lungs because of the clear boundaries detected. However, when tumors are on the walls of organs or have similar intensity and texture as the surrounding soft tissues, or if some organs have low contrast against the background, the edge of objects can be occluded or hard to discern. In the future, we will incorporate prior shape information to make the results more accurate and robust in case of occluded edges. We will also incorporate different delineation initializations from multiple observers, and evaluate our algorithm in more datasets and other organs, such as the heart, the kidney, the liver, and the prostate.