Automatic Liver Contouring for Radiotherapy Treatment Planning

Abstract

Purpose:

To develop automatic and efficient liver contouring software for planning 3D CT and four dimension computed tomography (4D-CT) for application in clinical radiation therapy treatment planning systems.

Methods:

The algorithm is comprised of three steps for overcoming the challenge of similar intensities between liver region and its surrounding tissues. First, the total variation model with L1 norm (TV-L1) which has the characteristic of multi-scale decomposition and edge preserving property is used for removing the surrounding muscles and tissues. Second, an improved level set model which contains both global and local energy functions is utilized to extract liver contour information sequentially. In the global energy function, local correlation coefficient (LCC) is constructed based on the gray level co-occurrence matrix (GLCM) both of the initial liver region and the background region. LCC can calculate the correlation of a pixel with the foreground and background region respectively. LCC is combined with intensity distribution models to classify pixels during evolutionary process of level set based method. The obtained liver contour is used as the candidate liver region for the following step. In the third step, voxel-based texture characterization is employed for refining the liver region and obtaining the final liver contours.

Results:

The proposed method has been validated on planning CT images of a group of 25 patients undergoing radiation therapy treatment planning. These included 10 lung cancer patients with normal appearing livers and 10 patients with hepatocellular carcinoma or liver metastases. The method was also tested on abdominal 4D-CT images of a group of 5 patients with hepatocellular carcinoma or liver metastases. The false positive volume percentage, false negative volume percentage and dice similarity coefficient between liver contours obtained by the developed algorithm and the current standard which is delineated by the expert group are on an average 2.15%−2.57%, 2.96%−3.23%, 91.01%−97.21% for CT images with normal appearing livers, and 2.28%−3.62%, 3.15%−4.33%, 86.14%−93.53% for CT images with hepatocellular carcinoma or liver metastases, and 2.37%−3.96%, 3.25%−4.57%, 82.23%−89.44% for 4D-CT images also with hepatocellular carcinoma or liver metastases, respectively.

Conclusion:

The proposed three step method can achieve efficient automatic liver contouring for planning CT and 4D-CT images with following treatment planning and should find widespread application in future treatment planning systems.

1. INTRODUCTION

Currently in radiation therapy treatment planning (RTTP) systems, large amounts of data from cancer and normal tissues are obtained from different modality imaging systems such as computed tomography (CT), positron emission tomography (PET), magnetic resonance image (MRI), these images are used for better tumor localization or normal tissue identification within critical regions, and this is very helpful for efficient RTTP^1–2. In current RTTP systems, the tumor target volume and infield critical organs inside the region of interest (ROI) need to be contoured accurately for radiation dose calculation and treatment planning implementation³. Currently, this work is done manually by experienced radiation oncologists or physicists, and is very time consuming and subjective with inter-observer variability. Specially, with the use of image guided precise adaptive radiation therapy, large numbers of daily images are collected before, during the treatment process, or even afterwards. For example, daily cone beam CT (CBCT) images containing on-board information of the patients during radiotherapy treatment⁴, and four dimensional CT (4D-CT) images containing the respiratory characteristics of patients⁵, are required to provide information for tumor or critical organ location during the stereotactic body radiation therapy (SBRT) process. This is in order to reduce the uncertainty of the radiation therapy process⁶. The tumor volume and organ at risk (OARs) inside the ROI need to be contoured accurately for a quantitative analysis, such as GTV (Gross Tumor Volume) with respiratory movement and following dose evaluation.

The liver, which is one of the important dose limiting organs for the treatment of cancers within thoracic and abdominal regions⁷, is always delineated manually by the experienced radiation oncologist and physicists. Currently, the manual methods used by the experienced physicians incorporate their prior knowledge into the contouring process. However, with the dramatically increased numbers of images, obtaining images by delineating liver contours image by image manually is time consuming during radiotherapy treatment planning process⁸. Especially with the emergence of daily CBCT and 4D-CT images used in the clinic for hepatic carcinoma radiation therapy treatment planning^9–10, oncologists need quantitative analysis of the liver movement caused by normal free breathing. Accurate contouring of the liver region is the first important step. In these cases, it becomes impractical to use manual contours obtained in a slice by slice manner. Furthermore, CT images are also frequently used in computer aided liver cancer detection and treatment such as image guided radiation therapy (IGRT)^11–12, surgery planning for tumor resection¹³, liver volume measurement¹⁴ etc. Liver contouring using an automatic or semi-automatic image processing algorithm from CT images is a fundamental step with these studies¹⁵. However, it is a challenging task to contour liver region accurately and automatically from CT images because of the fuzzy liver boundary posed by the similar intensity distribution between the liver and its surrounding tissues and organs (e.g. heart, kidney and spleen), the high variation in liver shape and size among different patients^16–17, and artifact introduced by liver motion.

Some automatic and semi-automatic liver contouring methods have been proposed for various clinical purposes¹⁸. For example, a threshold based method has been applied to liver segmentation by analyzing the histogram of CT images to determine the thresholds and then edge. However, the segmented results commonly include the irrelevant tissues. The spatial information is not included in the threshold based method. Leinders et al. used the threshold based method to remove the ribs, spine and kidneys and then refined the liver region results by an adaptive fast marching method¹⁹. Region growing based method has been used employing the similar gray values of close voxels, but it can lead to over segmentation between the liver and its surrounding tissues. In the region growing based segmentation method, the segmentation process starts from the seeds selected inside the liver organ and grows into the entire liver region by utilizing the similarity criterion. Lu et al. selected seed points by using Quasi-Monte Carlo method, and improved the region growing algorithm²⁰.

More recently, active contour models have been frequently used to achieve automatic liver contouring or segmentation^21–22. These included snake model²³ and the level set based model^24–25. In these active contouring models, the initial contour always moved to the liver boundary which was driven by the energy function defined based on the features of images²⁶. These methods were sensitive to the initial contour. Liu et al.²³ utilized gradient vector flow (GVF) snake to delineate the liver contours automatically on contrast enhanced CT images. However, when the method was employed to the low-contrast CT images, the results would contain false boundaries in many cases. Lee et al.²⁴ developed the two step seeded region growing to detect initial shape from level set speed functions. This improved computational efficiency of the level set based method, but it also only delineated liver region from contrast enhanced CT images accurately. A graph based liver segmentation algorithm was a user-steered segmentation technique, wherein CT images were treated as weighted and undirected graphs, and pixels were regarded as vertexes. Neighboring vertexes were connected by graph edges, and then their weights measured the cost of two connected vertexes. However, this method required user interaction and the segmented results were highly affected by user skills. Beichel et al.¹⁶ used graph cuts to generate an initial liver contour and refined it by utilizing the volume mesh based segmentation approaches. The gray-level based method was introduced to perform liver segmentation automatically. The basic step was to estimate the liver gray levels by statistical analysis or histogram analysis of manually segmented CT image slices. Lim et al.¹⁴ analyzed the intensity distribution of several manually segmented liver and muscle, and then exploited multilevel thresholds based on statistical information to eliminate other organs and tissues. Furthermore, mathematical morphology filtering and K-means algorithm were performed to acquire the initial liver boundary. Finally, active contour models were applied to smooth the liver boundary. However, the statistical information was difficult to contain the high variability with different CT images.

Automatic liver segmentation is rather difficult especially for low-contrast CT images when the algorithm is purely utilizing gradient and intensity information within images. Therefore, model based methods has been proposed to improve the segmentation performance. These include probabilistic atlas based methods and statistical shape models (SSM) based techniques²⁷, which provide the anatomical information about size, shape and position of liver regions in CT images. Okada et al.²⁸ used a probabilistic atlas to obtain an initial liver region for SSM fitting CT images. This took advantage of principle component analysis (PCA) to describe and capture the shape. Wang et al.²⁹ developed sparse shape representation to address the complex shape variation for improving segmentation performance. The model needed to contain all possible liver shapes. In the probabilistic atlas based method³⁰, the probability of every pixel was the spatially average of the registered training sets. These were segmented manually as a priori knowledge, and then they were incorporated into the Bayesian framework. The pixels with maximal posteriori probability were regarded as the liver region which was extracted by an optimization process. However, in atlas based method the segmentation performance depends heavily on the accuracy of the registration process³¹.

Due to similar regional tissue intensities, multiple shapes and great differences in size of livers acquired from different patients or different time, it is difficult for a single method to solve the different segmentation problems completely. A combination of methods can take advantage of individual methods and achieve an efficient segmentation process³². In this study, a three-step liver segmentation algorithm is presented for automatic liver contouring. This can be used for both planning 3D CT and 4D-CT images for radiation therapy treatment planning. First, edge preserving TV-L1 model is used to reduce the related organs and then extract initial liver region. It aims to reduce the risk of error segmentation between the liver and its surrounding structures. The TV-L1 model can also provide abundant contour information for the edge based energy function in the following level set based model. In the second step, LCC, which contains the correlation information of pixels with the initial liver region and the background region, is incorporated into an improved level set based framework. The global intensity distribution and local edges information are included for overcoming the high variation of livers among different patients. We can obtain the candidate region of liver from original target CT images. Finally, texture analysis is employed to eliminate the incorrectly classified pixels including the abnormal region within the candidate liver in order to obtain the final liver contour.

2. METHODS AND MATERIALS

2. A. Platform for developing the software

The proposed liver contouring algorithm is based on two open source code platforms Insight Toolkit (ITK) and the Visualization Toolkit (VTK). These two toolkits with open source code were developed under the sponsorship of National Library of Medicine. The ITK software developing platform has many popular segmentation and registration algorithms. It is very efficient and useful for researchers to use in developing their own programs. The VTK open source platform is mainly used for the experiment results visualization during medical image processing. In this study, the developed software runs on a personal computer (PC). The details of PC are Intel Core (TM) i5-4300M, 2.60 GHz with dual cores, 8G memory, windows 7 with 64-bit operation system.

2. B. Clinical data studied

We used CT images of 25 patients (mean age 59 years) to evaluate the effectiveness of the proposed method. The data were from 10 lung cancer patients with normal appearing livers, and 10 patients with hepatocellular carcinoma (5 patients) or liver metastases (5 patients). By the normal appearing livers we meant that the livers in these patients appeared normal on CT scans based on the judgment of clinical doctors. An additional 5 patients with hepatocellular carcinoma (3 patients) or liver metastases (2 patients), all of whom had 4D-CT scans, were also used evaluated to effectiveness of the proposed method. The reconstructed dimension/size for each trans-axial slice of the CT images was 512×512 voxels with a voxel size of $1.17\times 1.17\times 3{\text{mm}}^3$. The scanning thickness was 3mm, and the resolution was 0.931 pixels per mm. 4D-CT images were obtained during SBRT treatment. The reconstructed dimension/size for each trans-axial slice was 512×512 voxels with a voxel size of $0.84\times 0.84\times 3{\text{mm}}^3$. The scanning thickness was 3 mm. The resolution was 1.193 pixels per mm. Two experienced radiation therapy oncologists delineated the liver region on CT slices and the results are checked and reviewed by a third physician to ensure the accuracy of the manually delineated results. This delineation of the liver was taken as current standards for comparison with the automated contouring methods. CT images of additional 10 patients with normal appearing livers are employed to train the optimal $\lambda$ value of the TV-L1 model to extract the initial liver region from original images.

In our study, we utilized false positive volume percentage $(\mathit{\text{FPVP}})$, false negative volume percentage $(\mathit{\text{FNVP}})$ and the dice similarity coefficient $(DSC)$ for quantitative comparison, and we compared the performance of the proposed algorithm with the popular level set based method and active contour model to validate the effectiveness of the proposed method. $\mathit{\text{FPVP}}$ is defined as $\mathit{\text{FPVP}}=\left|L_P-L_G\right|/\left|{\text{L}}_{\text{G}}\right|$, where $\left|L_P-L_G\right|$ is defined as the number of pixels obtained by the automatic algorithm and not included in delineated result pixels $L_G$ by the expert group. $\mathit{\text{FNVP}}$ is defined by $\mathit{\text{FNVP}}=\left|N{L}_P-N{L}_G\right|/\left|N{L}_G\right|$, where $\left|N{L}_P-N{L}_G\right|$ is described as the total number of manually delineated pixels, which are not included in the automatic algorithm result voxels $N{L}_G.DSC$ is defined as $DSC=2\left(L_P\cap L_G\right)/(\left(L_P\cap L_G\right)+\left(L_P\cup L_G\right))$, which is to measure how much the automatic segmented results $L_P$ agree with the manual delineated results $L_G$. Furthermore, average ± standard deviation $(\text{A}\pm \text{SD})$ was calculated from contouring volumes by different methods.

2. C. Overview of the automatic contouring process

Figure 1 shows the flowchart of our proposed automatic liver contouring algorithm. For given incoming planning 3D-CT or 4D-CT images, the liver region is contoured in three steps. In the first step, as in Figure 1 (a), TV-L1 model is applied to extract the initial liver contour, and then the initial liver contour divides the original image into two parts. In the second step, an improved level set based method is used to segment candidate region which is gained by the first step. In the third step, texture analysis is exploited to eliminate the abnormal liver regions to obtain the final liver region. During the second step we improved the level set based method for efficient automatic liver segmentation as shown in Figure 1 (b). We use finite Gaussian model and mixture Gaussian model to estimate the intensity distribution of abdomen region. LCC is defined to reduce the misclassified pixels with the distribution models in the energy function for overcoming the disadvantage of Gaussian models.

Figure 1:

Flow chart of the proposed hybrid three-step liver contouring framework. Figure 1 (a) is the three steps of the segmentation process, and Figure 1 (b) is the illustration for the improved level set based method within the second step of our proposed method

2. D. Initial liver region extracted by TV-L1 model

On CT images, the liver boundary is sometimes blurry, since the neighboring organs and tissues shares similar intensity. Furthermore, liver shapes are of great variety for different patients. Liver segmentation methods which are purely based upon intensity information may fail to extract accurate liver contours. Aiming to address these challenges and reduce the risk of error segmentation between the liver and its surrounding structures, we exploited the edge preserving multi-scale decomposition property of the TV-L1 model to extract liver contour as the initial target region³³. The TV-L1 model selected edge and contour information of organs from CT images according to their geometric size rather than the intensities. The edge preserving model based on TV-L1 can also remove noise and artifacts which are usually introduced during the nonlinear reconstruction of CT scans. It was able to maintain the ideal contour and edge information of the selected organ. Therefore the obtained initial liver region by TV-L1 model was capable of providing meaningful contour and edge information for the improved level set energy model in the following step. It was more important that the initial liver region was regarded as the starting point of the following level set based method, and this optimized initial contour was contributed to speed up the minimization of the energy function within the level set model and improve the segmented accuracy and robustness. The initial liver contour can divide the original image into target region and background region, and the GLCM was calculated within these two parts to estimate the neighboring resolution cells.

In our method, for each patient we choose a starting CT slice in which the liver region was larger than other surrounding organs and tissues. For clinical RTTP system this starting slice can be chosen manually by physician or other treatment planning designer in clinic, which will make it easy to extract the first slice of initial liver region by TV-L1 model. Furthermore, in other slices within the same volume we needn’t implement the TV-L1 processing, because we can use contouring results from neighbor CT slices as the initial contour for the following contouring steps. Let $\Omega$ be the entire template CT image domain, and $\mathfrak{R}$ is the real component. Let $u_0:\Omega \to \mathfrak{R}$ is the given CT images. The (x, y) is the coordinate of pixels in the image domain. The contour was selected by solving the TV-L1 model which was formulated as following.

$$E(u,\lambda )=\underset{u}{\text{min}}\underset{\Omega }{\int }|\nabla u(x,y)|\mathit{\text{dxdy}}+\lambda \left|u_0(x,y)-u(x,y)\right|\mathit{\text{dxdy}}$$ (1)

Based on our previous work^{11, 34}, it have been proven that minimization processing for the TV-L1 energy function is equivalent to solving the level set based geometrical problem. We can get known the geometric multi-scale decomposition property which was capable of selecting contours and edges from different organs within medical image according to their geometric size rather than gray intensities by the merit of edge preserving property. The original image $u_0$ was decomposed in specific scale by adjusting the parameter $\lambda$ and the decomposed original image on scale $\lambda$ were gained by solving the formulation $u_0=u(\lambda )+v(\lambda )$, where $u(\lambda )$ preserved the edge and contour information of original image at scale $\lambda$, and $v(\lambda )$ was the excluded details or noise at corresponding scale. In this study, $u(\lambda )$ mainly contained the initial liver region, and in Figure 2 the liver region which was extracted by TV-L1 model on different scale $\lambda$ was shown. In an attempt to improve the robustness and accuracy of the segmentation algorithm, a verification test was designed to decide the optimal parameter $\lambda$ of the TV-L1 model for obtaining the initial liver contour from incoming CT images. More details about the edge-preserving multi scale decomposition (both of 2D and 3D) property can be found in our previously work^{11, 34}.

Figure 2:

The liver region obtained by TV-L1 multi-scale decomposition model from the template CT image. Images from ‘a’ to ‘e are the original image, output images by using $\lambda =0.20,$ 0.15, 0.10, 0.05 respectively

2. E. Parameter estimation with TV-L1 model

In our study, we used the area overlap measure (AOM) to validate the optimal value of $\lambda$ in TV-L1 model, which attempt to improve the automation and robustness of the liver segmentation algorithm. AOM was defined as $\text{AOM}\left({\Omega }_1,{\Omega }_2\right)=2\left({\Omega }_1\cap {\Omega }_2\right)/\left({\Omega }_1\cup {\Omega }_2\right)$, where ${\Omega }_1$ and ${\Omega }_2$ are the region area. First, a group of 10 CT images with normal appearing liver regions are used as initial images to extract liver regions on different scales by different $\lambda$ using TV-L1 model. Then geodesic active contours (GAC) are exploited to track the boundaries of the obtained liver region on different scales. GAC is also obtained from ITK code library. Furthermore, AOM is calculated on different scales. Then we compared the AOM results to determine the optimal $\lambda$ value for extraction the initial liver region. During this process, ${\Omega }_1$ is the liver region area contoured automatically by the GAC on different scale based on initial liver region extracted by TV-L1 model, while ${\Omega }_2$ is the liver region area delineated manually by the experts group as reference image. When the value of $\text{AOM}\left({\Omega }_1,{\Omega }_2\right)$ is 1, there is perfect segmentation from target volume by the TV-L1 model.

In our study, a range of $\lambda$ was used to determine the average value of $\text{AOM}\left({\Omega }_1,{\Omega }_2\right)$. According to Figure 3, when $\lambda =0.125$, the segmentation result has the highest AOM value in our experiment. In our following studies, we let $\lambda =0.125$ within the TV-L1 model, and then it was used to extract the liver region from CT images.

Figure 3:

Performance evaluation using AOM method by different $\lambda$ value.

2. F. Candidate liver region detected by improved level set based method

Due to the high variation of liver shapes and sizes across different patients which is caused by age, sex, height, even with or without liver tumor, the liver region always share similar intensity with other organs and tissues, and it is hard to delineate liver contour directly and automatically especially with the tumor existing within the patient CT images. So in this next step, we utilized level set based method to delineate the liver contour alongside the whole liver outline, which is regarded as candidate liver region. Specially, if the liver region within patient CT images is normal appearing, we can get the final contour from this step. If not, the texture features based method in the last step are used to classify the tumor region and normal liver region within the candidate liver and obtain the final liver contour for radiation therapy planning.

In the level set based method the flexibility of curve evolution is used to track the contour information and has advantages to handle complex topological changes. Level set based method is able to reduce the segmentation error induced by the high clinical variation in shape and size. In level set based formulation, a closed curve C as the zero level set was embedded into the level set function $\varphi (x,y,t)$ and divided the image domain $\Omega$ into two parts: inside(C) and outside(C)²⁶, which are represented as foreground and background respectively. During the evolution process, zero level set curve moves towards the boundary forced by the energy function.

In this work, a novel energy function based upon global intensity distribution and local contour information of CT images was used to track the liver contour alongside the whole liver region. Here we assumed that intensity values of the liver region in CT images can be modeled as a finite Gaussian distribution. Actually, we do not know the exact number of regions within background of the patient CT images. However, intensity distribution of the background was divided into two parts by intensity distribution of the liver region. We used the distribution descriptors in global energy function to extract the liver contour information, where a Gaussian model $e_1(x,y)$ was applied to represent the liver region and a Gaussian mixture model $e_2(x,y)$ within two classes was exploited to describe the background information³⁵. Edge based energy function $E_e(\varphi )$ utilizing the local edge information was combined with the global energy function to further improve segmentation accuracy.

In the intensity distribution descriptors, higher probability density of each pixel within the foreground and background distribution energy functions determined the label class. It is hard to classify pixels which have the same probability density values in these two energy function terms. Furthermore, the liver region always shared similar intensity values with its surrounding tissues and organs which can cause misclassification. The misclassified pixels were mainly those liver pixels which had higher probability density within the background distribution of the energy function, and other pixels within background which had higher probability density in the foreground distribution energy function term. To reduce the uncertainty of the misclassification of pixels, we defined the local correlation coefficient (LCC) which calculated the correlation of pixels with target region and background region respectively, and it was embedded into the distribution descriptors.

In this study, LCC was constructed by utilizing the gray level co-occurrence matrix (GLCM) which contained the texture information of the distance and angular relationship between the neighboring resolution cells³⁶. In order to avoid huge computation caused by many gray levels, the range of CT image gray value was scaled to 256 gray levels covering the range of −1000 to 1000 HU. To obtain the statistical information about the gray level of the target and background region respectively, we calculated the $P_1\left(i,j,d_{\beta },\theta \right)$ within the initial liver region and $P_2\left(i,j,d_{\beta },\theta \right)$ within the background region to ensure that the correlation was proportional to the probability of the pixels which belonged to the liver region or belonged to the background region. This property can be used to reduce the misclassified pixels within the Gaussian model. The GLCM was obtained by calculating $P_r\left(i,j,d_{\beta },\theta \right)(r=\mathrm{1,2})$, and then it was normalized by the normalizing constant ${\text{R}}_{{\theta }_{\text{r}}}$³⁶. We got $P_r\left(i,j,d_{\beta },\theta \right)=P_r\left(i,j,d_{\beta },\theta \right)/R_{{\theta }_r}$, where $0\le P_r\left(i,j,d_{\beta },\theta \right)\le 1$ was the probability of the simultaneous occurrence of two pixels. Then, the symmetric property of the normalized GLCM was used to simplify the process of calculating the LCC by letting $P_r\left(i,j,d_{\beta },\theta \right)=P_r\left(i,j,d_{\beta },\theta \right)+P_r\left(j,i,d_{\beta },\theta \right)$, when $i\ne j$. Finally in the LCC, four directions for $p_r(x,y)$ of each pixel within its neighboring pixels was $\theta =0°,45°,90°,135°$ and the distance was $d_{\beta }=\mathrm{1,4}$ and $\beta =\mathrm{1,2}$. This captured both of the near and far repeated pairs for pixel based texture analysis. $p_1(x,y)$ estimated the correlation of pixel $(x,y)$ with target region, and $p_2(x,y)$ estimated the correlation of pixel $(x,y)$ with background region. LCC calculated the occurring probability of local neighboring pixels simultaneously to estimate the relation of pixels of both foreground and background region. It can be formulated as follows:

$$p_r(x,y)=\frac{1}{2}\sum _{\beta =1}^2\frac{1}{4}(c_{0^o}{P}_r\left(f(x,y),f(x,y+1),d_{\beta },0^o\right)+c_{{90}^o}{P}_r\left(f(x,y),f(x+1,y),d_{\beta },{90}^o\right)+c_{{45}^o}{P}_r\left(f(x,y),f(x-1,y+1),d_{\beta },{45}^o\right)+c_{{135}^o}{P}_r\left(f(x,y),f(x-1,y-1),d_{\beta },{135}^o\right))$$

Where $c_n\left(n=0°,45°,90°,135°\right)$ is a constant coefficient. When the pixels had no corresponding directional neighboring pixels, $c_n=0$ or $c_n=1$. The calculated probability sets were regarded as LCC which contained the correlation information of CT images. The LCC was embedded into distribution descriptors and we got the energy function as follows:

$$E(\varphi ,{\mu }_t,{\mu }_m^B,{\sigma }_t,{\sigma }_m^B)=-{\lambda }_1{\int }_{\Omega }{p}_1(x,y)e_1(x,y)H(\varphi )\mathit{\text{dxdy}}-{\lambda }_2{\int }_{\Omega }{p}_2(x,y)e_2(x,y)(1-H(\varphi ))\mathit{\text{dxdy}}+E_e(\varphi )+\eta \int p(|\nabla \varphi |)\mathit{\text{dxdy}}$$ (2)

Where:

$$e_1(x,y)=\frac{1}{\sqrt{2\pi }{\sigma }_t}\text{exp}(-\frac{{\left(u_0(x,y)-{\mu }_t\right)}^2}{2{\sigma }_t^2})$$ (2a)

$$e_2(x,y)=\frac{1}{\sqrt{2\pi }{\sigma }_m^B}\text{exp}(-\frac{{\left(u_0(x,y)-{\mu }_m^B\right)}^2}{2{\left({\sigma }_m^B\right)}^2}$$ (2b)

$$E_e(\varphi )=\mu {\int }_{\Omega }g\delta (\varphi )|\nabla \varphi |\mathit{\text{dxdy}}+v{\int }_{\Omega }H(\varphi )\mathit{\text{dxdy}}$$ (2c)

$$p(|\nabla \varphi |)=\left\{\begin{array}{ll}\frac{1}{(2\pi {)}^2}(1-\text{cos}(2\pi |\nabla \varphi |)),& \mathit{\text{if}}|\nabla \varphi |\le 1\\ \frac{1}{(2\pi {)}^2}(|\nabla \varphi |-1{)}^2,& \mathit{\text{if}}|\nabla \varphi |\ge 1\end{array}\right.$$ (2d)

In the formulation above, ${\lambda }_1,{\lambda }_2>0$ and $\mu ,v,\eta \ge 0$ are constant weight parameters. ${\mu }_t,{{\mu }_m}^{\text{B}}$ and ${\sigma }_t,{{\sigma }_m}^{\text{B}}$ are means and standard deviation of foreground and background, respectively. $E_{\text{e}}(\varphi )$ is the edge based energy function. The first term was applied to constrain the zero level curve during its evolution process and minimized the energy function when it lied on object boundaries. The second term is the weighted area of the target region that is used to speed up the zero level contour evolution. The last term of the energy function is the distance regularized term used to maintain the regularity of level set based method during its evolution³⁷. The Heaviside function $H(\varphi )$ and Dirac Delta function $\delta (\varphi )$ are usually replaced for computation by smoothed functions ${\delta }_{\epsilon }(\varphi )$ and ${\text{H}}_{\epsilon }(\varphi )$ defined as follows:

$${\delta }_{\epsilon }(\varphi )=H_{\epsilon }^{\prime }(\varphi )=\left\{\begin{array}{ll}\frac{1}{2\epsilon }[1+\text{cos}(\frac{\pi \varphi }{\epsilon })],& \left|x\right|\le \epsilon \\ 0,& \left|x\right|>\epsilon \end{array}\right.$$ (3)

In our study, the standard gradient descent method³⁵ was used to minimize the energy function $E\left(\varphi ,{\mu }_t,{{\mu }_m}^{\text{B}},{\sigma }_t,{{\sigma }_m}^{\text{B}}\right)$. So the level set function in this study can be formulated as follows:

$$\frac{\partial \varphi }{\partial t}={\delta }_{\epsilon }(\varphi )\left(\mu \cdot div\left(g\frac{\nabla \varphi }{|\nabla \varphi |}\right)+v\cdot g-\left({\lambda }_1{p}_1(x,y)e_1(x,y)-{\lambda }_2{p}_2(x,y)e_2(x,y)\right)\right)+\eta \cdot div\left(d_p(|\nabla \varphi |)\nabla \varphi \right)$$ (4)

Where $d_p\stackrel{\Delta }{=}\frac{p^{\prime }(|\nabla \varphi |)}{|\nabla \varphi |}$

The improved energy function based on the level set concept combined the intensity distribution and correlation of pixels with target and background region to determine the label class. In our study, parameters in our minimization function used the empirical value which was frequently used in other literature^{35, 37}. We let parameters in object region and background equal ${\lambda }_1={\lambda }_2=1$ for our improved level set based method. We let time step $\Delta t=1$ to speed up the curve evolution, and $\eta =0.2$ to satisfy the Courant-Friedrichs-Lewy condition³⁵, and $\eta \cdot \Delta t<1/4$ in the difference equations, and $\mu =5,v=3$ within the method. The parameter $\epsilon$ of smoothed functions ${\delta }_{\epsilon }(\varphi )$ and $H_{\epsilon }(\varphi )$ was 0.5.

2. G. Refinement of the liver region by texture analysis

For cancer RTTP, the whole liver region from hepatocellular carcinoma patients always contains the liver tumor. It is necessary to distinguish the normal liver from the tumor liver or metabolic liver within the whole liver region. Specially, these liver regions usually have similar intensity distribution with its neighboring normal liver region. The proposed level set based method cannot recognize the blurry boundary between the normal liver region and the tumor liver region. The candidate liver region which was contoured by the improved level set based method may also contain tumor region. Considering this possibility, we knew that texture analysis for target image had advantages on soft tissue segmentation by using the texture features. To obtain the internal liver region, we made use of texture features iteratively within the candidate liver region for each pixel to extract the liver region from the abnormal liver region including the liver tumor, and classify these pixels into normal and abnormal components.

The class label for each pixel belonging to normal or abnormal components within the candidate liver region was determined by the texture features of the corresponding pixel. This means that the class label is decided by the majority classes of its surrounding neighbors. For this purpose, a two dimensional window function was defined and centered on each pixel, texture features of which were assigned by the calculated texture features within the window region.

In this strategy, first $P\left(i,j,d_{\beta },\theta \right)$ was calculated within the candidate liver region, so the near and far repeated pairs were captured for texture analysis. Second, the size of the window was set using $l\times l(l=15,13,9,7,5)$ for texture analysis. Considering the Haralick texture descriptors based on GLCM, four independent descriptors including energy, contrast, correlation and entropy were used in proposed method^{36, 38}. These four descriptors were informative enough to describe the texture information of the soft tissues. However, the correlation of the local neighboring pixels had been utilized in the above step, so energy $f_{(\mathit{\text{energy}})}$, contrast $f_{(\mathit{\text{contrast}})}$ and entropy $f_{(\mathit{\text{entropy}})}$ had been calculated within the window region, and then they were assigned on the centered pixels. In the next step, a decision rule was used to determine the class label of the centered pixel. According to our study, if $f_{(\mathit{\text{energy}})}>0.87,f_{(\mathit{\text{contrast}})}<0.42$ and $f_{(\mathit{\text{entropy}})}<0.36$, ninety percentage of pixels in the region were target liver pixels. Therefore the centered pixel was assigned as final liver pixel. Next, we let $l=15$, and the relative larger size was able to detect all the potential normal liver pixels. However, some tumor pixels were possible to be misclassified as the normal liver region, so the window size was reduced for improving the accuracy of the assigned centered pixel and removing the misclassified pixels. From our evaluation, we found that the contour results were the best when the window size was reduced to 7 mm×7 mm. When the size was smaller than 7 mm, the misclassified pixels were increased.

3. EXPERIMENT RESULTS

Both planning 3D CT images and 4D-CT scans were used for measuring the effectiveness of the proposed method. 4D-CT images were currently used to improve the radiation therapy treatment of the chest and abdominal cancer in image guided radiation therapy system by considering the effect of breathing on the shapes and locations of critical organs and tumor targets. Figure 4 showed the comparison results of liver regions segmented by various methods, which were including level set based method (LS), region-based active contour model (ACM), the proposed method (PM), and manually delineation (MD) by expert group. The images in the first line were extracted by the TV-L1 model (TV). The obtained images were used as a starting point in the subsequent step. The second line images showed the final liver regions segmented by a popular level set based method (LS)³⁷. This method utilized the edge and contour information for segmentation. The third line images illustrated the liver regions results segmented by the classic active contour model (ACM) from ITK code library. This method used the regional information to achieve the segmentation process. Shown in the fourth line in the Figure.4 were the liver regions, which combined the edge, region and texture information to track the liver boundary. The last line images showed liver regions delineated manually (MD) by expert group for comparison.

Figure 4:

Experiment results display: image slices with liver contouring using different methods. CT images for normal appearing livers, ‘a’ to ‘e’ were extracted by TV, LS, ACM, PM, MD, respectively. CT Images of liver metastasis, ‘f’ to ‘j’ were extracted by TV, LS, ACM, PM, MD, respectively. CT Images of hepatocellular carcinoma, ‘k’ to ‘o’ were extracted by TV, LS, ACM, PM, MD on 4D-CT image respectively.

For the first column in Figure 4, the contouring results acquired by ACM, PM and MD looked similar to each other on CT image of lung cancer patient with normal appearing liver. However, the PM results were more close to the MD results with smoother edges, and this was time saving when designing the treatment planning, because no further editing was needed by clinical physician. The contouring results were relatively worse by LS method, which was mainly accounting for purely utilizing boundary and contour information of the images, and it was hard for LS method to track the blurred edge. In the second and third columns, the contouring results on planning CT images and 4D-CT images with liver metastasis or hepatocellular carcinoma were shown respectively. Comparing with the contouring results by LS and ACM, the result obtained by PM was still the closest to the results obtained by MD. Specially, in Figures 4 (g) and (h) results with LS and ACM lost some liver region which was shown in red ovals because of the liver metastasis, and they also cannot recognize the edge of liver metastasis shown in gold ovals. Similarly in Figures 4 (l) and (m), results with LS and ACM cannot track the liver region with hepatocellular carcinoma. The reason was that it was difficult to recognize the liver metastasis from liver region purely based on gray intensity and gradient information within image. However, with our proposed method, texture information within image was used to classify the soft tissues containing liver metastasis or hepatocellular carcinoma from liver region and the results were shown in Figure 4 (i) and (n). The results with PM was relatively close to the results by MD, and this was mainly because that the improved level set based method can delineate the outlines of contour with good initial liver region, and then texture analysis was used to classify the soft tissues with hepatocellular carcinoma or liver metastasis for the final contour.

The quantitative results were displayed in the following tables. The liver volumes were given in Table 1 of the contouring results from CT images for 10 lung cancer patients with normal appearing livers. Table 2 summarized the contouring volumes from CT images of 10 patients with hepatocellular carcinoma or liver metastases. Table 3 showed the delineation volumes from 4D-CT images with hepatocellular carcinoma or liver metastases for 5 patients. In Table 3 the contouring results for every patient for the 4D-CT cases with different methods were the average value of 10 phases within one respiratory cycle. From these experiment results, the mean volume segmented by the proposed method was the closest to the manual delineations. The mean volume determined by the classic level set based method was the most different to the manual delineations among those contouring results on average. From Table 1, we can get known that the average values of the contouring volume results were similar to each other, and this implied that the contouring results by LS, ACM, PM and MD were similar on planning CT images with normal appearing livers. However, from Table 2, the average values of the contouring results with four methods were significantly different on planning CT images with hepatocellular carcinoma or liver metastases. Comparing with the average values by both of LS and ACM, the average values with PM was closer to the values by MD. We can get known that the proposed method was more robust when contouring the liver region from CT image with hepatocellular carcinoma or liver metastases. It was mainly accounting for that the proposed method utilized the advantages of three steps to overcome the misclassification problem during liver contouring process. Table3 also showed contouring volume results for 4D-CT images with hepatocellular carcinoma or liver metastases.

Table1:

Contouring volumes (cm³) by different methods from CT images for 10 lung cancer patients with normal appearing livers

Patients	LS	ACM	PM	MD
1	1520.2	1537.5	1573.9	1653.2
2	1632.8	1652.4	1670.2	1733.6
3	1487.6	1465.5	1522.4	1472.8
4	1417.3	1426.3	1403.8	1529.7
5	1374.5	1402.1	1399.3	1332.5
6	1819.4	1854.7	1889.9	1801.3
7	1532.8	1498.9	1515.7	1619.8
8	1414.7	1368.3	1302.6	1428.4
9	1408.1	1452.0	1467.5	1399.5
10	1439.6	1473.8	1493.1	1567.7
A±SD	1504.7±134.5	1513.1±143.4	1523.8±163.6	1553.9±150.4

Table2:

Contouring volumes (cm³) from CT images for 10 patients with hepatocellular carcinoma or liver metastases

Patients	LS	ACM	PM	MD
1	797.4	833.8	1129.8	1000.5
2	1018.2	1029.6	1157.2	1201.3
3	1327.3	1378.3	1592.4	1511.4
4	1200.1	1162.7	1427.6	1332.8
5	953.5	924.8	1219.7	1126.7
6	1098.6	1134.7	1248.8	1283.6
7	1312.4	1266.4	1528.6	1451.1
8	859.8	913.7	1179.3	1062.6
9	1083.2	960.5	1141.4	1178.8
10	975.9	1121.2	1194.5	1231.7
A±SD	1062.6±178.2	1072.6±171.2	1281.9±169.9	1238.0±161.8

Table3:

Contouring volumes (cm³) from 4D-CT images for 5 patients

Patients	LS	ACM	PM	MD
1	1548.7	1615.6	1710.2	1788.3
2	1320.4	1267.8	1501.3	1573.2
3	863.5	936.5	1163.5	1102.4
4	1167.3	1239.4	1529.8	1465.5
5	1098.2	1076.3	1446.7	1356.1
A±SD	1199.6±255.3	1227.1±254.9	1470.3±197.9	1457.1±254.5

The validation results using FPVP, FNVP, and DSC were also shown in Table 4. As shown in Table 4, the volume measurement errors were all below 5%. For all kinds of CT images, our proposed method can achieve the least errors comparing LS and ACM. From Table 4, for CT images with normal appearing livers, the errors among three methods were not obvious, while for the CT and 4D-CT images with hepatocellular carcinoma or liver metastases the errors among three methods were relatively larger. The results demonstrated that our proposed method achieved better contouring results when the incoming CT and 4D-CT images had hepatocellular carcinoma or liver metastases.

Table 4:

Quantitative comparison (A±SD) of the liver contouring results by proposed method and other methods on CT and 4D-CT images; where CT-N means CT images with normal appearing livers; CT-Liver means CT images with hepatocellular carcinoma or liver metastases; 4D-CT means 4D-CT images with hepatocellular carcinoma or liver metastases;

(1) $\mathit{\text{FPVP}}$ (%)
Method	CT-N	CT-Liver	4D-CT
LS	2.57±0.72	3.62±0.43	3.96±0.58
ACM	2.42+0.51	3.00±0.56	3.11±0.19
PM	2.15±0.38	2.28±0.19	2.37±0.42
(2) $\mathit{\text{FNVP}}$ (%)
Method	CT-N	CT-Liver	4D-CT
LS	3.23±0.31	4.33±0.23	4.57±0.18
ACM	3.17±0.29	3.73±0.42	3.94±0.78
PM	2.96±0.45	3.15±0.18	3.25±0.64
(3) $\mathit{D}\mathit{S}\mathit{C}$ (%)
Method	CT-N	CT-Liver	4D-CT
LS	91.01±0.72	86.14±0.37	82.23±0.18
ACM	92.43±0.25	88.32±0.54	83.62±0.67
PM	97.21±0.43	93.53±0.85	89.44±0.93

Our experiment demonstrated that the proposed method was better than other methods with liver contouring. This is mainly because the hybrid method can take advantage of different methods to overcome the challenge and weakness existing in automatic liver contouring for radiotherapy treatment planning. The automatic contouring results on the patients with normal appearing livers by proposed method were more accurate than those segmented results of the patients with hepatocellular carcinoma or liver metastasis. The normal appearing liver regions can always be modeled better by the finite Gaussian model and the improved level set based method can delineate the liver contours more accurately. However, for those patients with hepatocellular carcinoma or liver metastasis, the candidate liver region obtained by the second step usually contained tumors. In these cases, texture analysis method should be used to classify the soft tissues within the candidate liver region. In most cases, we can use the contouring results from the automatic contouring. Even in the difficult cases, the contouring results from our proposed method can then be used as a starting point by the experienced physicians to refine the contours. It can still save time and improve the efficiency of the radiation therapy treatment planning.

According to our study, the experiment on 20 planning CT images and 5 groups of 4D-CT scans, we found that the average time of the level set based method, active contour model and the proposed method on every slice CT images with normal appearing livers was 1.15 minutes, 3.03 minutes and 0.69 minutes, respectively. The average time of the level set based method, active contour model and the proposed method on every slice CT images of 10 patients with hepatocellular carcinoma or liver metastases was 1.32 minutes, 3.37 minutes and 0.81 minutes, respectively. The average time of the level set based method, active contour model and the proposed method on every slice 4D-CT images was 1.49 minutes, 3.82 minutes and 0.92 minutes, respectively. The average time for each of the three steps is 0.13±0.07 minutes, 0.44±0.08 minutes and 0.21±0.04 minutes, respectively. Normally, when physicians or oncologists design the radiation therapy treatment planning in clinic, the average time with every slice CT image is about 1.0 minute. The reason why our method is relatively faster is because the initial liver contour extracted by TV-L1 model is very close to the real liver contour, and it can speed up the convergence of the level set based method. Furthermore, LCC was combined into the energy function, where LCC was obtained by constant calculating process which was very fast. Our proposed method utilized the advantages from different method to solve problems in liver contouring for automatic radiation therapy planning, and it can improve the contouring efficiency.

4. DISCUSSION

The ability to do automatic contouring is important for radiation therapy treatment planning, and it can greatly improve the efficiency of cancer treatment planning. In this study, we proposed a three-step hybrid method by using the edge, intensity distribution and voxel-based texture characterization to contour the liver boundary and any intra-haptic tumors automatically. The experimental results demonstrated the efficiency of our proposed method. The liver contouring results by our proposed method were compared with those of the manual delineation and two other automatic segmentation methods on both CT images and 4D-CT scans. Our results demonstrated that the contouring volumes of our proposed method were the closest to that of manual delineation results by the expert group. The method was robust even though the liver shapes were very variable especially in the cases with liver tumors. This was mainly because the proposed method not only used the intensity information within the CT data but also the texture information which was important for the contouring process. From the experiment results, the proposed method can exclude the hepatocellular carcinoma and metastatic liver from normal liver volume, whereas the other methods cannot get ideal results in these situations. The improved level set based method can combine the intensity distribution with the defined local correlation which was used to track the large liver boundary alongside the liver outline. Then the texture analysis method was also used to classify the normal and abnormal image voxels within the liver region.

In our study, the TV-L1 model, which had the characteristic of multi-scale decomposition and edge preserving property, was used to extract the initial liver region from the original patient image according to organ size but not intensity. It was able to remove the surrounding muscles and tissues to obtain a reasonable and robust initial liver region as a starting point. Furthermore, the obtained initial liver contour can divide the original patient image into two parts, and we can estimate the gray pair distribution by the GLCM computation within the foreground and background region. In the second step, an improved level set based method containing the intensity distribution and local correlation information was used to overcome the variable liver shapes and different sizes. In this step, finite Gaussian model and mixture Gaussian model were used to estimate the liver intensity distribution by classifying the pixels within images. To reduce the regions of overlapping distribution area, we defined LCC for calculating the correlation of each pixel within the target region and background region. LCC was based on the GLCM within the foreground and background region. However, the improved level set based method segmented the target liver region which may contain tumor regions, and the tumor within the liver region may have the similar intensity information and make it difficult to remove the tumor from the initial liver region by intensity information. In the last step, voxel based texture characterization was used to classify these soft tissue and remove the tumor region to obtain the final liver region. During this process, an iterative strategy was used to improve the accuracy within the texture analysis based method. Our experiment results showed that the proposed method was more accurate with clinically acceptable errors as shown in tables and figures above.

The proposed three steps hybrid method was also more robust than other methods in contouring planning 3D CT images and 4D-CT scan images from the clinic patients. In the last step within the proposed method, we used the voxel based texture characterization for the candidate liver region, and mainly to remove the tumor from the normal liver region. When our automatic contouring method was used in the clinical treatment planning system, we only employed the first two steps of the proposed method for segmenting the liver region which is normal appearing. The results were accurate and robust without the texture analysis step. For the normal appearing liver region, we performed the volume calculations with and without the last texture analysis step, and the results with the CT images were almost the same as shown in Figure 5.

Figure 5:

Image (a) was the CT images with normal appearing liver region segmented by the three steps hybrid method. Image (b) was CT images with normal appearing liver region obtained by the first two steps of the proposed method.

The second step using improved level set based method was critical for the whole contouring process. When the target images were planning CT or 4D-CT images with normal appearing livers, the improved level set based method can achieve the final liver contour for radiation therapy planning. However, when the target images were planning CT or 4D-CT images with hepatocellular carcinoma or liver metastases, the improved level set based method was just used for tracking the outside contour alongside the outline of the liver region, which was regarded as candidate liver region, and it contained all possible liver region and can avoid the liver region loss which may be caused by hepatocellular carcinoma or liver metastases. During this process, we combined LCC and Gaussian models to construct the energy function, and LCC based on GLCM contained the local correlation of pixels and can help Gaussian model classify pixels during the evolution of level set function. The improved method utilized the intensity distribution and local correlation information to classify the pixels. Figure 7 showed that LCC could help Gaussian model to classify the pixels with related liver tissues efficiently, and made the automatic contouring results closer to the manually delineated results by the expert group.

Figure 7:

Image (a) was the liver segmented result of CT image with widespread liver metastases by the proposed method, and image (b) was the manually delineated result with the same image by clinical radiation therapy physicians.

Furthermore, the proposed method also has the advantage of being faster than the other two methods. This is mainly because the TV-L1 model can extract the initial liver contour from the original liver region according to size of the organ, and we can get an optimized initial contour following the level set based method. The initial liver contour was always located on the liver boundary, so it can save the evolution time during the contouring process.

Although the proposed method is robust and efficient, it still has some limitations, especially when the liver region is heterogeneous caused by complex solid tumors or severe liver metabolic diseases. This is because these liver regions usually cannot be only modeled by a finite Gaussian model. It may not exclude the tumor pixels from the normal liver region. In these cases, the segmented results by the proposed method are larger than the manual segmentation results in most cases. To overcome the problem in the future, we try to use a mixture Gaussian to model the liver region when it has the characteristic of complex tumor. Figure 7 (a) shows the bad contouring results caused by the severe metastatic liver disease and image Figure 7 (b) shows the manually delineated results by the expert group. From Figure 7 we readily see that there is a big difference with the contouring results between the proposed method and results delineated manually by the physicians. In Table 5, the contouring volumes results were shown for comparison between the proposed method and manually delineation for the difficult cases with complex solid tumors or severe liver metastatic in CT images. In these difficult cases there were large errors with our proposed method. However, from the evaluation of the expert group, even in these worse cases, the results from the proposed method can still be a good starting point for subsequent editing by clinical physician, and can improve the efficiency of treatment planning.

Table 5:

Contouring volumes from CT images for 10 patients with complex solid tumors or severe liver metastatic diseases (cm³)

Patient	PM	MD
1	1443.1	1020.5
2	1667.3	1372.8
3	897.4	964.2
4	1653.5	1120.9
5	1394.6	973.4
6	1533.2	1188.7
7	1598.7	1417.3
8	1214.5	808.5
9	1576.8	1243.6
10	698.9	788.5
A±SD	1367.8±332.3	1089.8±217.5

Furthermore, the proposed method (PM) was suffering from CT images with non-uniform distribution noises or artifacts. The results were shown in figure 8. The reason was that although the Gaussian model and local correlation coefficient (LCC) in level set based method can overcome the effect of the uniform noise and artifacts, however, if the noises or artifacts were non-uniform, the proposed method cannot achieve good performance which was just shown in figure 8 (a) and (b). Specially, considering the mental artifacts in figure 8 (b), the texture values were very hard to model. There were scattering artifacts and volume artifacts existing in the CBCT images, and some liver pixels were classified into the non-liver region, so the proposed method cannot contour the liver boundary from CBCT images which was including scattering artifacts just as shown in figure 8 (c) and (d). In the future work, we will try to solve this problem by modeling the noise.

Figure 8:

Image (a) and (b) were CT images with non-uniform distribution noises or artifacts. Image (c) and (d) were CBCT images with non-uniform distribution noises or artifacts.

5. CONCLUSION

In this work, a three step hybrid framework was proposed for automatic liver contouring with application in radiation therapy treatment planning. From quantitative and qualitative analysis of the experiment results on the clinical dataset, we concluded that the proposed method was able to achieve liver contouring from planning 3D CT images and 4D-CT images automatically. An advantage of our proposed method is that the hybrid method can overcome the challenge of similar intensities between liver region and its surrounding tissues. This method can also be used for automatic contouring of other organs. It should find widespread application in future treatment planning systems.

Figure 6:

Images (a), (b), (c) were the liver segmented results of CT images with the normal appearing liver region by the proposed method without LCC, with LCC, and MD, respectively. Images (d), (e), (f) were the liver segmented results of CT images with liver metastases by the proposed method without LCC, with LCC, and MD, respectively. Images (g), (h), (j) were the liver segmented results of 4D-CT images with hepatocellular carcinoma by the proposed method without LCC, with LCC, and MD, respectively.