Iterative Mesh Transformation for 3D Segmentation of Livers with Cancers in CT Images

Abstract

Segmentation of diseased liver remains a challenging task in clinical applications due to the high inter-patient variability in liver shapes, sizes and pathologies caused by cancers or other liver diseases. In this paper, we present a multi-resolution mesh segmentation algorithm for 3D segmentation of livers, called iterative mesh transformation that deforms the mesh of a region-of-interest (ROI) in a progressive manner by iterations between mesh transformation and contour optimization. Mesh transformation deforms the 3D mesh based on the deformation transfer model that searches the optimal mesh based on the affine transformation subjected to a set of constraints of targeting vertices. Besides, contour optimization searches the optimal transversal contours of the ROI by applying the dynamic-programming algorithm to the intersection polylines of the 3D mesh on 2D transversal image planes. The initial constraint set for mesh transformation can be defined by a very small number of targeting vertices, namely landmarks, and progressively updated by adding the targeting vertices selected from the optimal transversal contours calculated in contour optimization. This iterative 3D mesh transformation constrained by 2D optimal transversal contours provides an efficient solution to a progressive approximation of the mesh of the targeting ROI. Based on this iterative mesh transformation algorithm, we developed a semi-automated scheme for segmentation of diseased livers with cancers using as little as five user-identified landmarks. The evaluation study demonstrates that this semiautomated liver segmentation scheme can achieve accurate and reliable segmentation results with significant reduction of interaction time and efforts when dealing with diseased liver cases.

I. Introduction

Liver cancer is the third most common cause of death from cancer worldwide [1], and liver is the most common metastatic spread sites of cancers after the lymph nodes, which have been found in 30-70% of patients who are dying of cancer [2]. With the technical advances of computed tomography (CT), hepatic CT scanning has become one of the major routinely-used clinical imaging modality in cancer diagnosis, staging, and treatment evaluation because of the relatively low cost and wide availability [3]. Segmentation of liver in CT images is crucial for liver-related clinical applications. However, accurate and reliable liver segmentation continues to be a challenging task especially for diseased livers because of the deformation of the liver shape caused by large tumors, the complexity of the diseased liver pathologies, the variability of image presence of different hepatic diseases (such as hypodense or hyperdense lesions), and the fuzzy boundaries between liver and the adjacent organs (such as heart and stomach). Therefore, liver segmentation attracts continuously researchers' attentions in medical imaging and graphics community [4, 5].

Numerous automated, semi-automated and interactive methods for liver segmentation have been developed, including statistical shape model [6-8], atlas matching method [9-11], deformable model [12, 13], level-set method [14-17], and other hybrid methods based on user-interaction [18], graph-cut [19], region-growing [20], machine-learning [21], etc. With regard to segmentation accuracy and reliability, there is a clear tendency that interactive methods outperform semi-automated methods, which in turn outperform fully automated approaches. However, user interaction time and efforts tend to have a reversed tendency: interactive segmentation of a liver on CT images is labor-intensive, time-consuming and prone to inter-operator variability. Clinical applications seek an accurate and reliable segmentation method for diseased livers with minimum amount of required user interactions.

The technique challenges of the segmentation of diseased livers result from the high inter-patient variability in liver shapes, sizes and pathologies as well as the longitudinal intra-patient variability. It is not uncommon that cancers and metastases change the liver size and deforms the liver shape, and thus result in a very large range of variations among patients. Figure 1 demonstrates the inter-patient variability in shapes and sizes of diseased livers from patients with cancers or metastases. The sizes of livers vary significantly with more than four times of difference. More importantly, the shapes of livers show substantial difference caused by cancers or metastases or treatments: 1) liver shapes may be bended or twisted or deformed severely due to the growing or shrinking of tumors; 2) different hepatic lobes may be shrunk or expanded extraordinarily; and 3) tumor tissues show substantially different CT intensity compared to normal liver parenchyma. This inter-patient variability makes the modeling of a statistic shape for liver segmentation an extreme challenging task. To deal with the complex shape variations of diseased livers, sparse shape composition model [22] and auto context multi-atlas model [23] were proposed to handle errors or outliers of statistic models and preserve local details for different shapes of liver. These atlas-based or shape-prior-based automatic methods require learning from available shapes of segmented livers, which may be itself a difficult task.

Figure 1.

Examples of the inter-patient variability in sizes and shapes of diseased lives from patients with cancers or metastasis. All the patients were scanned by the same protocol at supine position. All the resulting images are displayed in the anterior viewing orientation. The sizes of livers vary significantly with more than four times of difference and more importantly the shapes vary substantially too. The CT intensities of liver parenchyma and cancerous tissues are apparently different.

As long as fully-automated segmentation methods fail to produce satisfactory segmentations on these diseased cases, certain amounts of user interaction are still required. Deformable models offer the essential options of interactive control over the segmentation process via interactive manipulating of a deformable surface [24]. Traditional mesh deformation methods simulate the mechanism behavior of a surface by minimizing the stretching and bending energies of the surface, such as active geometric deformed models [25] and active contour models or snakes [26]. However, one limitation of these energy-based mesh deformable methods is that the difference between the initial mesh and resulting mesh must be sufficiently small, which may turn out a large amount of interactive manipulations. Motivated by the deformation transfer model [27], which transfers the mesh deformation using deformation gradients instead of local energies, mesh may be deformed efficiently using a small set of targeting points while allowing accurate detail-preserving interactions.

In this paper, we proposed a multi-resolution mesh segmentation algorithm, called iterative mesh transformation, that deforms the mesh of a region-of-interest (ROI) in a progressive manner by iterations between mesh transformation and contour optimization. The initial constraint set for mesh transformation can be defined by a very small number of targeting vertices, namely landmarks, and progressively updated by adding the targeting vertices selected from the optimal transversal contours calculated in contour optimization. This iteration between 3D mesh transformation and 2D transversal contours optimization provides an efficient solution to a progressive approximation of the targeting ROI mesh with minimum user interactions. By using this iterative mesh transformation algorithm, we developed a semi-automated image segmentation scheme for diseased livers with cancers. This scheme can effectively segment the diseased livers with cancers using as little as five user-identified landmarks. It achieves accurate and reliable segmentation results when dealing with these diseased liver cases, and reduces significantly the interaction time and efforts. The resulting 3D liver mesh provides the meta-data for quantitative image analysis of diseased livers in different clinical applications.

The major contributions of the proposed method are twofold.

• Technical contributions: Diseased livers tend to have high inter-patient variability of shapes and sizes, which makes the modeling of a statistic shape for liver segmentation an extreme challenging task. The transformation optimization in iterative mesh transformation provides an effective solution for the global approximation of a diseased liver, which is superior to those of energy-based mesh deformation that tends to converge to a local boundary caused by the inhomogeneity in diseased regions. On the other hand, the transversal contours optimization provides the local refinement for detecting the precise boundaries of a liver. As a result, the combination of 3D transformation optimization and 2D contour optimization provides an accurate and efficient technical solution for the segmentation of diseased livers.

• Clinical contributions: Manual contouring remains the major clinical procedure for segmentation of diseased livers with pathologies, which is labor-intensive, time-consuming and prone to inter-operator variability. The proposed iterative liver segmentation scheme is easy-of-use and reduces the interaction effort as little as five user-identified landmarks. It takes in average 5 minutes to segment a diseased liver case, which is more efficient than most of existing liver segmentation methods. The accuracy and running time have been evaluated by the MICCAI cases as well as the real clinical cases collected in our institute.

II. Method

A. Mesh Segmentation Model

Let I(v) denote the pixel value at a vertex v = [x,y,z]^T ∈ R³ in a CT volume, where R³ is 3-dimensional (3D) Euclidean space. A mesh can be defined as a 3-tuple M = (V,E,Z), where V is a vector of vertices, E is a set of edges connecting two vertices, Z is a set of faces defined by a closed set of edges, in which a triangle face has three edges. A mesh segmentation is defined as an optimization process that deforms an initial mesh (such as a polyhedron) under certain constrains for segmentation of a ROI, which can be represented as an evolution process Λ(θ) = (M₀,M_θ, Ω), where M₀ is the initial mesh, M_θ is the resulting ROI mesh, Ω is the constrain set, and θ is the steps of evolution.

Figure 2 illustrates the mesh evolution in the segmentation of a sphere, in which the initial mesh M₀ is defined as an icosahedron. During each evolution, a triangle face is decomposed into four smaller triangle faces by adding a middle point on each edge of the triangle (see point D on edge AC in Figure 2(a)). The corresponding targeting vertex of the newly-added middle point is the boundary point on the sphere surface (see point D₁ in Figure 2(b)). Mesh segmentation is a very effective way to approximate the surface or boundary of an object, because the number of faces increases in an exponential manner during the evolution. Table 1 lists the properties of the meshes at each step of evolution. The vertices set of M_θ contains all the vertices of M_θ−1.

Figure 2.

Mesh evolution in the segmentation of a sphere. (a) Initial mesh of an icosahedron is evolved by replacing of a triangle face with 4 smaller triangle faces. The newly-added targeting vertex D₁ is the boundary point of the middle point D of edge AC along the direction of OD, where O is the center point of the sphere. (b) The resulting mesh after 5 times of evolution, which consists of 10242 vertices, 30720 edges and 20480 triangle faces.

Table 1.

Number of vertices, edges, and faces of meshes at different step θ. Mesh evolution indicates V(M_θ–1) ⊂ V(M_θ). In Figure 2, meshes of a sphere at θ = 0, 1, …, 5 are illustrated.

θ	0	1	2	3	4	5
Vertices	12	42	162	642	2562	10242
Edges	30	120	480	1920	7680	30720
Faces	20	80	320	1280	5120	20480

Without the loss of generality, we may assume that the targeting ROI is a closed region and thus can be represented by a 2-dimensional (2D) manifold, aka a boundary surface. This assumption indicates that the segmented ROI is locally homeomorphic to a 2D Euclidean space (such as a plane), and thus its boundary surface can be modeled by a triangle mesh. Considering the fact that a sphere is a 2D manifold and is homeomorphic to the boundary surface of a targeting ROI, the homeomorphic theory states that there is a continuous one-to-one mapping between these two surfaces, as indicated below:

If X and Y are homeomorphic, there exists a mapping ψ: X → Y that satisfies the following criteria: (1) ψ is a one-to-one correspondence, (2) ψ is continuous, and (3) ψ⁻¹ is continuous.

If such a mapping ψ exists, we can transform a sphere mesh M^Sphere to the ROI mesh M^ROI by using ψ defined by a set of transformations between M^Sphere and M^ROI. Therefore, mesh segmentation is defined as a multi-resolution mesh optimization process by using an objective function subject to a set of correspondence constraints (Ω):

$${\mathit{M}}_{\mathit{R}}^{\ast }(\theta )=\underset{\mathit{s}.\mathit{t}.\Omega }{{\mathit{\text{argmin}}}_{{\mathit{M}}^{\mathit{\text{ROI}}}}}(w_0\mathit{f}(M_0)+w_1\mathit{f}(M_1)+\cdots +w_{\theta }\mathit{f}(M_{\theta })),$$ (1)

where ${\mathit{M}}_{\mathit{R}}^{\ast }(\theta )$ is the approximation of the ROI mesh M^ROI after the evolution of θ times, f is the objective function of the transformation optimization defined on mesh M_i, w_i is the weight of f(M_i). Constraint Ω is a set of known correspondence between the sphere mesh M^Sphere and the ROI mesh M^ROI.

The weights in Equation (1) control the “level-of-detail” (or granularity) of deformation at different resolutions of the resulting mesh. In general, w₀ controls the coarsest or overall deformation and w_θ does the finest deformation for detail preservation. Rigid organs tend to have larger value of w₀ and smaller value of w_θ, whereas flexible ones employ smaller value of w₀ and larger value of w_θ. In our study, θ is set to 5, which indicates the number of vertices of the resulting mesh of a liver is 10242 (θ = 5). w₀∼w₅, are set to 0.1, 0.1, 01, 0.2 and 0.5, respectively. These values were estimated empirically based on the flexibility of a liver.

B. Mesh Transformation

The homeomorphic assumption indicates that a local boundary of a ROI can be mapped to a local surface of a sphere if the local boundary is small enough. By dividing each transversal slice of a ROI into n sectors, as illustrated in Figure 3, we may establish the mapping correspondence ψ between the sphere surface and a ROI boundary when the sector is divided small enough. It results in a continuous one-to-one mapping between the ROI boundary point ( ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$) and the sphere mesh vertex ( ${\mathit{v}}_{\mathit{i}}^{\mathit{S}}$).

Figure 3.

${\mathit{v}}_i^S$ is a vertex on the mesh of a sphere and ${\mathit{v}}_i^{\mathit{\text{ROI}}}$ is a vertex on the boundary of a ROI, such as a liver. When the size of the section is small enough, there is a continuous one-to-one mapping between the ROI boundary point ( ${\mathit{v}}_i^{\mathit{\text{ROI}}}$) and the sphere mesh vertex ( ${\mathit{v}}_i^S$).

Let V^s and V^ROI be the vertex set of a sphere surface and a ROI boundary respectively, and the mapping function ψ be a linear transformation. We may represent ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$ as the transformed vertex of ${\mathit{v}}_{\mathit{i}}^{\mathit{S}}$:

$$\mathit{T}\times {\mathit{v}}_{\mathit{i}}^{\mathit{S}}+\mathit{D}={\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}},$$ (2)

where ${\mathit{v}}_{\mathit{i}}^{\mathit{S}}$ is the vertex of the initial sphere mesh, ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$ is the vertex of the ROI boundary, T is a 3×3 affine transformation matrix, and D is a translation vector.

Considering the geometric continuity during a mesh deformation, we assume that the four vertices v₁∼v₄ around an edge E_i in a mesh share the same affine transformation T and translation D, as shown in Figure 4(a). Thus, ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$ can be represented below:

Figure 4.

(a) T is a 3 × 3 affine transformation matrix defined by four non-coplanar vertices (v₁∼v₄)around an edge E_i. ${\mathit{v}}_i^S$ is the known variable before deformation and ${\mathit{v}}_i^{\mathit{\text{ROI}}}$ is the unknown variable after deformation. (b) The roughest face on the original icosahedron. (c) The finest face on the segmented mesh by θ times of evolution. Z_i is a triangle face of the mesh.

$$\begin{array}{c}\mathit{T}\times {\mathit{v}}_{\mathbf{1}}^{\mathit{S}}+\mathit{D}={\mathit{v}}_{\mathbf{1}}^{\mathit{\text{ROI}}},\mathit{T}\times {\mathit{v}}_{\mathbf{2}}^{\mathit{S}}+\mathit{D}={\mathit{v}}_{\mathbf{2}}^{\mathit{\text{ROI}}}\\ \mathit{T}\times {\mathit{v}}_{\mathbf{3}}^{\mathit{S}}+\mathit{D}={\mathit{v}}_{\mathbf{3}}^{\mathit{\text{ROI}}},\mathit{T}\times {\mathit{v}}_{\mathbf{4}}^{\mathit{S}}+\mathit{D}={\mathit{v}}_{\mathbf{4}}^{\mathit{\text{ROI}}}\end{array}$$ (3)

By eliminating D in Equation (3), the affine transformation T for the edge E_i between V^S and V^ROI can be determined by Equation (4):

$$\mathit{T}(E_i)={\mathit{T}}^{\mathit{\text{ROI}}}\times {[{\mathit{T}}^{\mathit{S}}]}^{-\mathbf{1}},$$ (4)

where both T^ROI and T^S are 3×3 matrix, ${\mathit{T}}^{\mathit{\text{ROI}}}=[{\mathit{v}}_{\mathbf{2}}^{\mathit{\text{ROI}}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{\text{ROI}}},{\mathit{v}}_{\mathbf{3}}^{\mathit{\text{ROI}}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{\text{ROI}}},{\mathit{v}}_{\mathbf{4}}^{\mathit{\text{ROI}}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{\text{ROI}}}]$, and ${\mathit{T}}^{\mathit{S}}=[{\mathit{v}}_{\mathbf{2}}^{\mathit{S}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{S}},{\mathit{v}}_{\mathbf{3}}^{\mathit{S}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{S}},{\mathit{v}}_{\mathbf{4}}^{\mathit{S}}-{\mathit{v}}_{\mathbf{1}}^{\mathit{S}}]$.

Because all initial vertices are defined on a sphere, any four vertices ${\mathit{v}}_{\mathbf{1}}^{\mathit{S}}~{\mathit{v}}_{\mathbf{4}}^{\mathit{S}}$ around an edge E_i are non-coplanar and thus [T^S]⁻¹ always exists. Therefore, the transformation matrix T(E_i) for an edge E_i represented by Equation (4) is a linear function of the unknown coordinates of four deformed vertices ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$, i=1∼4, as illustrated in Figure 4(a). This indicates that the mapping ψ between V^S and V^ROI is a function of unknown vertices ${\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}}$ and can be represented by a linear combination of T(E_i), where E_i is an edge in the edge set of all the triangles that share the vertex of ${\mathit{v}}_i^S:E_i\in \{E(Z_i),\text{where}{\mathit{v}}_i^S\in \mathit{V}(Z_i)\}$.

C. Mesh Objective Function

The objective function f(M_i) in Equation (1) is formed by the weighted sum of three types of constraints: smooth constraint (C_S) deforming constraint (C_D) and distance constraint (C_L):

$$\mathit{f}=w_S{\mathit{C}}_{\mathit{S}}+w_D{\mathit{C}}_D+w_L{\mathit{C}}_{\mathit{L}},$$ (5)

where w_S, w_D, w_L are the weights of the corresponding constraints.

The smooth constraint C_S is so defined that the transformation matrix T(E_i) (see Equation (4)) for three edges of a triangle face should be as equal as possible. In specific, for each triangle face Z_i in a mesh M, as shown in Figure 4(b and c), C_S is defined below:

$${\mathit{C}}_{\mathit{S}}(\mathit{M})=\sum _{{\mathit{Z}}_{\mathit{i}}\in \mathit{Z}(\mathit{M})}\sum _{\begin{array}{c}j\ne k=1\\ E_j\in \mathit{E}({\mathit{Z}}_{\mathit{i}})\\ E_k\in \mathit{E}({\mathit{Z}}_{\mathit{i}})\end{array}}^3{\Vert (\mathit{T}(E_j)-\mathit{T}(E_k)\Vert }_F^{\mathbf{2}},$$ (6)

where Z(M) indicates the triangle face set of mesh M, Z_i is the i^th face, T(E_j) and T(E_k) are the transformation matrix of edge E_j and E_k in the triangle Z_i respectively, ${\Vert \cdot \Vert }_F^2$ is the Frobenius norm of a matrix.

The deforming constraint C_D transforms the vertices ( ${\mathit{v}}_i^{\mathit{\text{ROI}}}$) of the mesh as close as possible to the targeting vertices ( ${\mathit{v}}_i^{TC}$) of a ROI:

$${\mathit{C}}_{\mathit{D}}(\mathit{M})={\sum }_{({\mathit{v}}_{\mathit{i}}^{\mathit{\text{ROI}}},{\mathit{v}}_{\mathit{i}}^{\mathit{\text{TC}}})}{\Vert ({\mathit{v}}_i^{\mathit{\text{ROI}}}-{\mathit{v}}_i^{\mathit{\text{TC}}})\Vert }^2,$$ (7)

where Ω is the set of known vertex correspondence between the sphere mesh and the object boundary of a ROI.

Targeting vertices are known boundary points of a ROI, which may be established by the user-identified landmarks of the ROI V^landmark, the segmented neighboring organs such as V^chest (for liver segmentation), and the selected contour points V^contour on optimal transversal contours. For our application of liver segmentation, we can define $\mathit{\Omega }=\{({\mathit{v}}_i^{\mathit{\text{ROI}}},{\mathit{v}}_i^{\mathit{\text{TC}}})|{\mathit{v}}_i^{\mathit{\text{ROI}}}\in \mathit{V}(\mathit{M})\mathit{\text{and}}{\mathit{v}}_i^{\mathit{\text{TC}}}\in ({\mathit{V}}^{\mathit{\text{landmark}}}\cup {\mathit{V}}^{\mathit{\text{chest}}}\cup {\mathit{v}}^{\mathit{\text{contour}}})\}$.

In order to maintain the shape regularity of a triangle and prevent the mesh from drastic deformation, we add the distance constraint C_L for maintaining the minimum signed distance change before and after deformation:

$${\mathit{C}}_{\mathit{L}}(\mathit{M})=\sum _{\begin{array}{c}{E}_i\in \mathit{E}(\mathit{M}),\\ {\mathit{v}}_{\mathbf{1}}\in \mathit{V}(E_i),\\ {\mathit{v}}_{\mathbf{2}}\in \mathit{V}(E_i)\end{array}}{\Vert ({\mathit{v}}_1^{\mathit{\text{ROI}}}-{\mathit{v}}_2^{\mathit{\text{ROI}}})-({\mathit{v}}_1^s-{\mathit{v}}_2^s)\Vert }^2,$$ (8)

where E(M) indicates the edge set of mesh M, V(E_i) is a set of end points of edge E_i, i.e., v₁ and v₂ are two end points of an edge E_i.

The weights in Equation (5) balance three deformations applied to each triangle: w_S and w_L are smoothing terms in which w_S controls the continuity of transformations among adjacent triangles and w_L controls the shape regularity of a triangle; w_D is a positive term which controls the transformation between mesh vertices and the targeting vertices, i.e. the detected boundary points. In our study, we set w_S = 1, w_D = 100 and w_L = 1. The large value of w_D implies the strong preservations of the known boundary vertices.

D. Mesh Optimization

Based on Equation (1), mesh segmentation is an optimal process for minimizing the objective function by searching a new set of unknown vertices V^ROI. Because all the vertices in the resulting deformed mesh M_θ = (V, E, Z) are independent, Equation (1) can be represented by a vector-valued system of ${\mathit{V}}^{\mathit{\text{ROI}}}=[{\mathit{v}}_1^{\mathit{\text{ROI}}},{\mathit{v}}_2^{\mathit{\text{ROI}}},\dots ,{\mathit{v}}_n^{\mathit{\text{ROI}}}]$ which includes all vertices of meshes M₀∼M_θ. In addition, because T(E_i) is a function of ${\mathit{v}}_i^{\mathit{\text{ROI}}}$ (see Equation (4)), all constrains in the objective function f form a second order polynomial of ${\mathit{v}}_i^{\mathit{\text{ROI}}}$, i.e., f can be represented by a linear system:

$$\mathit{f}={\sum }_{\xi =1}^{N^{\ast }}{\Vert {\mathit{A}}_{\xi }\mathit{V}+{\mathit{B}}_{\xi }\Vert }_F^2,$$ (9)

where N*(*=S, D, L) is the total number of items in Equation (6), (7) and (8), A_ξ is a N* × n sparse coefficient matrix and B_ξ is a N column vector.

Therefore, the object function f in Equation (5) can be reformed to a system of linear equations of variable vector V^ROI. Because function f has a representation of a sparse matrix, we may use the sparse LU factorization [28] to efficiently solve the optimized V^ROI in the linear system, which results in the transformed ROI mesh. The detailed process of solving the minimization problem in Equation (1) can be found in Appendix 1.

E. Contour Optimization

Initial mesh segmentation based on the constraint set Ω without V^contour in Equation (7) may result in the mismatches between the intersection polylines of the mesh and the true ROI boundaries on 2D transversal image planes in CT volume due to the lack of local boundary constraints (V^contour) in Ω. Thus, the constraint set Ω needs to be updated before the next evolution of mesh deformation to achieve accurate segmentation. This constraints updating is performed automatically by adding targeting vertices ( ${\mathit{v}}_i^{\mathit{\text{TC}}}$) selected from the local boundaries of a ROI into V^contour, which are searched in the neighborhood of the initial intersection polylines.

Assuming that the boundary of a ROI is located in the neighborhood of the intersection polylines of the initial mesh on the 2D transversal images. This neighborhood is defined as a band on the transversal images centered by the intersection polylines of the initial mesh, as shown in Figure 5. The width of the band can be adjusted in terms of the size of the ROI. In our study, we set the band width to be 40 pixels. The local boundary of a ROI will be detected by optimal-path searching within this local band.

Figure 5.

The band (marked in blue) is centered by the intersection polylines of the mesh. The seed points ( ${\mathit{v}}_i^{\mathit{\text{seed}}}$) on the intersection polyline are used to search the local boundary of a ROI on the transversal image plane. L is the width of the band, which is set to 40 in this study.

A transversal CT image can be represented by a 2D weighted bi-directed graph, in which one node corresponds to a pixel in the image. Each node (pixel) has 8 edges connecting to its neighborhood pixels, and each node and each edge have a related cost, respectively. A local ROI boundary on a transversal image plane is defined as the optimal path with the minimum cost between two corresponding nodes in the graph. We defined five cost functions on a node or an edge employing the forms similar to those described in [29].

(1) The Laplacian zero-crossing cost

The Laplacian zero-crossing is defined as the points with local maximal or minimal gradient magnitude, which corresponds to a boundary point. Thus, a node has a low cost if it is a zero-crossing point:

$$S_{ZX}(\mathit{v})=\{\begin{array}{cc}0;& \mathit{\text{if}}{\nabla }^2(\mathit{v}){\nabla }^2(\mathit{u})<0\mathit{\text{and}}|{\nabla }^2(\mathit{v})|<|{\nabla }^2(\mathit{u})|\\ 1;& \mathit{\text{Otherwise}}\end{array}$$ (10)

where ∇²() is the Laplacian operator u is a neighboring pixel of v.

(2) Gradient magnitude cost

A large gradient magnitude tends to indicate a local boundary. Thus, a gradient magnitude is considered as the local cost of a boundary:

$$S_g(\mathit{v},\mathit{u})=k(1-\frac{|\nabla (\mathit{u})|}{\mathit{max}(|\nabla ({\mathit{u}}_{\mathit{i}})|)}),$$ (11)

where ∇() is the gradient operator, k is 1 if v and u are diagonal neighbors, and k is $\sqrt{2}$ if v and u are horizontal or vertical neighbors.

(3) Gradient direction cost

We assume that a local boundary is perpendicular to the gradient direction. Thus, the cost is low if the direction of an edge is perpendicular to the gradient direction of the pixel:

$$S_d(\mathit{v},\mathit{u})=|\frac{\mathit{v}\mathit{u}}{|\mathit{v}\mathit{u}|}\cdot \frac{\nabla (\mathit{v})}{|\nabla (\mathit{v})|}|,$$ (12)

where u is a neighboring node of v.

(4) Contour direction cost

Because the edge in the graph is bi-direction, we may define that the ROI region is located at the right side of the contour when walking through the contour in the clockwise direction. In order to separate a ROI region with low or high intensity compared to its background, we construct a cost function that compares the normal direction of the contour and the gradient direction:

$$S_c(\mathit{v})=\frac{1}{2}(1+k\cdot \mathit{n}\cdot \frac{\nabla (\mathit{v})}{|\nabla (\mathit{v})|}),$$ (13)

where n is the normal of the contour (see Figure 6(b)), k is 1 for brighter ROI such as liver parenchyma (enhanced liver tissue is brighter than its neighborhood), and is -1 for darker ROI such as tumor (tumor tends to be darker than normal liver tissues). The default value of k is 1 and is set to -1 manually for tumor ROI.

Figure 6.

The construction of an isocontour map on a transversal image plane. (a) The green contour is the intersection polylines of the initial mesh on the 2D transversal image, which is defined as the zero-isocontour. The region of the ROI is marked in light brown color, which is the inner region of the contour. (b) The e axis is the distance to the zero-isocontour: inner region has negative distances, whereas outer region has positive distance. n is the normal of the contour (from inner to outer) and g is the direction of the local image gradient. The maximum distance value is defined 8 pixels and the minimum is -8 pixels.

(5) Isocontour cost

Assume that a local ROI boundary has a similar profile. We define an iso-distance contour map (see Figure 6) of a local contour: if two neighboring pixels are located at the same level of isocontour, its cost is low; otherwise the cost is high. The maximum isocontour value is defined as 8 pixels and the minimum isocontour is -8 pixels. The initial zero iso-contour is the intersection polylines of the initial mesh. To normalize the cost, the value of the cost is divided by 16:

$$S_{\mathit{\text{Iso}}}(\mathit{v},\mathit{u})=|e(\mathit{v})-e(\mathit{u})|/16,$$ (14)

where e(v) is the isocontour value at pixel v.

The local cost function from vertex v to vertex u is the weighted sum of five aforementioned costs defined on vertex v and edge vu:

$$S(\mathit{v},\mathit{u})=\{\begin{array}{cc}{\sum }_{\mathit{\text{Cost}}},& \mathit{\text{if}}\mathit{u},\mathit{v}\in B\\ \infty ,& \mathit{\text{otherwise}}\end{array},$$ (15)

where Σ_Cost= w_ZXS_ZX(v) + w_gS_g(v,u) + w_dS_d(v,u) + w_cS_C(v) + w_IsoS_Iso(v,u), v,u are two vertices located within the band B shown in Figure 5. We set w_ZX=0.15, w_g=0.15, w_d=0.1, w_c=0.5, and w_Iso=0.1. These values were set empirically, which may be fine-tuned in different clinical applications and imaging modalities.

We use the dynamic-programming (DP) graph search algorithm [29] to search the local boundaries of a ROI within the band, such as the Dijkstra's algorithm or the A* algorithm [30]. To accelerate the searching of an optimal path with minimum cost, we employed an active list that sorts nodes by the total cost of starting nodes. This sorted active list takes computation time O(N) for sorting of N nodes (pixels) [30].

The seed points ( $v_i^{\mathit{\text{Seed}}}$) for searching the optimal path are selected on the intersection polyline at an interval of n pixels (n=60 in our study). Contour optimization searches the optimal path with minimal cost between each pair of seed points ( $v_i^{\mathit{\text{Seed}}},v_{i+1}^{\mathit{\text{Seed}}}$) by using Equation (15) and then the resulting optimal paths are connected to form the optimized transversal contour C_Opt of the ROI.

The local boundary constraints (V^contour) in constraint set Ω in Equation (7) is updated by adding a set of boundary points that are selected from the optimized transversal contour C_Opt as the constraint targeting vertices ( ${\mathit{v}}_i^{\mathit{\text{TC}}}$). To ensure that the selected ${\mathit{v}}_i^{\mathit{\text{TC}}}$ is at or near the boundary of the ROI, we applied the following selection criteria:

• The middle segment of each optimal path between ( $v_i^{\mathit{\text{Seeds}}},v_{i+1}^{\mathit{\text{Seeds}}}$) is selected to reduce the influence of inaccurate positions of initial seed points.

• For each point at the middle segment of a path, the vertex with the lowest cost is selected as the candidate of ${\mathit{v}}_i^{\mathit{\text{TC}}}$.

• All the ${\mathit{v}}_i^{\mathit{\text{TC}}}$ candidates on the transversal contour C_Opt are filtered to remove those that have a roof-edge, considering the segmented ROI has a step-edge.

Mesh transformation and contour optimization is performed in an iterative manner: mesh transformation initializes the transversal contours, and contour optimization updates the constraint targeting vertices for the refinement of the targeting mesh in the next iteration of mesh transformation. At each mesh transformation, ROI mesh is evolving from low to high resolution, as indicated by Equation (1). This iteration between mesh transformation and contour optimization is repeated in a multi-resolution progressively-optimized manner to ensure an efficient and accurate approximation of the ROI mesh.

III. Liver Segmentation Scheme

By applying the iterative mesh deformation algorithm, we developed a semi-automated segmentation scheme for diseased livers with cancer on CT images. The algorithmic steps of the semi-automated liver segmentation scheme are illustrated in Figure 7.

Figure 7. The semi-automated liver segmentation scheme.

The mesh segmentation of a liver is initialized by 5 manually identified liver anatomical landmarks and a set of chest wall points V^chest that are detected automatically [31]. The five initial liver landmarks are illustrated in Figure 8, which are the liver top (superior) point (v^t), the liver bottom (inferior) point (v^b), the liver posterior point (v^p), the liver anterior point (v^a), and the liver left point (v^l), respectively. Because the right boundary of a liver is adjacent to the chest wall, a set of points at the boundaries of the chest wall is selected as the initial targeting constraint vertices, which are shown as the red points ${\mathit{v}}_i^c$ in Figure 8(a).

Figure 8.

Liver mesh segmentation is initialized by five user-identified liver anatomical landmarks and a set of points from the chest wall detected automatically. (a) Five liver-landmarks V^landmark = {v^t, v^b, v^p, v^a, v^l} are manually identified and the chest wall points ${\mathit{V}}^{\mathit{\text{chest}}}=\{{\mathit{v}}_i^C\}$ are detected in an automated manner. Their correspondence points on the sphere mesh are {v^st, v^sb, v^sp, v^sa, v^sl, ${\mathit{v}}_i^{\mathit{\text{SC}}}$}. (b) The initialized liver mesh M⁰ is deformed based on the constraint set Ω₀ using Equation (1).

The initial homeomorphic sphere mesh for constructing M⁰ is built by setting the center at $\frac{1}{2}({\mathit{v}}^t+{\mathit{v}}^b)$ and the radius of $\frac{1}{2}|{\mathit{v}}^t-{\mathit{v}}^b|$ as shown in Figure 8(a). To establish the correspondence constraints Ω, each targeting vertex on CT images ( ${\mathit{v}}_i^{\mathit{\text{TC}}}$ in Equation (7)) needs to be mapped to the corresponding vertex on the sphere mesh, which is defined as the nearest vertex on the sphere mesh for each ${\mathit{v}}_i^{\mathit{\text{TC}}}$, as shown in Figure 8(a). By using the initial correspondence constraints Ω₀, the sphere mesh is deformed to the initial liver mesh M⁰ based on mesh transformation in Equation (1). The resulting M⁰ is displayed in Figure 8(b).

Because the deformed liver mesh M⁰ intersects with all transversal slices of CT images between the top and bottom landmarks, we may calculate a series of intersection polylines of the mesh on each transversal slice. These intersection polylines are re-sampled at an interval of 60 pixels to generate a set of seed points ${\mathit{v}}_i^{\mathit{\text{Seed}}}$, shown as the red points in Figure 9(a). Between each pair of neighboring seed points ( ${\mathit{v}}_i^{\mathit{\text{Seed}}},{\mathit{v}}_{i+1}^{\mathit{\text{Seed}}}$), an optimal path with minimal cost is searched. The optimal transversal contour is created by connecting all the optimal paths between every pair of seed points ( ${\mathit{v}}_i^{\mathit{\text{Seed}}},{\mathit{v}}_{i+1}^{\mathit{\text{Seed}}}$) on the transversal image, shown as the yellow lines in Figure 9(b).

Figure 9.

Iterations between mesh transformation and contour optimization in a liver segmentation process. The red line is the intersection polylines of mesh Mⁱ on the transversal slice. Red points are seed points re-sampled at an interval of 60 pixels on the intersection polylines. The yellow line is the optimal path calculated using the re-sampled seed points ( ${\mathit{v}}_i^{\mathit{\text{seed}}}$). The initial seed points are replaced by the middle point ( ${\mathit{v}}_i^{\mathit{\text{mid}}}$) at each segment of the optimal path, which is selected as the targeting point and added to the constraint set Ω_i+1. (b) The intersection polylines are replaced by the optimal path re-calculated using middle points ( ${\mathit{v}}_i^{\mathit{\text{mid}}}$) in (a). Please note the re-calculated contour is closer to the liver boundary compared to the initial optimal path in (a). (c) The red mesh is the updated mesh Mⁱ⁺¹ by using the updated constraints Ω_i+1, whereas the yellow mesh is the initial mesh Mⁱ in (a). The default number of iterations is 3.

Due to the lack of local boundary constraint in Ω_i, the initial intersection polylines and the seed points ${\mathit{v}}_i^{\mathit{\text{Seed}}}$ may not closely match the local liver boundary. By contour optimization, the updated transversal contours approximate more accurately to the local boundary than do the initial polylines, as shown the yellow lines in Figure 9(a) and (b) respectively. Thus, we use the middle points ( $v_i^{\mathit{\text{mid}}}$) of each optimal path as the targeting points and replace constraint set Ω_i+1 with them to refine the targeting vertices. Occasionally, in case that the $v_i^{\mathit{\text{mid}}}$ is off the liver boundary, we may adjust or add or delete $v_i^{\mathit{\text{mid}}}$ interactively. In general, approximately 1∼3 pairs of extra correspondences may be needed for one case. Based on the updated constraint set Ω_i+1, a new mesh Mⁱ⁺¹ is calculated using Equation (1). Figure 9(c) compares the difference between the updated mesh Mⁱ⁺¹(red) and the initial mesh Mⁱ (yellow).

IV. Experiments and Results

Forty (40) hepatic CT cases with biopsy-confirmed liver cancers or metastases were retrospectively collected for the evaluation of the accuracy and efficiency of our proposed semi-automated liver segmentation scheme. These hepatic CT cases were obtained by multi-detector CT scanners with the following parameter settings: 2.5–5 mm collimation, 1.25–2.5 mm reconstruction interval, 175 mA tube current, and 120 kVp tube voltage. All cases were acquired with use of an intra-venous contrast agent (ISOVUE; GE Healthcare, Milwaukee, WI).

The livers in these 40 hepatic CT cases were segmented manually by two radiologists in consensus on CT images. The volumetric size of these 40 cancerous livers ranged from 1079.2 CC to 4652.3 CC, in which the tumor volume percentages (ratio of tumor volume/liver volume) ranged from 1.77% to 53.54%. Some of these cases are shown in Figure 1.

The manually-segmented liver volumes were compared with those obtained by the computerized liver volumes segmented by the proposed semi-automated scheme. The computerized liver volumes ranged from 1100.4 CC to 4443.5 CC.

Figure 10 to Figure 12 demonstrate some resulting images of three cases segmented by the proposed liver segmentation scheme. All these cases contain different sizes of tumors. Figure 10 demonstrates a case that was finished without any additional changes of the seeding points, i.e. by using only 5 user-identified liver landmarks. Figure 11 shows a case with a few manual adjustments on couple seed points on three slices. After the first run of the scheme, some seed points on the contours were adjusted manually by the user (see the upper row). After adjustments, the contours were finalized by the second run of the scheme (see the lower rower). Figure 12 shows an example case with large tumors. After the initial mesh segmentation, nine seed points were adjusted manual for contour refinement. It takes less than 5 minutes to get the result shown in Figure 12.

Figure 10.

Resulting liver contours (yellow curves) without additional manual adjusting seeds, i.e. the resulting contours were generated by using 5 liver anatomical landmarks identified by users. The contours of chest wall (blue curves) are generated by an automated segmentation method for chest cavity.

Figure 12.

Resulting contours of a liver with a large amount of tumors. In this case with 57 original CT slices, 9 seeding points (the large red points) were manually adjusted after the initial mesh segmentation. It took less than 90s to adjust the ambiguous seed points. The total time for the entire segmentation is approximately 5 minutes.

Figure 11.

The upper row is the initial contours generated by the scheme with 5 liver landmarks. The arrows indicate the seeds need to be adjusted interactively. The bottom row shows the resulting contours after the adjustments of the seeding points.

We evaluated the performance of the proposed liver segmentation scheme by using the evaluation criterion proposed in [4], which includes five metrics: (1) VOE: volumetric overlap error (%), (2) RVD: relative absolute volume difference (%), (3) ASD: average symmetric surface distance (unit: mm), (4) RSD: root-mean-square of symmetric surface distance (unit: mm), and (5) MSD: maximum symmetric surface distance (unit: mm). According to the criterion [4], a score of 75 is “comparable to human performance” of liver segmentation, which corresponds to VOE=6.4%, RVD=4.7%, ASD=1.0 mm, RSD=1.8 mm, and MSD=19 mm.

The evaluated performance of the proposed scheme is listed in Table 2. Our proposed liver segmentation achieved the average score of 84.75±3.69. This implies that the accuracy of the proposed liver segmentation scheme is superior to that of averaged “human performance”. In average, the proposed semi-automated liver scheme took approximately 5 minutes including about 2 minutes of computing time. It demonstrates that our scheme is accurate and efficient in segmentation of livers considering that our study used diseased liver cases with pathologies including cancers and metastases, which are in general difficult cases compared to those healthy liver cases (such as liver donor cases in transplantation) in the literatures,.

Table 2.

Performance of the proposed liver segmentation scheme in the 40 liver CT cases with cancers or metastases in our study.

Part I (40 cases)	Accuracy					Score
	VOE	RVD	ASD	RSD	MSD	VOE	RVD	ASD	RSD	MSD	Overall
Average	5.97	2.46	0.54	1.15	7.82	76.69	86.91	86.45	83.99	89.71	84.75
Maximum	7.53	5.54	1.24	3.72	25.00	81.79	99.09	92.64	89.84	93.15	89.21
Minimum	4.66	0.17	0.29	0.73	5.20	70.58	70.51	68.89	48.35	67.11	68.64
Std. Dev	0.70	1.02	0.19	0.50	3.37	2.74	5.40	4.76	6.95	4.44	3.69

In order to compare our method with other existing liver segmentation methods in terms of accuracy, running time, ease of use, and consistency, we selected five semi-automated or interactive liver segmentation methods from open-source segmentation software ITK Snap (www.itksnap.org) and Seg3D (www.sci.utah.edu/cibc-software/seg3d.html), and compared our method with these five selected methods by using 10 liver cases downloaded from MICCAI database (www.sliver07.org). We conducted a user study by evaluating the performance of these methods in two user groups: 1) the novice group of five college students, and 2) the expert group of five 3D imaging analysts in radiology department. After one day training of these tools, each participant finished the 10 MICCAI cases using five selected tools as well as our tool. Because interactive manual segmentation methods tends to outperform semi-automated or automated segmentation methods at a high cost of time and effort, all participant were required to apply the same criteria of accuracy in the evaluation of each method. We compared the accuracy scores, the running time, and the easy-of-use index of each method in two groups. The result is showed in Table 3.

Table 3.

Comparison of our method with five liver segmentation methods selected from ITK-Snap (www.itksnap.org) and Seg3D (www.sci.utah.edu/cibc-software/seg3d.html) in terms of accuracy, running time, easy-of-use and consistency by using 10 liver cases downloaded from MICCAI database. We apply an index (1∼5) to indicate easy-of-use: 1 means easiest and 5 means most difficult. Running time is the average of the times spent on 10 cases by 5 participants in each group.

(a) Results of novice group with 5 participants.
Methods and Description	Average Score	Std. Dev.	Running Time (minutes)	Easy-Of-Use
Manual Contouring (ITK Snap)	80.15	1.21	43	1
Region Competition Snakes (ITK Snap)	79.01	1.64	31	4
Edge Based Snake (ITK Snap)	80.01	2.12	29	4
Manual Paint and Contouring (Seg3D)	79.56	1.87	44	2
Speedline (Seg3D)	79.05	1.74	30	3
Our method	79.72	1.17	5	1
Average	79.60	1.63	30.3
(b) Results of expert group with 5 participants.
Methods and Description	Average Score	Std. Dev.	Running Time (minutes)	Easy-Of-Use
Manual Contouring (ITK Snap)	81.25	1.63	32	1
Region Competition Snakes (ITK Snap)	79.85	1.63	20	4
Edge Based Snake (ITK Snap)	80.79	1.60	19	4
Manual Paint and Contouring (Seg3D)	80.26	1.80	31	2
Speedline (Seg3D)	79.67	1.65	20	3
Our method	80.02	1.22	4.7	1
Average	80.30	1.58	21.1

The average accuracy scores of all methods were approximately 80, which indicate the similar levels of accuracy for all tested methods. Compared with the five selected liver segmentation methods, our method demonstrated the highest level of easy-of-use as that of manual contouring, but took only about 5 minutes for finishing a liver segmentation task, which was much more efficient than other methods, in particular about 30-40 minutes for the manual contouring methods. Overall, the expert group outperformed the novice group with higher accuracy score (80.3 vs79.6) and shorter running time (21.2 vs 30.3). However, we observed that the running time in our method was almost constant in both groups, whereas it was approximately 10 min less (about 25%-35%) in expert group than that in novice group for other methods. The low standard deviation of our methods indicate that our liver segmentation scheme achieved a consistent accuracy score among different user groups

V. Discussion and Conclusion

The proposed semi-automated liver segmentation scheme based on iterative mesh transformation combines the advantages of 3D shape continuity and slice-to-slice constraints to achieve an quick and accurate segmentation with minimal user interaction. From our user study, we observed that our method is quicker and more consistent than other methods in both user groups. This outperformance of our method may result from the algorithm and user familiarity with our method.

• The energy-based contour deformable models (such as snakes) requires that the distances between the initial contours (or mesh) and the resulting contours (or mesh) of a ROI must be sufficiently small. User needs to manually initialize contours on each slice in the neighborhood of ROI or use interpolation to estimate the initial contours. The former may increase the interaction time and the latter is prone-to errors which may increase the error-correction time. In addition, deformable models tends to run a large number of iterations such as 100 or 200 iterations to approximate the contours or mesh of a ROI in images.

• Different from energy-based deformation models, 3D mesh transformation model relies on small number of targeting vertices, five landmarks (feature points) in our liver segmentation scheme, to approximate the initial mesh of a ROI. The sparse LU factorization algorithm is very efficient to solve the minimization of the objective function (solve the resulting mesh) and contour optimization also takes O(NLogN) to find the optimized ROI contours on each image. In addition, the mesh is updated automatically every time when user corrects one vertex on a contour, i.e. the error-correcting is propagated to neighboring contours, which may substantially reduce the error-correcting time.

• We had to admit that the authors and the participants in the user study were more familiar with the features of our software than those of other tools. This may cause bias to the user study, in particular the user time. The running time in Table 3 relies on the experience of the image analyst and their familiarity to the software to certain degree. The results might not reflect the best performance of these selected methods.

We acknowledge that the proposed iterative liver segmentation method and the evaluation study have four major limitations.

1. Mapping between a sphere mesh and a ROI boundary: The mesh transformation is established based on the one-to-one homeomorphic mapping between a sphere mesh and a ROI boundary. This assumption of continuous one-to-one mapping may not be easily held in the case of a concave or convex boundary, as shown in Figure 13(a). Because the ROI contour is searched within the band centered by the intersection polyline of the transformed mesh, the resulting contour may miss the concave hole if the band cannot cover the entire hole, as shown in Figure 13(b). Our current solution relies on interactive adding of a targeting point to cover the concave hole. In future, we will replace the sphere mesh with an initial liver mesh that has the same topology including these major concave holes.

2. Parameter setting: There are several important parameters in our liver segmentation scheme that were set empirically, such as the weights in equations (1), (5) and (15). We did not explore the possibility to optimize them further in this study because an optimal setting of the parameters may be related to the specific liver scanning protocols. Thus, we chose the parameters to be general, not optimized for our specific liver scanning protocol. Another important parameter is the bandwidth for searching 2D optimal contours. In our current implementation, we used a bandwidth of 40 pixels in each iteration for simplicity. A large bandwidth may erroneously take the neighboring large gradient boundaries such as bone as the liver boundaries, as pointed by the white arrows in Figure 13(b). An ideal solution is to adjust the bandwidth parameter at each iteration: from a large bandwidth at initial iteration to a small bandwidth at the final iteration.

3. User interaction: In comparison to previous semi-automated segmentation methods, our method reduced the user-interaction to as little as identification of five landmarks. However, users interaction is still needed to review and adjust targeting points on 2D transversal contours after each iteration, in particular at first or second iterations. This is the major interaction time spent in our current segmentation scheme. In future, this interactive identification of five landmarks may be lifted by automated landmark labeling and neighboring organs segmentation. In addition, the selection of constraint targeting points on transversal contours will be refined to minimize the constraint set with the most salient features on the contours.

4. Evaluation study: The hepatic CT images for evaluation were retrospectively collected from a radiochemotherapy study for patients with primary liver cancer or metastasis, and scanned with our institutional abdominal imaging protocol by using an intra-venous contrast agent. These cancer cases may present different texture patterns compared to other liver pathologies such as fatty liver or cirrhosis, and other liver imaging protocols. Further evaluation is required to apply this proposed liver segmentation scheme to hepatic CT images with other liver pathologies.

Figure 13.

(a) The continuous one-to-one mapping between a sphere mesh and a ROI mesh may not be easily held in the case of a concave hole, as indicated by the red arrow lines between the intersection polylines and a sphere, and the shading band area. (b) If the band cannot cover the entire concave hole, the resulting contour may miss part of the boundary of the concave hole, as indicated by the red arrow. On the other hand, large bandwidth may erroneously take the neighboring large gradient boundaries as liver boundary such as the bone or abdominal cavity pointed by the white arrows.

In summary, we proposed a semi-automated segmentation scheme for diseased livers with cancers based on an iterative mesh transformation constrained by optimal transversal contours. The initial evaluation demonstrated that this semi-automated liver segmentation scheme can achieve an accurate and efficient 3D segmentation for diseased livers by using as little as five user-identified landmarks and an average of 5 minutes. Although this scheme was developed in the context of segmentation of diseased liver with tumors in hepatic CT images, this method may provide a general scheme for semi-automated segmentation of other organs or tumors.

Highlights.

• An iterative mesh transformation algorithm for liver segmentation is proposed.

• Multi-resolution transformation optimization controls the 3D shape of a liver mesh.

• Dynamic-programming searches the 2D liver contours on CT images.

• A semi-automated segmentation scheme for diseased livers is developed.

• Interaction is reduced to as little as five user-identified landmarks.

Appendix 1: Solving the minimization problem in Equation (1)

Based on Equation (1) in Section II, mesh transformation is a minimization problem to solve the objective function f(Mⁱ) formed by the weighted sum of three constraints: the smoothing constraint (C_S: Equation (6)), the deforming constraint (C_D: Equation (7)), and the distance constraint (C_L: Equation (8)).

For an edge E_j, let the four vertices around E_j be v_Ej[x], where E_j[x] is the index of the vertices in the mesh, x = 1∼4. Let $t_{E_j}^{(r,c)}$ be the element (r, c) of the 3 × 3matrix [T^S]⁻¹ in Equation (4), where r, c=1∼3. Equation (6) can be represented in the vector form of ${\mathit{v}}_i^{\mathit{\text{ROI}}}$, shown in Equation (A.1).

$${\mathit{C}}_{\mathit{S}}(\mathit{M})=\sum _{\begin{array}{c}\mathit{i}=\mathbf{1},\\ {\mathit{Z}}_{\mathit{i}}\in \mathit{Z}(\mathit{M})\end{array}}^{|\mathit{Z}(\mathit{M})|}\sum _{\begin{array}{c}j\ne k=1,\\ {\mathit{E}}_j\in \mathit{E}({\mathit{Z}}_{\mathit{i}}),\\ {\mathit{E}}_k\in \mathit{E}({\mathit{Z}}_{\mathit{i}})\end{array}}^{\mathbf{3}}{\Vert (\mathit{T}({\mathit{E}}_j)-\mathit{T}({\mathit{E}}_k)\Vert }_F^2=\sum _{\begin{array}{c}\mathit{i}=\mathbf{1}\\ {\mathit{Z}}_{\mathit{i}}\in \mathit{Z}(\mathit{M})\end{array}}^{|\mathit{Z}(\mathit{M})|}\sum _{\begin{array}{c}\mathit{j}\ne \mathit{k}=\mathbf{1},\\ {\mathit{E}}_{\mathit{j}}\in \mathit{E}({\mathit{Z}}_{\mathit{i}}),\\ {\mathit{E}}_{\mathit{k}}\in \mathit{E}({\mathit{Z}}_{\mathit{i}})\end{array}}^{\mathbf{3}}\sum _{\mathit{c}=\mathbf{1}}^{\mathbf{3}}{\Vert \sum _{r=1}^3{t}_{E_j}^{(r,c)}({\mathit{v}}_{E_j[r+1]}^{\mathit{\text{ROI}}}-{\mathit{v}}_{E_j[1]}^{\mathit{\text{ROI}}})-\sum _{r=1}^3{t}_{E_k}^{(r,c)}({\mathit{v}}_{E_k[r+1]}^{\mathit{\text{ROI}}}-{\mathit{v}}_{E_k[1]}^{\mathit{\text{ROI}}})\Vert }^2$$ (A.1)

Let N^S, N^D, N^L be the numbers of the items in Equation (A.1), (7) and (8), respectively. N^S = |Z(M)| × 3 × 3, N^D = |Ω|, N^L = |E(M)|, where |Z(M)| is the number of triangle faces, |Ω| is the number of targeting constraint vertices in the constraint set, and |E(M)| is the number of edges in the mesh.

Therefore, the three constraints C_* (*=S,D,L) can be represented by a linear system:

$${\mathit{C}}_{\ast }={\sum }_{\xi =1}^{N^{\ast }}{\Vert {\mathit{A}}_{\xi }^{\ast }\mathit{V}+{\mathit{B}}_{\xi }^{\ast }\Vert }_{\mathit{F}}^{\mathbf{2}},$$ (A.2)

where ${\mathit{A}}_{\xi }^{\ast }$ is an N* × n matrix, n is the number of vertices in the mesh, and ${\mathit{B}}_{\xi }^{\ast }$ is an N* × 1 vector. For the smooth constraint C_S, based on Equation (A.1), ${\mathit{B}}_{\xi }^{\mathit{S}}$ is a zero vector and ${\mathit{A}}_{\xi }^{\mathit{S}}$ is a sparse matrix as shown in Figure (A.1).

Figure A.1.

Distribution of non-zero elements in ${\mathit{A}}_{\xi }^S$, which is a N^S × n matrix and all the elements except those labeled as e_j or e_k in the ξ^th row are zero. The top labels are the indices of vertices, and the left labels are number of item in Equation (A.1), where ξ = 9i + 3k + c, i, k and c are variables of Equation (A.1). If two indices are equal, for example, E_j[2]=E_k[3]=i, the element in (ξ, i) is e_j[2]+e_k[3].

The non-zero elements in ${\mathit{A}}_{\xi }^S$ can be calculated from Equation (A.1):

$$\begin{array}{c}{\mathit{e}}_{\mathit{j}}[\mathbf{1}]=-{\sum }_{\mathit{r}=\mathbf{1}}^{\mathbf{3}}{\mathit{t}}_{{\mathit{E}}_{\mathit{j}}}^{(\mathit{r},\mathit{c})},{\mathit{e}}_{\mathit{j}}[\mathit{i}]={\mathit{t}}_{{\mathit{E}}_{\mathit{j}}}^{(\mathit{i}-\mathbf{1},\mathit{c})},\mathit{i}=\mathbf{2},\mathbf{3},\mathbf{4}.\\ {\mathit{e}}_{\mathit{k}}[\mathbf{1}]={\sum }_{\mathit{r}=\mathbf{1}}^{\mathbf{3}}{\mathit{t}}_{{\mathit{E}}_{\mathit{k}}}^{(\mathit{r},\mathit{c})},{\mathit{e}}_{\mathit{k}}[\mathit{i}]=-{\mathit{t}}_{{\mathit{E}}_{\mathit{k}}}^{(\mathit{i}-\mathbf{1},\mathit{c})},\mathit{i}=\mathbf{2},\mathbf{3},\mathbf{4}.\end{array}$$ (A.3)

For the deforming constraint C_D, ${\mathit{A}}_{\xi }^D$ and ${\mathit{B}}_{\xi }^D$ in the form of linear system of Equation (A.1) can be represented in Figure (A.2). For the distance constraint C_L, ${\mathit{A}}_{\xi }^{\mathit{L}}$ and ${\mathit{B}}_{\xi }^{\mathit{L}}$ are shown in Figure (A.3).

The distribution of the non-zero elements in ${\mathit{A}}_{\xi }^{\ast }$ and ${\mathit{B}}_{\xi }^{\ast }$ (Figure A.1, A.2 and A.3) indicates that: ${({\mathit{A}}_{\mathit{i}}^{\ast })}^{\mathit{T}}{\mathit{B}}_{\mathit{j}}^{\ast }=0(i\ne j)$. Thus, Equation (A.2) can be rewritten as:

$$\begin{array}{c}{\mathit{C}}_{\ast }=\sum _{\xi }^{{\mathit{N}}^{\ast }}{\Vert {\mathit{A}}_{\xi }^{\ast }{\mathit{V}}^{\mathit{\text{ROI}}}+{\mathit{B}}_{\xi }^{\ast }\Vert }_F^2=\sum _{\xi }^{{\mathit{N}}^{\ast }}({({\mathit{A}}_{\xi }^{\ast }{\mathit{V}}^{\mathit{\text{ROI}}})}^{\mathit{T}}+{({\mathit{B}}_{\xi }^{\ast })}^{\mathit{T}})·({\mathit{A}}_{\xi }^{\ast }{\mathit{V}}^{\mathit{\text{ROI}}}+{\mathit{B}}_{\xi }^{\ast })\\ =\sum _{\xi }^{{\mathit{N}}^{\ast }}({\mathit{V}}^{{\mathit{\text{ROI}}}^{\mathit{T}}}{({\mathit{A}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{A}}_{\xi }^{\ast }{\mathit{V}}^{\mathit{\text{ROI}}}+{\mathit{V}}^{{\mathit{\text{ROI}}}^{\mathit{T}}}{({\mathit{A}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{B}}_{\xi }^{\ast }+{({\mathit{B}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{A}}_{\xi }^{\ast }{\mathit{V}}^{\mathit{\text{ROI}}}+{({\mathit{B}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{B}}_{\xi })\end{array}$$ (A.4)

Figure A.2.

(a). Distribution of non-zero elements in ${\mathbf{A}}_{\xi }^D$, which is a N^D × n matrix and all the elements except element (ξ, i) are zero, where ξ is the index of constraint in Ω. (b) Distribution of non-zero elements in ${\mathit{B}}_{\xi }^D$.

The partial derivatives of C_* is:

$$\frac{\partial {\mathit{C}}_{\ast }}{\partial {\mathit{V}}^{\mathit{\text{ROI}}}}=2(\sum _{\xi }^{{\mathit{N}}^{\ast }}{({\mathit{A}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{A}}_{\xi }^{\ast }){\mathit{V}}^{\mathit{\text{ROI}}}+2(\sum _{\xi }^{{\mathit{N}}^{\ast }}{({\mathit{A}}_{\xi }^{\ast })}^{\mathit{T}}{\mathit{B}}_{\xi }^{\ast })$$ (A.5)

Thus, the partial derivatives of f has the form below:

$$\frac{\mathit{\partial }\mathit{f}}{\mathit{\partial }{\mathit{V}}^{\mathit{\text{ROI}}}}=2({\mathit{w}}_{\mathit{S}}\sum _{\xi }^{{\mathit{N}}^{\mathit{S}}}{({\mathit{A}}_{\xi }^{\mathit{S}})}^{\mathit{T}}{\mathit{A}}_{\xi }^{\mathit{S}}+{\mathit{w}}_{\mathit{D}}{\sum }_{\xi }^{{\mathit{N}}^{\mathit{D}}}{({\mathit{A}}_{\xi }^{\mathit{D}})}^{\mathit{T}}{\mathit{A}}_{\xi }^{\mathit{D}}+{\mathit{w}}_{\mathit{L}}{\sum }_{\xi }^{{\mathit{N}}^{\mathit{L}}}{({\mathit{A}}_{\xi }^{\mathit{L}})}^{\mathit{T}}{\mathit{A}}_{\xi }^{\mathit{L}}){\mathit{V}}^{\mathit{\text{ROI}}}+2({\mathit{w}}_{\mathit{S}}\sum _{\xi }^{{\mathit{N}}^{\mathit{S}}}{({\mathit{A}}_{\xi }^{\mathit{S}})}^{\mathit{T}}{\mathit{B}}_{\xi }^{\mathit{S}}+{\mathit{w}}_{\mathit{D}}\sum _{\xi }^{{\mathit{N}}^{\mathit{D}}}{({\mathit{A}}_{\xi }^{\mathit{D}})}^{\mathit{T}}{\mathit{B}}_{\xi }^{\mathit{D}}+{\mathit{w}}_{\mathit{L}}\sum _{\xi }^{{\mathit{N}}^{\mathit{L}}}{({\mathit{A}}_{\xi }^{\mathit{L}})}^{\mathit{T}}{\mathit{B}}_{\xi }^{\mathit{L}})$$ (A.6)

Finally, the minimization problem in Equation (1) can be solved by evaluating the following equation:

$$\frac{\partial w_0\mathit{f}({\mathit{M}}_{\mathbf{0}})}{\partial {\mathit{v}}^{\mathit{\text{ROI}}}}+\frac{\partial w_1\mathit{f}({\mathit{M}}_{\mathbf{1}})}{\partial {\mathit{v}}^{\mathit{\text{ROI}}}}+\cdots +\frac{\partial w_{\theta }\mathit{f}({\mathit{M}}_{\theta })}{\partial {\mathit{V}}^{\mathit{\text{ROI}}}}=0$$ (A.7)

In terms of Equation (A.6), Equation (A.7) can be converted to a standard vector-formed linear system, which can be solved by using the sparse LU factorization algorithm [28].

Figure A.3.

(a) Distribution of non-zero elements in ${\mathbf{A}}_{\xi }^L$, which is a N^L × n matrix and all the elements except elements (ξ, i₁) and (ξ, i₂) are zero, where i₁ and i₂ are the indices of ${\mathit{v}}_1^{\mathit{\text{ROI}}}$ and ${\mathit{v}}_2^{\mathit{\text{ROI}}}$ in the mesh, and ξ is the index of edge E(M) in the mesh M. (b) Distribution of non-zero elements in ${\mathit{B}}_{\xi }^L$.