THE LAYERED NET SURFACE PROBLEMS IN DISCRETE GEOMETRY AND MEDICAL IMAGE SEGMENTATION

Abstract

Efficient detection of multiple inter-related surfaces representing the boundaries of objects of interest in d-D images (d ≥ 3) is important and remains challenging in many medical image analysis applications. In this paper, we study several layered net surface (LNS) problems captured by an interesting type of geometric graphs called ordered multi-column graphs in the d-D discrete space (d ≥ 3 is any constant integer). The LNS problems model the simultaneous detection of multiple mutually related surfaces in three or higher dimensional medical images. Although we prove that the d-D LNS problem (d ≥ 3) on a general ordered multi-column graph is NP-hard, the (special) ordered multi-column graphs that model medical image segmentation have the self-closure structures and thus admit polynomial time exact algorithms for solving the LNS problems. Our techniques also solve the related net surface volume (NSV) problems of computing well-shaped geometric regions of an optimal total volume in a d-D weighted voxel grid. The NSV problems find applications in medical image segmentation and data mining. Our techniques yield the first polynomial time exact algorithms for several high dimensional medical image segmentation problems. Experiments and comparisons based on real medical data showed that our LNS algorithms and software are computationally efficient and produce highly accurate and consistent segmentation results.

1. Introduction

In this paper, we study the layered net surface (LNS) problems and their extensions in discrete geometry in the d-D space (d ≥ 3 is any constant integer). These problems arise in d-D medical image segmentation and other applications.

Image segmentation, a central problem in medical image analysis, aims to define accurate boundaries for the objects of interest captured by image data. 3-D image segmentation for improved medical diagnosis promises to revolutionize the current medical imaging practice which is inherently 2-D. Although intensive research has been done on 2-D image segmentation in several decades, efficient and effective 3-D image segmentation still poses one of the major challenges in image understanding. In common practice, to identify surfaces representing the boundaries of the sought 3-D objects, first 2-D image slices are more or less analyzed independently; then the 2-D results are stacked together to form the 3-D segmentation output. This approach has some inherent limitations — the most fundamental one stems from the lack of contextual slice-to-slice information when analyzing a sequence of consecutive 2-D images. Performing the segmentation directly on a 3-D image can produce a more consistent segmentation result, yielding 3-D surfaces for object boundaries instead of a set of individual 2-D contours. However, most 2-D image segmentation approaches are difficult to extend to a direct 3-D segmentation, not to mention higher dimensional cases (e.g., a time-series of 3-D images).

We present novel techniques for the LNS problems, i.e., for a simultaneous segmentation of multiple inter-related surfaces in three and higher dimensional medical images. This is an important step towards achieving a practical 3-D surface detection tool since many surfaces in medical images appear in mutual relations. A number of medical imaging problems can benefit from an efficient method for simultaneous detection of multiple inter-related 3-D surfaces.^{11,18,23,24,25,35,19}

We use vascular MR images of femoral arteries to exemplify the segmentation of multiple inter-related surfaces and the underlying constraints. Figure 1(a) is a schematic cross-sectional anatomy of a diseased artery, and Figure 1(b) shows a 2-D cross-sectional vascular MR image of a human femoral artery specimen. A 3-D MR artery image consists of a sequence of such 2-D cross-sections. To detect the vascular layer structures shown in Figure 1(a), we first perform a polar resampling in each 2-D slice along the centerline of the vessel (see Figure 1(b)), and then embed the resampled 2-D geometric space into a 3-D grid ℐ(x, y, z), as in Figure 1(c). This process is called “unfolding”.²² Each of the sought surfaces in ℐ(x, y, z) for the object boundaries (e.g., lumen, internal and external elastic lamina, etc) then contains exactly one voxel in every column of ℐ(x, y, z) that is parallel to the z-axis. Finally, the “unfolded” surfaces in ℐ(x, y, z) are segmented. Some geometric constraints on these surfaces should be satisfied by the segmentation. In this example, pairs of the sought surfaces must be non-crossing and within a given range of distances apart. Since many anatomical structures are smooth, the segmented surfaces must be sufficiently “smooth”. Generally speaking, the smoothness is related to the surface curvature and means that an object boundary cannot change abruptly.

Fig. 1.

Illustrating the “unfolding” operations on the transversal cross-sections of vascular MR images of a human femoral artery specimen. (a) A schematic cross-sectional anatomy of a diseased artery. (b) Performing a polar resampling. (c) Each transversal cross-section is embedded as an xz-slice in the 3-D xyz-space.

The simultaneous detection of multiple inter-related surfaces has been studied by the medical image analysis community for a long time. For the 2-D case, there are several satisfactory results.^{23,22,25,2,29} However, little work has been done on the three and higher dimensional cases. Previous attempts^26,27,8 on extending graph-search based segmentation methods for the 2-D case to identifying even a single optimal surface in 3-D medical images either made the methods computationally intractable or traded their ability to achieve global optima for computational efficiency. Motivated by this segmentation problem, Wu and Chen³⁰ introduced the optimal net surface problems and presented efficient polynomial time exact algorithms for them. But, the algorithms in the paper³⁰ can detect only one optimal surface in 3-D. An implementation of their algorithms and experimental validation based on real 3-D medical images were presented in the paper.¹⁵ More recently, Li et al.¹⁶ extended the approach¹⁵ to segmenting multiple inter-related surfaces in 3-D. However, their new method does not incorporate the very important region information (e.g., homogeneity) for the surface detection.

Modeling the simultaneous detection of multiple inter-related surfaces in high dimensional medical images, we introduce the layered net surface (LNS) problems on an interesting type of geometric graphs, called ordered multi-column graphs, embedded in the d-D discrete space for d ≥ 3 (to be defined in Section 2). We further extend the LNS problems to a more general ordered multi-column graph. Motivated by segmenting anatomical structures with a relatively regular geometric shape, such as the left ventricles, kidneys, livers, and lungs, we also study several net surface volume (NSV) problems, which aim to find well-shaped regions of an optimal “volume” in a d-D weighted voxel grid. These well-shaped geometric regions are closely related to monotonicity and convexity in d-D discrete spaces (Section 6). Our main results in this paper are summarized as follows.

• We develop an efficient algorithm for solving the LNS problem on an interesting type of ordered multi-column graphs in polynomial time, by formulating it as computing a minimum-cost closed set in a vertex-weighted directed graph.

• We prove that the LNS problem on a general ordered multi-column graph is NP-hard. However, the (special) ordered multi-column graphs that model medical image segmentation applications have additional properties, and the LNS problem on such graphs is polynomially solvable.

• We extend our LNS technique to solving the NSV problems of computing several classes of optimal well-shaped geometric regions in a d-D weighted voxel grid. These NSV problems arise in data mining,^9,7 image segmentation,² and data visualization.¹ The classes of regions that we study can be viewed as generalizations of some of the pyramid structures in the paper.⁷

• We apply our polynomial time LNS algorithms to segmenting multiple inter-related object boundaries in 3-D medical images. Our method can be easily extended to higher dimensional image segmentation. We also show implementation and experimental results of our algorithms.

Our LNS approach was inspired by Wu and Chen’s algorithms³⁰ for solving the optimal net surface problems. However, instead of searching for a single optimal surface as in the paper,³⁰ our algorithms concurrently identify multiple optimal inter-related surfaces. Our approach is based on a characterization of the self-closure structures of the underlying graphs and a sophisticated graph transformation scheme, in order to enforce the constraints of the LNS problems and to ensure the optimality of the solutions. The extension of our LNS approach to the NSV problems is quite elegant. To compute a sought optimal d-D region, we exploit the monotonicity property of the target region and transform in a nontrivial fashion the NSV problem to the LNS problem.

The rest of the paper is organized as follows. In Section 2, we formally define the layered net surface (LNS) problems. The modeling of medical image segmentation as our optimal LNS problems are discussed in Section 3. Our algorithm for solving the LNS problem is presented in Section 4. We prove the NP-hardness of the LNS problem on a general ordered multi-column graph in Section 5 and show the problem on a special ordered multi-column graph is polynominal-time solvable. Section 6 gives extensions of our LNS techniques to solving the NSV problems. Finally, our implementation and experimental results are shown in Section 7.

2. The Layered Net Surface (LNS) Problems

A multi-column graph G = (V, E) embedded in the d-D discrete space is defined as follows. For a given undirected graph B = (V_B, E_B) embedded in (d − 1)-D (called the net model) and an integer κ > 0, G is an undirected graph in d-D generated by B and κ. For each vertex v = (x₀, x₁, …, x_d−2) ∈ V_B, there is a sequence Col(v) of κ vertices in G corresponding to v; Col(v) = {(x₀, x₁, …, x_d−2, k) | k = 0, 1, …, κ−1}, called the v-column of G. We denote the vertex (x₀, x₁, …, x_d−2, k) of Col(v) by v_k. If an edge (v, u) ∈ E_B, then we say that the v-column and u-column in G are adjacent to each other. For each vertex v_k ∈ Col(v), v_k has edges in G to a non-empty list of consecutive vertices in every adjacent u-column Col(u) of Col(v), say u_k′, u_k′+1, …, u_k′+s (s ≥ 0); we call (u_k′, u_k′+1, …, u_k′+s), in this order, the edge interval of v_k on Col(u), denoted by I(v_k, u). For an edge interval I, we denote by Bottom(I) (resp., Top(I)) the d-th coordinate of the first (resp., last) vertex in I (e.g., Bottom(I(v_k, u)) = k′ and Top(I(v_k, u)) = k′ + s in the above example).

Two adjacent columns Col(v) and Col(u) in G are said to be in proper order if for any two vertices v_k and v_k+1 in Col(v), Bottom(I(vk, u)) ≤ Bottom(I(v_k+1, u)) and Top(I(v_k, u)) ≤ Top(I(v_k+1, u)), and if the same holds for any two vertices u_k and u_k+1 of Col(u) on Col(v). The corresponding edge (v, u) ∈ E_B is called a proper edge. If all pairs of adjacent columns in G are in proper order, then we call G a properly ordered multi-column graph (briefly, a properly ordered graph). Figures 2(a)–2(b) show a net model and a properly ordered graph.

Fig. 2.

(a) A 2-D net model B. (b) A 3-D properly ordered multi-column graph G generated by B and κ = 4 (the edges between Col(u_i) and Col(u_i+1), i = 0,1, 2, are symmetric to those between Col(v_i) and Col(v_i+1), and the edges between Col(v_j) and Col(u_j), j = 1, 2, are symmetric to those between Col(v₃) and Col(u₃); all these edges are omitted for a better readability). (c) Two (l, 2)-separate net surfaces in G marked by heavy edges. (d) Two net surfaces divide the vertex set of G into three disjoint vertex subsets.

Note that in medical image segmentation, the boundaries of the target objects (e.g., organs) are often sufficiently “smooth”. The smoothness constraint on the sought surfaces is modeled by the proper ordering of the edges in a multi-column graph G, that is, the edges connecting each vertex v_k in G to every adjacent column Col(u) of Col(v) form a vertex interval on Col(u), and such vertex intervals for any two adjacent columns of G are in proper order.

A net surface in G (also called a net) is a subgraph of G defined by a function : V_B → {0,1, …, κ − 1}, such that for every edge (v, u) ∈ E_B, (v_k′, U_k″), with k′ = (v) and k″ = (u), is also an edge in E. For simplicity, we denote a net by its function . Intuitively, a net in G is a special mapping of the (d − 1)-D net model B to the d-D space, such that “intersects” each v-column of G at exactly one vertex and preserves all topologies of B. can be viewed as a functional “surface” of B in d-D defined on the (d − 1)-D space in which B is embedded.

Given two integers L and U, 0 < L < U, two nets and of a properly ordered graph G are said to be (L, U)-separate if L ≤ (v) − (v) ≤ U for every vertex v ∈ V_B. Roughly speaking, and do not cross each other and are within a specified range of distance to each other. Figure 2(c) shows two net surfaces. For a given set of l − 1 integer parameter pairs {(L_i, U_i) | 0 < L_i < U_i, 1 ≤ i < l}, l ≥ 2, we consider l net surfaces = {, , …, } in G such that is “on top” of (i.e., ∀ v ∈ V_B, (v) > (v)), and and are (L_i, U_i)-separate (i = 1,2, …, l − 1). Then, these l net surfaces partition the vertex set V of G into l + 1 disjoint subsets R_i, with R₀ = {v_k | v ∈ V_B, 0 ≤ k ≤ (v)}, R_i; = {v_k | v ∈ V_B, (v) < k ≤ (v)} for i = 1,2, …, l −1, and R_l = {v_k | v ∈ V_B, (v) < k ≤ κ} (see Figure 2(d)).

Motivated by medical image segmentation,^22,5,6,36,19 we assign costs to every vertex of G as follows. Each vertex V_k ∈ V has an on-surface cost b(v_k), which is an arbitrary real value. For each region R_i (i = 0,1, …, l), every vertex v_k ∈ R_i ⊆ V is assigned a real-valued in-region cost c_i(v_k). The on-surface cost of each vertex in G is inversely related to the likelihood that it may appear on a desired net surface, while the in-region costs C_i(·) (i = 0,1, …, l) measure the inverse likelihood of a given vertex preserving the expected regional properties of the partition {R₀, R₁, …, R_l}. Both the on-surface and in-region costs for image segmentation can be determined by using simple low-level image features.^22,36,33,19

The layered net surface (LNS) problem seeks l net surfaces = {, , …, } in G such that the total cost α() induced by the l net surfaces in , with

$$\alpha (\mathcal{N}\mathcal{S})=\sum _{i=1}^{l}b({\mathcal{N}}_i)+\sum _{i=0}^l{c}_i(R_i)=\sum _{i=1}^l\sum _{u\in V({\mathcal{N}}_i)}b(u)+\sum _{i=0}^l\sum _{u\in R_i}{c}_i(u),$$

is minimized, where V(H) denotes the vertex set of a graph H.

In fact, our algorithmic framework is general enough for the cases in which each vertex has only an on-surface cost, only in-region costs, or both. We illustrate our unified approach as an example for the case where each vertex has both the on-surface and in-region costs.

In this paper, we also consider the layered net surface problem on a more general ordered multi-column graph, defined as follows. Note that any two adjacent columns of a properly ordered multi-column graph are in proper order. We now define the reverse order on two adjacent columns Col(v) and Col(u) in a d-D multi-column graph G = (V, E) generated by a (d − 1)-D net model B = (V_B, E_B): If for any two vertices V_k and V_k+1 in Col(v), Bottom(I(v_k, u)) ≥ Bottom(I(v_k+1, u)) and Top(I(v_k, u)) ≥ Top(I(v_k+1, u)), and if the same holds for any two vertices u_k and u_k+1 of Col(u) on Col(v), then we say that Col(u) and Col(v) are in reverse order. The corresponding edge (v, u) ∈ E_B is called a reverse edge. If every two adjacent columns in G are in either proper order or reverse order, then we call G a d-D ordered multi-column graph. Further, for two (L, U)-separate nets and in G, if two adjacent columns Col(v) and Col(u) are in reverse order, then L ≤ (v) − (v) ≤ U and L ≤ (u) − (u) ≤ U. We prove that the LNS problem on a general d-D ordered multi-column graph (d ≥ 3) is NP-hard (see Section 5).

Interestingly, if the ordered multi-column graph is generated by a special net model, which also models key medical image segmentation problems(see Section 3), the LNS problem is polynomially solvable. The special net model B = (V_B, E_B) is defined in the following way. First, remove from B all reverse edges; the remaining B is a set of connected components with proper edges only. Then, contract each connected component of into a single vertex. Finally, for each (removed) reverse edge (v, u) ∈ E_B, say, v in C′ ∈ and u in C″ ∈ (C′ = C″ is possible), add an edge between the contracted vertices of C′ and C″. The resulting graph is called the p-contracted graph of B. The bipartite LNS (BLNS) problem is defined on a d-D ordered multi-column graph with a net model B whose p-contracted graph is bipartite. For the BLNS problem, we naturally assume that each vertex in G has only an on-surface cost.

3. Modeling Medical Image Segmentation

This section discusses the modeling of medical image segmentation problems. Numerous medical applications can benefit from simultaneous detection of multiple optimal interacting surfaces representing terrain-like or tubular structures, which can be modeled as our optimal LNS problems. If we also consider the motion of these anatomical structures, then we have a time-series of 3-D images, forming a 4-D image.

It is easy to embed a 3-D image with a terrain-like object into the 3-D space. Let ℐ(x, y, z) = {(x, y, z) | 0 ≤ x < X, 0 ≤ y < Y, 0 ≤ z < Z} be a 3-D image of size X × Y × Z, where X, Y, and Z denote the image size in the x, y, and z dimensions, respectively. Each desired terrain-like boundary surface contains exactly one voxel ℐ(x, y, z) in each column of ℐ(x, y, z) that is parallel to the z-axis. Since many anatomical structures are smooth, one may expect the resulting surfaces to be sufficiently “smooth”. The smoothness constraints guarantee the surface continuity in 3-D. Specifically, we use two smoothness parameters, Δ_x and Δ_y, to specify the maximum allowed change in the z-coordinate of a feasible surface along each unit distance change in the x and y directions, respectively. Note that here we assume a 4-neighbor adjacency (the modeling described below can be easily extended to other adjacency settings). In addition, assume that we are looking for l boundary surfaces and any two adjacent desired surfaces and are within a certain range of distances apart (e.g., for all 0 ≤ x < X and 0 ≤ y < Y, ${\delta }_i^l\le {\mathcal{S}}_{i+1}(x,y)-{\mathcal{S}}_i(x,y)\le {\delta }_i^u$). A cost c(x, y, z) is assigned to each voxel ℐ(x, y, z) such that the cost is inversely related to the likelihood that a desired surface would contain the voxel, which is usually determined by using simple low-level image features.²² The multiple surface detection problem seeks l feasible surfaces in ℐ whose total sum of voxel costs is minimized.

This (terrain-like) surface detection problem can be modeled as a layered net surface problem on a properly ordered graph G = (V, E), as follows. The net model of G is a 2-D X × Y grid B. Each voxel ℐ(x, y, z) corresponds to exactly one vertex v(x, y, z) in G. Thus, any grid point (x, y) in B is associated with a column Col(x, y) of Z vertices of G, i.e., Col(x, y) = {v(x, y, z)|z = 0,1, …, Z − 1}. Then, for every vertex v(x, y, z) ∈ V with x < X − 1 (resp., y < Y − 1), there is an edge to vertex v(x + 1, y, z′) (resp., v(x, y + l, z′)) for any z′ such that 0 ≤ z′ < Z and |z′ − z| ≤ Δ_x (resp., |z′ − z| ≤ Δ_y). These edges are used to enforce the smoothness constraint of the target surfaces along the x-dimension (resp., y-dimension). Each vertex v(x, y, z) has an on-surface cost equal to the cost c(x, y, z) of the corresponding voxel ℐ(x, y, z). It is straightforward to show that G is a properly ordered graph defined on the net model B. We are then looking for l net surfaces {, …, } in G such that , and are ( ${\delta }_i^l,{\delta }_i^u$)-separate for i = 1,2, …, l − 1. Hence, this surface detection problem is modeled as a layered net surface problem on the properly ordered graph G. Note that in this application, we do not consider the region-based information for the segmentation though it is easy to be incorporated into G.

To segment a tubular structure in a 3-D image , we need to first “unfold” it, and then embed it into the 3-D xyz-space to obtain an image ℐ(x, y, z). In common medical practice, there are two useful unfolding approaches: One is based on the transversal cross-sections (TCS) of the tubular object; the other is based on the sagittal cross-sections (SCS) (see Figure 3(a)).

Fig. 3.

Illustrating the sagittal cross-section based unfolding method, (a) The sagittal and transversal cross-sections of a tubular object, (b) A transversal cross-section of a 3-D intravascular ultrasound vessel image. Resampling is performed at each angle for all transversal cross-sections to form a sagittal cross-section, (c) A sagittal cross-section of a 3-D intravascular ultrasound image at an angle θ_i. (d) A schematic unfolded 3-D image ℐ(x, y, z). Each sagittal cross-section at an angle θ_i is embedded as a yz-slice of ℐ at x = θ_i. (e) Splitting ℐ into two sub-images and . (f) The net model B for ℐ.

In the TCS-based approach, as illustrated in Figure 1, each transversal cross-section is unfolded by performing a polar resampling with respect to the center point O, and is embedded into an xz-plane, called an xz-slice of ℐ. Hence, any two adjacent columns on a same xz-slice satisfy the smoothness constraints. Note that on each xz-slice, the last column is treated as being adjacent to the first column, thus satisfying the smoothness constraints as well. Additionally, the corresponding columns on any two adjacent xz-slices also need to meet the smoothness constraints in the y-direction. Next, by applying the same modeling to ℐ(x,y,z) as for the terrain-like object, we transform this problem into that of computing multiple optimal interacting net surfaces in a properly ordered graph G, whose net model B is a 2-D grid with a wraparound in the x-dimension.

In the SCS-based method, each sagittal cross-section of the tubular structure, which consists of a sequence of profiles from all transversal cross-sections at the same angle, corresponds to a yz-slice of ℐ. Figure 3(b) shows a transversal cross-section of a 3-D intravascular ultrasound vessel image, in which each angle defines a sagittal cross-section. The sagittal cross-section corresponding to angle θi in Figure 3(b) is illustrated in Figure 3(c) and is embedded as a yz-slice of ℐ at x = θi (see Figure 3(d)). Note that the centerline of the tubular object is on every sagittal cross-section, thus forming the so-called O-plane in ℐ. The O-plane splits a desired tubular surface in the original image into two sub-surfaces, the upper sub-surface and lower sub-surface, as shown in Figure 3(d). To model this problem, we decompose the image ℐ along the O-plane into two sub-images and (see Figure 3(e)), each of which is used for searching the upper sub-surfaces and the lower sub-surfaces, respectively. One may search for the upper sub-surfaces and the lower sub-surfaces in and separately, by using our algorithm for the LNS problem. The net models B₁ and B₂ for and , respectively, are a 2-D grid with only proper edges. However, the upper (resp., lower) surface on the last sagittal cross-section (i.e., at x = θ_X−1 in Figure 3(e)) needs to be smoothly connected with the lower (resp., upper) surface on the first sagittal cross-section (i.e., at x = θ₀ in Figure 3(e)). To enforce this smoothness constraint, we introduce a reverse edge between the corresponding vertices on the last column of B₁ (resp., B₂) and the first column of B₂ (resp., B₁) (see Figure 3(f)), thus forming a net model B. Obviously, the p-contracted graph of B is bipartite. Hence, we model this segmentation problem as solving a BLNS problem.

4. Algorithm for the Layered Net Surface (LNS) Problem

This section gives our polynomial time algorithm for the layered net surface problem on a d-D properly ordered graph G = (V, E). We first exploit the self-closure structure of the LNS problem, and then model it as a minimum-cost closed set problem based on a nontrivial graph transformation scheme.

A closed set in a directed graph with arbitrary vertex costs w(·) is a subset of vertices such that all successors of any vertex in are also contained in .^20,14 The cost of a closed set , denoted by w(), is the total cost of all vertices in . Note that a closed set can be empty (with a cost zero). The minimum-cost closed set problem seeks a closed set in the graph whose cost is minimized.

4.1. Overview of the LNS algorithm

Our LNS algorithm is based on a sophisticated graph transformation scheme, which enables us to simultaneously identify l > 1 optimal inter-related net surfaces as a whole by computing a minimum closed set in a weighted directed graph G′ that we transform from G. The algorithm uses the following three main steps.

Step 1: Graph Construction

Build a vertex-weighted directed graph G′ = (V′, E′), which contains l vertex-disjoint subgraphs $G_i^{\prime }=(V_i^{\prime },E_i^{\prime })$. Each subgraph $G_i^{\prime }$ is used to search for the i-th net surface in ℐ. The separation constraints of the net surfaces are enforced by introducing a subset of directed edges between any two adjacent subgraphs, $G_i^{\prime }$ and $G_{i+1}^{\prime }$ (i = 1,2,…, l−1). The construction of the graph G (see Section 4.3) hinges on the self-closure structure exploited in Section 4.2.

Step 2: Computing a Minimum-cost Closed Set

Compute a minimum-cost non-empty closed set *in G′, which can be done by formulating it as computing a minimum s-t cut in an edge-weighted directed graph transformed from G′.

Step 3: Net Surfaces Reconstruction

The set of l optimal net surfaces is reconstructed from the minimum-cost closed set * with each net surface being specified by ${\mathcal{C}}^{\ast }\cap V_i^{\prime }$.

4.2. The self-closure property of the LNS problem

Our algorithm for the LNS problem hinges on the following observations about the self-closure structure of any feasible LNS solution. Recall that in a set of l feasible net surfaces = {, , …, } in G, is “on top” of , for each i = 1,2,…,l−1.

For a vertex v_k ∈ V (i.e., v ∈ V_B and 0 ≤ k < κ) and each adjacent column Col(u) of Col(v) (i.e., (v,u) ∈ E_B), the lowest-neighbor of v_k on Col(u) is the vertex in Col(u) with the smallest d-th coordinate that has an edge to v_k in G (i.e., the vertex in Col(u) with the smallest d-th coordinate that can possibly appear together with v_k on a same feasible net surface in G). In Figure 4(a), the lowest-neighbor of v₂ (resp., u₂) on Col(u) (resp., Col(v)) is u₂ (resp., v₁).

Fig. 4.

(a) Illustrating the proper ordering of the edges in G. (b) Constructing a subgraph $G_i^{\prime }$ from G. (c) Incorporating the surface separation constraints into the construction of $G_i^{\prime }$ and $G_{i+1}^{\prime }$ (with Li = 1 and U_i = 2).

Given the surface separation constraints, we define below the upstream and downstream vertices of any vertex in G, to help characterize the spatial relations between feasible net surfaces in G. For every vertex v_k ∈ V and 1 ≤ i < l (resp., 1 < i ≤ l), the i-th upstream (resp., downstream) vertex of v_k is v_k+Li (resp., v_{max{0,k−Ui−1}}) if k + L_i < κ (resp., k − L_i−1 ≥ 0). Intuitively, if v_k ∈ , then the i-th upstream (resp., downstream) vertex of v_k is the vertex in Col(v) with the smallest d-th coordinate that can be on (resp., ).

We say that a vertex v_k is below (resp., above) a net surface if (v) > k (resp., (v) < k), and denote by LO() the subset of all vertices of G that are on or below . For every vertex v_k ∈ LO(), consider its lowest-neighbor u_k′ on any adjacent Col(u) of Col(v). Let r = (v) and u_k″ be the lowest-neighbor of v_r on Col(u) (v_r ∈ Col(v) is on ). Then by the definition of net surfaces, k″ ≤ (u). Since k ≤ (v), we have k′ ≤ k″ due to the proper ordering. Thus, k′ ≤ (u), and further, u_k′ ∈ LO(). Hence, we have the following observation.

Observation 1

For any feasible net surface in G, if a vertex v_k is in LO(), then every lowest-neighbor of v_k is also in LO().

Observation 1 characterizes the self-closure property of every set LO(). However, our task is more involved since the l net surfaces in are inter-related. We thus need to further examine the closure structure between the LO()’^S. Consider any vertex v_k ∈ LO(). Since k ≤ (v), the i-th upstream vertex of is “above” the i-th upstream vertex of v_k (i.e., (v) + L_i ≥ k + L_i). In the meanwhile, (v) ≥ (v) + L_i (by the definition of the i-th upstream vertex). Hence, the i-th upstream vertex of v_k, v_k+Li, is in LO(). Using a similar argument, the i-th downstream vertex of v_k, v_{max{0,k−Ui−1}} is in LO(). Thus, the observation below follows.

Observation 2

Given any set = {, ,… } of l feasible net surfaces in G, the i-th upstream (resp., downstream) vertex of each vertex in LO() is in LO() (resp., LO()), for every 1 ≤ i < l (resp., 1 < i ≤ l).

Observations 1 and 2 show an important self-closure structure of the LNS problem, which is crucial to our LNS algorithm and suggests a connection between our target problem and the minimum-cost closed set problem.^20,14 In our LNS approach, instead of directly searching for an optimal set of l net surfaces, $\mathcal{N}{\mathcal{S}}^{\ast }=\{{\mathcal{N}}_1^{\ast },{\mathcal{N}}_2^{\ast },\dots ,{\mathcal{N}}_l^{\ast }\}$, we look for l optimal subsets of vertices in G, $LO({\mathcal{N}}_1^{\ast })\subset LO({\mathcal{N}}_2^{\ast })\subset \dots \subset LO({\mathcal{N}}_l^{\ast })$, such that each $LO({\mathcal{N}}_i^{\ast })$ uniquely defines the net surface ${\mathcal{N}}_i^{\ast }\in \mathcal{N}{\mathcal{S}}^{\ast }$.

4.3. The graph transformation scheme

This section presents the construction of the vertex-weighted directed graph G′ = (V′, E′) from the d-D properly ordered graph G = (V, E), which enables us to simultaneously identify l > 1 optimal inter-related net surfaces as a whole by computing a minimum closed set. This construction crucially relies on the self-closure structure shown in Section 4.2.

The graph G′ contains l vertex-disjoint subgraphs $\{G_i^{\prime }=(V_i^{\prime },E_i^{\prime })\mid i=1,2,\dots ,l\}$; each $G_i^{\prime }$ is for the search of the i-th net surface . $V^{\prime }={\cup }_{i=1}^l{V}_i^{\prime }$ and $E^{\prime }={\cup }_{i=1}^l{E}_i^{\prime }\cup E_s^{\prime }$. The surface separation constraints between any two consecutive net surfaces and are enforced in G′ by a subset of edges in $E_s^{\prime }$, which connect the corresponding subgraphs $G_i^{\prime }$ and $G_{i+1}^{\prime }$.

We first show the construction of every $G_i^{\prime }=(V_i^{\prime },E_i^{\prime })$ (i = 1,2, …,l) from G. For $G_i^{\prime }$, each vertex v_k in G corresponds to exactly one vertex $v_k^i\in V_i^{\prime }$. Note that $G_i^{\prime }$ is used for the search of LO() that defines the i-th net surface . If vertex v_k ∈ Col(v) appears on , then all vertices of Col(v) “below” v_k (i.e., those vertices of Col(v) whose d-th coordinate is ≤ k) are in LO(). Thus, for each v ∈ V_B of the net model B and k = 1,2,…,κ − 1, vertex $v_k^i$ has a directed edge ( $v_k^i,v_{k-1}^i$), forming a chain $v_{\kappa -1}^i\to v_{\kappa -2}^i\to \cdots \to v_0^i$ in $G_i^{\prime }$ for Col(v), denoted by Ch_i(v). These chains help capture the closure property of LO(). Two chains Ch_i(v) and Ch_i(u) in $G_i^{\prime }$ are said to be adjacent if their corresponding columns Col(v) and Col(u) in G are adjacent. Next, we put directed edges between any two adjacent chains in $G_i^{\prime }$ to ensure the feasibility of the net surface . For every vertex v_k of each Col(v) in G and its lowest-neighbor u_k′ on each adjacent Col(u) of Col(v), we put into $E_i^{\prime }$ a directed edge from $v_k^i\in C{h}_i(v)$ to $u_{k^{\prime }}^i\in C{h}_i(u)$. The role of this edge is to ensure that if v_k ∈ , then must contain a vertex in Col(u) that is no “lower” than u_k′ (i.e., (u) ≥ k′). This is because a feasible containing v_k can only contain a vertex in the edge interval I(v_k,u) of v_k on Col(u), i.e., I(v_k,u) = (u_k′, u_k′+1, …, u_k′+s) (s ≥0). It might appear that adding this edge to $G_i^{\prime }$ might not do the job. But, based on the proper ordering in G, we are able to prove that our graph transformation well enforces the feasibility of the net surfaces (see Lemmas 4 and 5). Figure 4(a) shows the part of the graph G associated with an edge (v,u) ∈ E_B, and Figure 4(b) illustrates the corresponding construction in $G_i^{\prime }$. This completes the construction of each $G_i^{\prime }$.

We now put directed edges into $E_s^{\prime }$ between $G_i^{\prime }$ and $G_{i+1}^{\prime }$, to enforce the surface separation constraints. Based on Observation 2, if vertex v_k ∈ , then its i-th upstream vertex v_k+Li must be on or below the net surface (i.e., v_k+Li ∈ LO()). Thus, for each vertex $v_k^i$ with k < κ − L_i on the chain Ch_i(v) in $G_i^{\prime }$, a directed edge is put in $E_s^{\prime }$ from $v_k^i$ to $v_{k+L_i}^{i+1}$ on Ch_i+1(v) in $G_{i+1}^{\prime }$ (see Figure 4(c)). Intuitively, these edges ensure that the net surface must be at a distance of at least L_i “above” (i.e., for each v ∈ V_B, (v) − (v) ≥ L_i). On the other hand, each vertex $v_k^{i+1}$ with k ≥ L_i on Ch_i+1(v) has a directed edge in $E_s^{\prime }$ to $v_{k^{\prime }}^i$ on Ch_i(v) with k′ = max{0, k − U_i} (note that v_k′ in G is the (i + 1)-th downstream vertex of v_k), making sure that must be at a distance of no larger than U_i “above” (i.e., for each v ∈ V_B, (v) − (v) ≤ U_i). Note that in this construction, any vertex $v_k^i$ with k ≥ κ − L_i has no edge to any vertex on Ch_i+1(v), forming the upper deficient vertex segment udv_i(v) of Ch_i(v), and any vertex $v_k^{i+1}$ with k < L_i has no edge to any vertex on Ch_i(v), forming the lower deficient vertex segment ldv_i+1 (v) of Ch_i+1(v). Each of these vertices of G′ is called a deficient vertices. This construction is applied to every pair of the corresponding chains of any two subgraphs $G_i^{\prime }$ and $G_{i+1}^{\prime }$, for i = 1, 2, …, l −1.

Recall that we aim to compute a minimum-cost non-empty closed set in G′, which can specify l optimal net surfaces in G. However, the graph G′ constructed up to this point does not yet work for this purpose. When a closed set in G′ includes only deficient vertices of a chain in G′, which is possible, it does not correspond to l feasible net surfaces. The reader may notice that for any deficient vertex in G′, the corresponding vertex in G cannot lie on a feasible net surface. Hence, it might seem safe to remove all deficient vertices from G′. However, this removal of deficient vertices may in turn cause other vertices in G′ to become deficient. (A vertex $v_k^i$ on Ch_i(v) is said to be deficient if either no vertex in $G_{i-1}^{\prime }$ or $G_{i+1}^{\prime }$ connects with it, or there exists an edge (v, u) ∈ E_B but no vertex on Ch_i(u) of $G_i^{\prime }$ is adjacent to it.) Moreover, we cannot simply delete all edges incident with the deficient vertices.

By utilizing the proper order of graph G, we develop a deficient vertex pruning scheme, as follows. Instead of removing the deficient vertices one by one, we grow the deficient vertex segments until no further changes can occur, and then safely remove all deficient vertex segments and their edges. We first consider the lower deficient vertex segment ldv_i(v) of each chain Ch_i(v) in G′. A key observation here is that when pruning the lower deficient vertex segments, if a vertex $v_k^i$ on Ch_i(v) is deficient, then all vertices $v_{k^{\prime }}^i$ on Ch_i(v) “below” $v_k^i$ (i.e., with k′ ≤ k) are also deficient. Hence, the deficient vertices on every Ch_i(v) always form a continuous segment and it is sufficient to consider the “topmost” deficient vertex $v_t^i$ on ldv_i(v) (i.e., $\forall v_k^i\in {\mathit{ldv}}_i(v)$, k ≤ t). For each column Col(u) adjacent to Col(v), recall that I(v_t, u) denotes the edge interval of v_t on Col(u) in G. If I(v_t, u) ⊆ ldv_i(u), then the removal of ldv_i(v) does not cause additional deficient vertices on Ch_i(u). Otherwise, consider the edge interval I(v_t+1, u) of v_t+1 on Col(u). Let q be the smallest d-th coordinate of the vertices in I(v_t+1, u). Due to the proper order of G, no vertices v_k with k > t have an edge connecting with u_q−1 ∈ Col(u). Thus, the vertex $u_{q-1}^i$ is deficient since the edge (v, u) ∈ E_B but the removal of all incident edges of the vertices in ldv_i(v) leaves no edge connecting $u_{q-1}^i$ with any vertex on Ch_i(v) in $G_i^{\prime }$. Let $u_{t^{\prime }}^i$ be the topmost deficient vertex on ldv_i(u). If q − 1 > t′, then the lower deficient vertex segment ldv_i(u) now grows up to $u_{q-1}^i$. Note that for each vertex $u_k^i$ with k ≥ q and u_k ∈ I(v_t, u), if u_k is on a feasible surface, then the vertex of Col(v) on that surface must be no lower than v_t+1; in fact, the edge (v_t+1, u_k) is in G. Thus, a directed edge from $u_k^i$ to $v_{t+1}^i$ is put in $G_i^{\prime }$ to enforce this relation. This pruning procedure is also applied to Ch_i(v) with respect to Ch_i+1(v) and Ch_i−1 (v). When no further changes are needed on the lower deficient vertex segments of G′, we just remove all these segments and their incident edges. Actually, we can maintain a queue to keep the chains of G′ with a changed lower deficient vertex segment. When eventually the queue is empty, it means that no lower deficient vertex segments will be further changed. The upper deficient vertex segments in G′ are handled in a similar way. It is not hard to see that the following lemma holds.

Lemma 1

The deficient vertex pruning scheme runs in O(|E′|) time on G′ = (V′, E′).

We simply denote the graph thus resulted also by G′. Note that in G′ if any chain Ch_i (v) = , then there is no feasible solution to the LNS problem. In the rest of this section, we assume that the LNS problem has feasible solutions. Then, for every v ∈ V_B and i = 1, 2, …, l, let μ_i(v) and κ_i(v) be the smallest and largest d-th coordinates of the vertices on the chain Ch_i(v) of $G_i^{\prime }$, respectively. We denote by Z₀ the set of the “lowest” vertices (i.e., with the smallest d-th coordinate) on every chains of G′ and by H₀ the induced subgraph of Z₀ in G′. Our above construction of G′ ensures the following lemma.

Lemma 2

(1) H₀ is a strongly connected component of G′. (2) Z₀ is a closed set in G′. (3) For any non-empty closed set in G′, Z₀ ⊆ .

Up to this point, the graph construction establishes the connection between a closed set in G′ and l feasible net surfaces in G. Our goal is to compute a non-empty minimum-cost closed set in G′, which can specify l optimal nets in G. Thus, we need to further assign a cost w(·) to each vertex in G′.

Note that the cost of is $\alpha (\mathcal{N}\mathcal{S})={\sum }_{i=1}^{l}b({\mathcal{N}}_i)+{\sum }_{i=0}^l{c}_i(R_i)$, where R_i’s are the disjoint subsets of V partitioned by the nets in . Further, recall that we are looking for LO() instead of directly, by using the subgraph $G_i^{\prime }$. Thus, we want to find a way to assign vertex costs to $V_i^{\prime }$, so that for any feasible in G, the cost b() can be “distributed” to those vertices of $G_i^{\prime }$ corresponding to LO() (i.e., ${\sum }_{v_k\in LO({\mathcal{N}}_i)}w(v_k^i)=b({\mathcal{N}}_i)$). We may go one step further to require that for any vertex v_r ∈ Col(v) in G, the total cost of the vertices $v_k^i$ on Ch_i(v) of $G_i^{\prime }$ with k ≤ r (i.e., ${\sum }_{k\le r}w(v_k^i)$) be equal to the cost of v_r. In addition, note that R₀ = LO(), R_i =LO() − LO() (i = 1, 2, …, l − 1), and R_l = V − LO(), and consider the removal of the deficient vertices as well. Thus, we have the following vertex-cost assignment scheme for each subgraph $G_i^{\prime }$ (i = 1, 2, …, l): For every v ∈ V_B,

$$w(v_k^i)=\{\begin{array}{ll}b(v_k)+{\sum }_{j=0}^k[c_{i-1}(v_j)-c_i(v_j)]\hfill & \text{if}k={\mu }_i(v),\hfill \\ [b(v_k)-b(v_{k-1})]+[c_{i-1}(v_k)-c_i(v_k)]\hfill & \text{for}k={\mu }_i(v)+1,\dots ,{\kappa }_i(v).\hfill \end{array}$$ (1)

This completes the construction of G′.

4.4. Computing optimal layered net surfaces for the LNS problem

The graph G′ thus constructed allows us to find l optimal net surfaces in G, by computing a non-empty minimum-cost closed set in G′. In order to do that, below we prove the following facts: (1) Any closed set ≠ in G′ defines l feasible net surfaces in G whose total cost differs from that of by a fixed value; (2) any set of l feasible net surfaces in G corresponds to a closed set ≠ in G′ whose cost differs from that of by a fixed value. Consequently, a non-empty closed set in G′ with the minimum cost can specify l optimal net surfaces in G.

Given any closed set ≠ in G′, we define l feasible net surfaces, = {, ,…, }, in G, as follows. Recall that we search for each net , in the subgraph $G_i^{\prime }=(V_i^{\prime },E_i^{\prime })$. Let ${\mathcal{C}}_i=C\cap V_i^{\prime }$. For each vertex v ∈ V_B, denote by (v) the set of vertices of on the chain Ch_i(v) of $G_i^{\prime }$. Based on the construction of $G_i^{\prime }$, it is not hard to show that (v) ≠ . Let r_i(v) be the largest d-th coordinate of the vertices in (v). Define the function as (v) = r_i(v) for every v ∈ V_B.

First, we show that thus defined in G indeed consists of l feasible net surfaces. Lemmas 3 and 4 summarize the fact.

Lemma 3

Each defined by is a feasible net surface in G.

Proof

For each edge (v, u) ∈ E_B, let k′ = (v) and k″ = (u). We need to prove that (v_k′, u_k″) is an edge in G. In G, let the edge interval I(v_k′, u) of v_k′ on Col(u) be (u_p, u_p+1,…, u_p+s) and the edge interval I(u_k″, v) of u_k″ on Col(v) be (v_q, v_q+1,…, v_q+t) (see Figure 5(a)). If vertex u_k″ is in the edge interval I(v_k′, u), then we are done. Otherwise, either p > k″ or p + s < k″. Consider the case with p > k″. By the observation that , ≠ is a closed set in $G_i^{\prime }$ and the construction of $G_i^{\prime }$, vertex $u_p^i\in {\mathcal{C}}_i$, which contradicts to the fact that k″ is the largest d-th coordinate of the vertices in (u). Thus, we only need to consider the case with p + s < k″ (see Figure 5(a)). By a similar argument, we have q + t < k′. Since there is an edge between vertices v_q+t and u_k″ in G (v_q+t ∈ I(u_k″, v)), vertex u_k″ thus is in the edge interval I(v_q+t, u) of v_q+t on Col(u). Hence, we have the largest d-th coordinate Top(I(v_q+t, u)) of vertices in I(v_q+t, u) is no less than k″. Note that k″ > p + s and p + s is the largest d-th coordinate Top(I(v_k′, u)) of vertices in I(v_k′, u). Thus, Top(I(v_q+t, u)) > Top(I(v_k′, u)), contradicting with the proper ordering of the graph G (since k′ > q + t). Hence, the lemma follows.

Fig. 5.

(a) Illustrating the proof of Lemma 3. (b) Illustrating Case (2) in the proof of Lemma 6. (c) Illustrating Case (3) in the proof of Lemma 6.

Lemma 4

Any two adjacent net surfaces and in G (i = 1, 2,…, l − 1) defined by a closed set ≠ in G′ are (L_i, U_i)-separate.

Proof

For each vertex v ∈ V_B, we need to prove that L_i < (v) − (v) ≤ U_i. Let k′ = (v) and k″ =(v). Denote by v_p the i-th upstream vertex of v_k′ By the construction of G′, there is a directed edge from $v_{k^{\prime }}^i$ to $v_p^{i+1}$, which indicates that $v_p^{i+1}$ is in (v) ⊂ . Note that k″ is the largest d-th coordinate of vertices in (v). Thus, we have p ≤ k″ and k″ − k′ ≥ p − k′ ≥ L_i (the definition of the upstream vertices). On the other hand, let v_q denote the i + 1-st downstream vertex of v_k″. Then, vertex $v_{k^{″}}^{i+1}\in G_{i+1}^{\prime }$ has a directed edge to $v_q^i\in G_i^{\prime }$. Similarly, q ≤ k′. From the definition of the downstream vertices, k″ − q ≤ U_i. Hence, k″ − k′ ≤ k″ − q ≤ U_i. This proves the lemma.

Then, we show that the cost α() of thus defined by differs by a fixed value from the total vertex cost w() of . Note that in $G_i^{\prime }$, if a vertex $v_q^i\in {\mathcal{C}}_i(v)$, then all vertices in { $v_k^i\mid v_k^i\in C{h}_i(v)$, k ≤ q} are also in (v). Hence, the total vertex cost of (v) is $w({\mathcal{C}}_i(v))={\sum }_{k={\mu }_i(v)}^{r_i(v)}w(v_k^i)$. Thus, we have

$$\begin{array}{l}\alpha (\mathcal{N}\mathcal{S})=\sum _{i=1}^{l}b({\mathcal{N}}_i)+\sum _{i=0}^l{c}_i(R_i)=\sum _{i=1}^l\sum _{u\in {\mathcal{N}}_i}b(u)+\sum _{i=0}^l\sum _{u\in R_i}{c}_i(u)\\ =\sum _{i=1}^l\sum _{v\in V_B}\left\{b(v_0)+\sum _{k=1}^{{\mathcal{N}}_i(v)}[b(v_k)-b(v_{k-1})]\right\}\\ +\left(\sum _{i=1}^l\sum _{v\in V_B}\sum _{k=0}^{{\mathcal{N}}_i(v)}[c_{i-1}(v_k)-c_i(v_k)]+\sum _{v\in V_B}\sum _{k=0}^{\kappa -1}{c}_l(v_k)\right)\\ =\sum _{i=1}^l\sum _{v\in V_B}\{\underset{\text{the}\text{cost}\text{of}\text{the}“\text{lowest}”\text{vertex}{v}_q^i(q={\mu }_i(v))\text{of}C{h}_i(v)\text{in}{G}_i^{\prime }}{\underbrace{\left(b(v_0)+\sum _{k=1}^{{\mu }_i(v)}[b(v_k)-b(v_{k-1})]+\sum _{k=0}^{{\mu }_i(v)}[c_{i-1}(v_k)-c_i(v_k)]\right)}}\\ +\sum _{k={\mu }_i(v)+1}^{r_i(v)}\underset{\text{the}\text{cost}\text{of}\text{vertex}{v}_k^i\in C{h}_i(v)}{\underbrace{([b(v_k)-b(v_{k-1})]+[c_{i-1}(v_k)-c_i(v_k)])}}\}+\sum _{v\in V_B}\sum _{k=0}^{\kappa -1}{c}_l(v_k)\\ =\sum _{i=1}^l\sum _{v\in V_B}\sum _{k={\mu }_i(v)}^{r_i(v)}w(v_k^i)+\sum _{v\in V_B}\sum _{k=0}^{\kappa -1}{c}_l(v_k)=\sum _{i=1}^l\sum _{v\in V_B}w({\mathcal{C}}_i(v))+\sum _{v\in V_B}\sum _{k=0}^{\kappa -1}{c}_l(v_k)\\ =w(\mathcal{C})+\sum _{v\in V_B}\sum _{k=0}^{\kappa -1}{c}_l(v_k)\end{array}$$ (2)

Note that the term ${\sum }_{v\in V_B}{\sum }_{k=0}^{\kappa -1}{c}_l(v_k)$, denoted by c_l(V), is fixed and is the total sum of the l-th in-region costs of all vertices in G. From Lemmas 3 and 4 and Equation (2), the following lemma holds.

Lemma 5

Any closed set ≠ in G′ specifies l feasible net surfaces in G whose total cost differs from that of by a fixed value c_l(V).

Next, we argue that any l feasible net surfaces, = {, , …, } in G correspond to a closed set ≠ in G′. Based on the construction of G′, every vertex v_k on the net corresponds to a vertex $v_k^i$ in $G_i^{\prime }$ ( $v_k^i$ is not a deficient vertex). We construct a closed set ≠ in $G_i^{\prime }$ for each net , as follows. Initially, let = . For each vertex v ∈ V_B, we add to , the subset ${\mathcal{C}}_i(v)=\{v_k^i\mid k\le {\mathcal{N}}_i(v)\}$ of vertices on Ch_i(v) of $G_i^{\prime }$. Let $\mathcal{C}={\cup }_{i=1}^l{\mathcal{C}}_i$. We prove that is a closed set in G′.

Lemma 6

Any l feasible net surfaces, = {, ,…, }, in G define a closed set in G′.

Proof

For each vertex $v_p^i\in \mathcal{C}$, due to the construction of the graph G′, the only successors of $v_p^i$ are: (1) $v_{p-1}^i$ (if p ≥ μ_i(v)) on the same chain Ch_i(v); (2) one vertex $u_q^i$ on every adjacent chain Ch_i(u) of Ch_i(v) in $G_i^{\prime }$;(3) one vertex $v_{p^{\prime }}^{i+1}$ on the chain Ch_i+1(v) in $G_{i+1}^{\prime }$ (if i < 1); and (4) one vertex $v_{p^{″}}^{i-1}$ on the chain Ch_i−1(v) in $G_{i-1}^{\prime }$ (if i > 1).

Case (1)

Obviously, $v_{p-1}^i\in {\mathcal{C}}_i\subseteq \mathcal{C}$ due to the construction of .

Case (2)

Consider each adjacent chain Ch_i(u) of Ch_i(v) in $G_i^{\prime }$. Let r = (v) and t = (u). We have r ≥ p since r is the largest d-th coordinate of vertices of Ch_i(v) that we put in . Since all vertices $u_k^i$ with μ_i(u) ≤ k ≤ t are in , if q ≤ t, then we are done. Hence, we assume that q > t (see Figure 5(b)). Since $u_q^i$ is a successor of $v_p^i$, the smallest d-th coordinate Bottom(I(v_p, u)) of the vertices in the edge interval I(v_p, u) of v_p on Col(u) is q (i.e., Bottom(I(v_p, u)) = q). Note that the edge (v_r, u_t) is on the net surface . Thus, u_t ∈ I(v_r, u) and Bottom(I(v_r, u)) ≤ t. Hence, we have Bottom(I(v_p, u)) = q > t ≥ Bottom(I(v_r, u)), a contradiction to the proper ordering property of the graph G (since r ≥ p). Consequently, q ≤ t and $u_q^i\in {\mathcal{C}}_i\subseteq \mathcal{C}$.

Case (3)

Let r = (v) and t = (v). Then, r ≥ p. We need to prove t ≥ p′. Since the successor of $v_p^i$ on Ch_i+1(v) is $v_{p^{\prime }}^{i+1}$, v_p′ is the vertex in Col(v) with the smallest d-th coordinate that can be on (v_p is on the net ). Note that p′ − p ≥ L_i, but p′ may not equal to p + L_i due to the deficient vertex pruning operations. Now assume that the successor of $v_r^i$ on Ch_i+1(v) is $v_{r^{\prime }}^{i+1}$. We conclude that r′ ≥ p′. Otherwise, since r′ − r ≥ L_i and r ≥ p, we have p′ − p > r′ − p ≥ r′ − r ≥ L_i (see Figure 5(c)). Hence, vertex v_r′ is the vertex in Col(v) with the smallest d-th coordinate that can be on , a contradiction. Thus, r′ ≥ p′. Since v_r ∈ and v_t ∈ , t ≥ r′. This indicates that t ≥ p′.

Case (4)

A similar argument as for Case (3) can be applied, which proves that $v_{p^{″}}^{i-1}\in \mathcal{C}$.

In conclusion, for each vertex $v_p^i\in \mathcal{C}$, all successors of $v_p^i$ are in . Thus, the lemma holds.

Using a similar argument as in Equation (2), we can show that w() = α( ) − c_l(V). Together with Lemma 6, the following lemma follows.

Lemma 7

Any set of l feasible net surfaces in G defines a closed set ≠ ∅ in G′ whose cost differs from that of by a fixed value.

By Lemmas 5 and 7, we compute a minimum-cost closed set * ≠ in G′, which specifies l optimal net surfaces in G. However, the minimum closed set * in G′ can be empty (with a weight zero), and when this is the case, * = gives little useful information on G′. Fortunately, our careful construction of G′ still enables us to overcome this difficulty. If the minimum closed set in G′ is empty, then it implies that the weight of every non-empty closed set in G′ is non-negative. To obtain a minimum non-empty closed set in G′, we do the following: Let M be the total weight of vertices in Z₀; pick an arbitrary vertex u ∈ Z₀ and assign a new weight w(u) − M − 1 to u. We call this a translation operation on G′. From Lemma 2, Z₀ ≠ is a closed set in G′ and is a subset of any non-empty closed set in G′. Further, observe that the total weight of vertices in the closed set Z₀ (after a translation operation on G′) is negative. This implies that any minimum closed set in G′ (after a translation operation on G′) cannot be empty. Also based on Lemma 2, we have the following lemma.

Lemma 8

For a non-empty closed set in G′, let w() denote the total weight of before any translation operation on G′. Then after a translation operation, the weight of is w() − M − 1.

Now, we can simply find a minimum closed set * in G′ after performing a translation operation on G′. Based on Lemma 8, * is a minimum non-empty closed set in G′ before the translation.

As in the paper,^20,14,30 we find a minimum closed set * ≠ in G′ by formulating it as computing a minimum s-t cut in a weighted directed graph G″ transformed from G′, with |G″| = O(|G′|). Note that G′ has O(l · n) vertices and $O(l·n·\frac{m_B}{n_B})$ edges, where n = |V| is the number of vertices in G, and n_B = |V_B| and m_B = |E_B| for the net model B. The deficient vertex pruning procedure takes $O(l·n·\frac{m_B}{n_B})$ time. By using the minimum s-t cut algorithm in the paper,¹² we obtain the following result.

Theorem 1

The LNS problem can be solved in $O(l^2{n}^2\frac{m_B}{n_B}log(\frac{l·n·n_B}{m_B}))$ time.

5. Algorithm for the Bipartite LNS (BLNS) Problem

In this section, we consider the layered net surface problem on a d-D (d ≥ 3) ordered multi-column graph. We prove that the LNS problem on such a multi-column graph is NP-hard and give a polynomial time algorithm for solving the BLNS problem. Recall that the BLNS problem is defined on an ordered multi-column graph such that the p-contracted graph of its net model B is bipartite.

Theorem 2

The optimal LNS problem on a d-D ordered multi-column graph (d ≥ 3) is NP-hard.

Proof

We can prove that the LNS problem on a general d-D ordered multi-column graph is NP-hard, by reducing to it the minimum vertex cover problem that is known to be NP-complete.¹⁰

Given an undirected graph B = (V_B, E_B), the minimum vertex cover problem seeks a vertex cover V′ ⊆ V_B such that for each edge (v, u) ∈ E_B, at least one of v and u is in V′ and |V′| is minimized. This problem is reduced to the optimal LNS problem, as follows. First, embed the graph B into the 2-D space. Next, a 3-D ordered multi-column graph G = (V, E) is constructed. For each vertex v = (x, y) ∈ V_B, there are two corresponding vertices in V, v₀ = (x, y, 0) and v₁ = (x, y, l). Assign a cost 0 to v₀ and a cost 1 to v₁. For every edge (v, u) ∈ E_B, v₀ connects with u₁ in G and v₁ connects with both u₀ and u₁. Obviously, the graph G thus generated is an ordered multi-column graph. Actually, any two adjacent columns in G are in reverse order. We simply assume l = 1. Note that a vertex v ∈ V_B is in a vertex cover V′ of B if and only if v₁ ∈ V is on a net surface in G, and the cost of the net surface equals the cardinality of the vertex cover. Hence, Theorem 2 holds.

The NP-hardness proof of the LNS problem actually shows that on a d-D ordered multi-column graph (d ≥ 3) generated by a net model with only reverse edges, the LNS problem is NP-hard. On the other hand, Section 4 shows that on any d-D ordered multi-column graph generated by a net model whose edges are all proper, the LNS problem is polynomially solvable. Hence, it is desirable to examine net models in order to explore the polynomial solvability of the LNS problem. We next consider the LNS problem on a d-D ordered multi-column graph G = (V, E) with a special net model B = (V_B, E_B) whose p-contracted graph is bipartite.

Observe that if the p-contracted graph of the net model B = (V_B, E_B) is a bipartite one, then essentially, we can partition V_B into two disjoint subsets Q and Q̄ (V_B = Q ∪ Q̄), such that each reverse edge of B connects one vertex in Q with one in Q̄, but both vertices of each proper edge of B are either in Q or in Q̄. Based on this observation, we can construct a properly ordered graph G ′ = (V′, E′) such that the optimal solution to the LNS problem in G′ defines an optimal solution to the BLNS problem in G.

Each vertex in G defines one and exactly one vertex in G′. For every vertex v_k in G, if v ∈ Q, then it corresponds to vertex $v_k^{\prime }$ in G′; otherwise, if v ∈ Q̄, then it corresponds to vertex $v_{\kappa -1-k}^{\prime }$ in G′ (see Figure 6). The cost of each vertex in G′ is equal to the on-surface cost of its corresponding vertex in G. We denote by Col(v′) the column of κ vertices in G′ that are associated with v ∈ V_B. Then, edges in G′ are introduced. For each edge (v_k, u_j) in G, there are three cases: (1) if both v and u are in Q, then put in G′ an edge ( $v_k^{\prime },u_j^{\prime }$); (2) if both v and u are in Q̄, then add in G′ an edge ( $v_{\kappa -1-k}^{\prime },u_{\kappa -1-j}^{\prime }$); and (3) if v ∈ Q and u ∈ Q̄, then introduce in G′ an edge ( $v_k^{\prime },u_{\kappa -1-j}^{\prime }$). Figure 6(b) illustrates the construction of the properly ordered graph G′ from the ordered graph G in Figure 4(a).

Fig. 6.

(a) Illustrating the reverse ordering of G. Columns Col(v) and Col(u) are in reverse order, (b) Constructing a properly ordered graph G′ from G (assume that a, v ∈ Q, u, b ∈ Q̄, and Q ∪ Q̄ = V_B). Each vertex in a parenthesis is a corresponding vertex in G.

Lemma 9

The graph G′ = (V′, E′) constructed from G is a properly ordered multi-column graph.

Proof

Consider any edge (v, u) ∈ E_B of the net model B. If (v, u) is a proper edge, obviously, the columns Col(v′) and Col(u′) in G′ are in proper order. We thus only investigate the case that (v, u) is a reverse edge. Without loss of generality (WLOG), assume v ∈ Q and u ∈ Q̄. In G, each vertex V_k has an edge interval I(v_k, u) = {u_p, u_p−1, …, u_p−s} (s ≥ 0) on Col(u). From the construction of G′, vertex $v_k^{\prime }$ connects with an interval of vertices on Col(u′), that is, { $u_{\kappa -1-p}^{\prime },u_{\kappa -p}^{\prime },\dots ,u_{\kappa -1-p+s}^{\prime }$}.

We now consider the edge intervals of two adjacent vertices $v_k^{\prime }$ and $v_{k+1}^{\prime }$ on Col(u′). Let $I(v_k^{\prime },u^{\prime })=\{u_p^{\prime },u_{p-1}^{\prime },\dots ,u_{p-s}^{\prime }\}$ (s ≥ 0) and $I(v_{k+1}^{\prime },u^{\prime })=\{u_q^{\prime },u_{q-1}^{\prime },\dots ,u_{q-t}^{\prime }\}$ (t ≥ 0). This indicates that the edge interval I(v_k, u) of v_k on Col(u) is {u_κ−1−p, u_κ−p, …, u_κ−1−p+s} and the edge interval I(v_k+1, u) of v_k+1 on Col(u) is {u_κ−1−q, u_κ−q, …, u_κ−1−q+t}. Since Col(v) and Col(u) are in reverse order, we have κ − 1 − p ≥ κ − 1 − q and κ − 1 − p + s ≥ κ − 1 − q + t. Alternatively, p ≤ q and p − s ≤ q − t, that is, $\mathit{Top}(I(v_k^{\prime },u^{\prime }))\le \mathit{Top}(I(v_{k+1}^{\prime },u^{\prime }))$ and $\mathit{Bottom}(I(v_k^{\prime },u^{\prime }))\le \mathit{Bottom}(I(v_{k+1}^{\prime },u^{\prime }))$.

On the other hand, consider any two adjacent vertices $u_k^{\prime }$ and $u_{k+1}^{\prime }$, and their edge intervals on Col(v′). Let( $I(u_k^{\prime },v^{\prime })=\{v_p^{\prime },v_{p-1}^{\prime },\dots ,v_{p-s}^{\prime }\}$ (s ≥ 0) and $I(u_{k+1}^{\prime },v^{\prime })=\{v_q^{\prime },v_{q-1}^{\prime },\dots ,v_{q-t}^{\prime }\}$ (t ≥ 0). Note that vertex $u_k^{\prime }$ (resp., $u_{k+1}^{\prime }$) in G′ corresponds to u_κ−1−k (resp., u_κ−2−k) in G. The edge interval of u_κ−1−k on Col(v) is {v_p, v_p−1, …, v_p−s} and the edge interval of u_κ−2−k on Col(v) is {v_q, v_q−1, …, v_q−t}. Due to the inverse order between Col(v) and Col(u), we have p ≤ q and p − s ≤ q − t, that is, $\mathit{Top}(I(u_k^{\prime },v^{\prime }))\le \mathit{Top}(I(u_{k+1}^{\prime },v^{\prime }))$ and $\mathit{Bottom}(I(u_k^{\prime },v^{\prime }))\le \mathit{Bottom}(I(u_{k+1}^{\prime },v^{\prime }))$.

Hence, Col(v′) and Col(u′) are in proper order. This proves the lemma.

We show in the following that the optimal solution to the LNS problem in G′ defines an optimal solution to the BLNS problem in G.

Lemma 10

Any l feasible net surfaces in G′ specify l feasible net surfaces in G with the same total cost, and vice versa.

Proof

By Lemma 9, G′ is a properly ordered graph. Let $\mathcal{N}{\mathcal{S}}^{\prime }=\{{\mathcal{N}}_1^{\prime },{\mathcal{N}}_2^{\prime },\dots ,{\mathcal{N}}_l^{\prime }\}$ be a feasible solution to the LNS problem in G′. For each net surface ${\mathcal{N}}_i^{\prime }$, we define a function in G, as follows. Consider every vertex v ∈ V_B. If v ∈ Q, then ${\mathcal{N}}_i(v)={\mathcal{N}}_i^{\prime }(v^{\prime })$; otherwise, if v ∈ Q̄, then ${\mathcal{N}}_i(v)=\kappa -1-{\mathcal{N}}_i^{\prime }(v^{\prime })$. The construction of G′ from G immediately indicates that each is a feasible net surface in G and any two adjacent net surfaces and satisfy the separation constraints. Furthermore, the total cost of equals to the total cost of ${\mathcal{N}}_i^{\prime }$. Hence, = {, ,…, } is a feasible solution to the BLNS problem in G, and α () = α(′). A similar argument can show that any l feasible nets in G define l feasible net surfaces in G′ with the same total cost.

Note that G′ has the same size as G (i.e., |V′| = |V|and |E′| = |E|), and by Theorem 1 an optimal set of l nets in G′ can be computed in $O(l^2{n}^2\frac{m_B}{n_B}log(\frac{l·n·n_B}{m_B}))$ time. Based on Lemma 10, Theorem 3 follows.

Theorem 3

The general BLNS problem can be solved in $O(l^2{n}^2\frac{m_B}{n_B}log(\frac{l·n·n_B}{m_B}))$ time, where l is the number of sought net surfaces, n = |V|, m_B = |E_B|, and n_B = |V_B|.

6. Algorithms for the Net Surface Volume (NSV) Problems

This section presents our algorithms for several optimal net surface volume (NSV) problems. Specifically, instead of looking for multiple inter-related net surfaces as in Section 4, for a given d-D voxel grid Γ = [0..N − 1]^d of n = N^d cells, with each cell x(x₀, x₁,…, x_d−1) ∈ Γ having an arbitrary real “volume” value vol(x), we seek multiple surfaces that enclose a well-shaped region R ⊆ Γ, such that the volume vol(R) of R, vol(R) = Σ_{x ∈R}^vol(x), is minimized (or maximized). Note that even the case of the NSV problem on finding an optimal simple polygon in a weighted 2-D grid is in general NP-hard.^2,28 The NSV problems in higher dimensions are so under-explored that few known methods actually address them. Interestingly, our LNS approach can be generalized to solving the optimal NSV problems on several nontrivial and useful classes of geometric regions in polynomial time. Recently, Chen et al.⁷ studied certain more restricted classes of regions than our classes.

6.1. The weakly watershed-monotone regions and watershed-monotone shells

We consider two classes of regions, called weakly watershed-monotone regions and watershed-monotone shells, defined as follows. For any integers 0 ≤ i < d and 0 ≤ c < N, let Γ_i (c) denote the hyperplane x_i = c in the domain of Γ, i.e., Γ_i (c) is orthogonal to the x_i-axis and consists of all voxels of Γ whose x_i-coordinate is c. A region R in Γ is said to be x_i-monotone if for any line l parallel to the x_i-axis, the intersection R ∩ l is either empty or a continuous segment. Further, we say that R is watershed-monotone with respect to Γ_i (c) if (1) R is x_i-monotone, and (2) for any line l orthogonal to Γ_i (c), if the intersection R∩l ne; , then R∩l intersects a voxel of R ∩ Γ_i (c). (Intuitively, the intersection of R and Γ_I(c) is equal to the projection of R onto Γ_i (c), and is like a “watershed” of R.) If for every i = 0,1,…, d − 1, R is watershed-monotone to a Γ_i (c_i) for an integer 0 ≤ c_i < N, then we say that R is watershed-monotone. In Figure 7(a), the region R is watershed-monotone to both Γ₀ (c₀) and Γ₁ (c₁), and thus R is watershed-monotone. A region R ⊆ Γ is weakly watershed-monotone if R is watershed-monotone to every axis in a set of d − 1 axes of Γ and is monotone (but need not be watershed-monotone) to the remaining axis (e.g., see Figure 7(b)). Clearly, watershed-monotone regions are a subclass of weakly watershed-monotone regions.

Fig. 7.

(a) A watershed-monotone region, (b) A weakly watershed-monotone region which is not watershed-monotone to any Γ₁(c). (c) A watershed-monotone shell. (For a better readability, a 2-D pixel grid Γ is used.)

Suppose R is watershed-monotone with respect to some Γ_i (c_i), for each i = 0,1,…, d − 1; then it is easy to see that ${\cap }_{i=0}^{d-1}{\mathrm{\Gamma }}_i(c_i)\ne \varnothing$. A voxel in ${\cap }_{i=0}^{d-1}{\mathrm{\Gamma }}_i(c_i)$ is called a kernel voxel of R. Our second region class is called the watershed-monotone shells. For any two watershed-monotone regions R₁ and R₂ such that R₁ and R₂ have a common kernel voxel c and R₂ ⊆ R₁, the region R in Γ bounded between R₁ and R₂, i.e., R = R₁−R₂, is a watershed-monotone shell (e.g., see Figure 7(c)).

6.2. Modeling the optimal NSV problem for weakly watershed-monotone regions

In this section, we extend our approach for the LNS problem to solving the optimal NSV problem on weakly watershed-monotone regions. WLOG, we assume that a voxel c(c₀, c₁,…, c_d−2, c_d−1) ∈ Γ is given and the target weakly watershed-monotone region R is x_d−1-monotone (but not necessarily watershed-monotone to any Γd₋₁(c)) and is watershed-monotone to Γ_i(c_i)for each i ∈ {0,1,…, d − 2}.

Denote by the projection of Γ onto the first d −1 dimensions, i.e., = [0..N − 1]^{d − 1}. A surface S′ in Γ is said to be x_i-monotone if for any line l parallel to the x_i-axis, the intersection of S′ and l either is empty or is in one single voxel. Observe that the boundary surface S of a weakly watershed-monotone region R in Γ can be split into two surface pieces, the “upper” boundary surface S^u and the “lower” boundary surface S^l, both being x_d−1-monotone. We let S^u consist of the voxels forming the upper boundary of R (S^u ⊆ R), and S^l consist of the voxels that are immediately below the voxels on the lower boundary of R (S^l R), with respect to Γ_d−1(0). Obviously, the domain of S^u and S^l may be only a subset of , denoted by . To obtain R, it is sufficient to compute S^u and S^l. Note that we may view as an implicit net model and Γ as an implicit ordered multi-column graph.

Our approach in Section 4 can simultaneously identify multiple inter-related net surfaces that are defined on the entire domain . However, the domain of S^u and S^l may just be a proper subset of . Besides, here we optimize a different “volume” criterion. We overcome these difficulties as follows. First, we “extend” (or “spread”) each of S^u and S^l to become an x_{d − 1} -monotone surface defined on the entire , denoted by $S_e^u$ and $S_e^l$, respectively (e.g., Figures 8(a) and 8(b)). Interestingly, based on the monotonicity of R, we can manage to extend S^u and S^l in such a way that $S_e^u-S^u=S_e^l-S^l$. This implies that for any voxel x ∈ − , the voxel of $S_e^u$ defined on x must be the same as the voxel of $S_e^l$ defined on x. (Recall that LO(S′) denote the region in Γ consisting of all voxels on or below a (net) surface S′.) Then, the volume vol(R) of the sought region R is equal to $\mathit{vol}(LO(S_e^u))-\mathit{vol}(LO(S_e^l))$. Hence, we look for two such extended surfaces $S_e^u$ and $S_e^l$ with $\mathit{vol}(LO(S_e^u))-\mathit{vol}(LO(S_e^l))$ minimized; further, both $S_e^u$ and $S_e^l$ must be of a “good” shape to form a weakly watershed-monotone region.

Fig. 8.

(a) The extended “upper” boundary surface $S_e^u$ of a weakly watershed-monotone region R (consisting of the shaded pixels). (b) The extended “lower” boundary surface $S_e^l$ of R.

Next, we construct two properly ordered graphs G₁= (V₁, E₁) and G₂ = (V₂, E₂), which are for searching for $S_e^l$ and $S_e^u$, respectively. $S_e^l$ and $S_e^u$ are both defined on the following net model B = (V_B, E_B). Each voxel x ∈ corresponds to exactly one vertex u_x ∈ V_B of B. For every two adjacent voxels of , x = (x₀, x₁,…, x_{d − 2}) and y = (y₀, y₁, …, y_{d − 2}), along any dimension (i.e., ${\sum }_{i=0}^{d-2}\mid x_i-y_i\mid =1$), we put an edge (u_x, u_y) in B.

Before we start constructing G₁ and G₂ to incorporate the watershed-monotonicity, we need to introduce some notation. For a voxel x(x₀, x₁,…, x_{d − 1}) ∈ Γ, let x(i, val) denote the voxel in Γ whose i-th coordinate is val and the remaining coordinate values are the same as those of voxel x. The i-th column Col(x, i) of voxel x consists of all voxels {x(i, k) | k = 0,1,…, N − 1} in Γ. Two columns Col(x, d − 1) and Col(y, d − 1) are said to be adjacent in the p-th dimension (p = 0,1,…, d−2), if x_i = y_i for every i ∉ {p, d − 1} and |x_p − y_p| = 1.

We first construct graph G₁ = (V₁, E₁). Each voxel x ∈ Γ defines exactly one vertex $v_{\mathbf{x}}^1$ in G₁ (if the voxel in Γ is denoted by x(i, val), we then denote by $v_{\mathbf{x}}^1(i,\mathit{val})$ the corresponding vertex in G₁). Each vertex $v_{\mathbf{x}}^1\in V_1$ is assigned a cost $c(v_{\mathbf{x}}^1)=-\mathit{vol}(\mathbf{x})$. We next introduce edges into G₁. For each dimension i (i = 0,1,…, d − 2), consider every pair of (d − 1)-st columns Col(x, d − 1) and Col(y, d − 1) adjacent in the i-th dimension. WOLG, we assume that y_i = x_i + 1. Recall that we want to compute a weakly watershed-monotone region R with respect to a given voxel c(c₀, c₁, … c_i,…, c_{d − 1}) in Γ. Two cases are distinguished. (1) x_i < C_i: Note that for any two voxels x(d − 1, k′) and y(d − 1, k″) on $S_e^l$, we have k′ ≥ k″. Thus, for every 0 ≤ k < N, let vertex $v_{\mathbf{x}}^1(d-1,k)$ connect with each vertex $v_{\mathbf{y}}^1(d-1,k^{\prime })$ with k′ ≤ k. (2) x_i ≥ c_i: For any two voxels x(d − 1, k′) and y(d − 1, k″) on $S_e^l$, we have k′ ≤ k″. Hence, for every 0 ≤ k < N, we connect vertex $v_{\mathbf{y}}^1(d-1,k)$ with each vertex $v_{\mathbf{x}}^1(d-1,k^{\prime })$ with k′ ≤ k. The construction of G₂ is similar to that of G₁. But the ways of putting edges and assigning vertex costs in G₂ are different. Every vertex $v_{\mathbf{x}}^2(d-1,k)$ in G₂ connects vertices $v_{\mathbf{y}}^2(d-1,k^{\prime })$ with k′ ≤ k if $v_{\mathbf{x}}^2(d-1,k)$’s corresponding vertex $v_{\mathbf{x}}^1(d-1,k)$ in G₁ connects vertices $v_{\mathbf{y}}^1(d-1,k^{″})$ with k″ ≥ k in G₁; and vice versa. In addition, each vertex $v_{\mathbf{x}}^2$ in G₂ is assigned a cost $c(v_{\mathbf{x}}^2)=\mathit{vol}(\mathbf{x})$. Lemma 11 immediately follows from the graph constructions.

Lemma 11

The graphs G₁ and G₂ thus obtained are both properly ordered graphs on the net model B.

Suppose we are given any two feasible net surfaces in G₁ and in G₂, with “above” (i.e., ∀u_x ∈ V_B, (u_x) ≥ (u_x)). We can specify a region R in Γ using and , which actually correspond to $S_e^l$ and $S_e^u$ respectively in Γ, as follows. Initially, R = . For every voxel x ∈ , if (u_x) > (u_x), then we put in R all voxels x(d − 1, k) with (u_x) ≥ k > (u_x). The next lemma shows that the region R thus obtained is aweakly watershed-monotone in Γ.

Lemma 12

Any feasible net surfaces in G₁ and in G₂ with above define a weakly watershed-monotone region R in Γ with vol(R) = c(LO()) + c(LO()).

Proof

First, by the construction of R, it is easy to see that R is x_{d − 1}-monotone. Hence, below we show that R is watershed-monotone to every Γ_i (c_i) for i = 0, 1, …, d − 2, i.e., R is x_i-monotone and the intersection of R with Γ_i (c_i) equals the projection of R to Γ _i(c_i). We prove this by contradiction.

Assume that there exists an h ∈ {0,1,…, d − 2} such that R is not x_h-monotone. Then, there is a voxel a(a₀,…, a_{h − 1}, c_h, a_h+1,…, a_{d − 1}) ∈ Γ_h (c_h) such that the intersection of R with the h-th column Col (a, h) of a has at least two disconnected voxel segments, denoted by I₀, I₁,…, I_q along the h-th dimension, in this order. Let a (h, le_j) and a(h, re_j) be the two end voxels of segment I_j. For any two segments I_j and I_j+1, the gap segment I_g consists of all voxels a (h, k) in between I_j and I_j+1 with re_j < k < le_j+1. Considering the possible position of I_g in Γ, I_g can be “above” $S_e^u$ (i.e., $I_g\subseteq \mathrm{\Gamma }-LO(S_e^u)$), or $I_g\subseteq LO(S_e^l)$; further, I_g can be to the “left” (i.e., le_j+1 ≤ c_h), the “right” (i.e., re_j ≥ c_h) of Γ _h(c_h), or intersect Γ _h(c_h) (i.e., re_j < c_h < le_j+1). Hence, there are six cases for the possible position of I_g. In one such case, for instance, I_g is to the left of Γ_h (c_h) and is a subset of $\mathrm{\Gamma }-LO(S_e^u)$. The voxel a(h,re_j) ∈ R implies that the vertex $v_{\mathbf{a}}^2(h,r{e}_j)\in LO({\mathcal{N}}_2)$. Due to the construction of G₂, all vertices $v_{\mathbf{a}}^2(h,k)$ in G₂ with re_j < k ≤ c_h are in LO(), that is, $I_g\subseteq LO(S_e^u)$. Hence, the only possibility for which I_g R is when $I_g\subseteq LO(S_e^l)$, implying that the vertex $v_{\mathbf{a}}^1(h,r{e}_j+1)\in G_1$ which corresponds to voxel a(h, re_j + 1) ∈ I_g is in LO(). By the construction of G₁, all vertices $v_{\mathbf{a}}^1(h,k)$ with re_j ≥ k ≥ 0 are in LO(). Hence, the voxel segment I_j is not in R, a contradiction. The other five cases can be handled similarly. Therefore, for every i = 0, 1, …, d − 2, R is monotone to Γ_i (c_i).

Then, we argue that for each i = 0,1,…, d − 2, the intersection of R with Γ_i (c_i) equals the projection of R to Γ _i(c_i). Assume otherwise, i.e., there exists a voxel a ∈ R whose projection a(h, c_h) on Γ _h (c_h) is not in R. Note that a ∈ R indicates that its corresponding vertex $v_{\mathbf{a}}^2\in G_2$ is in LO(), and its corresponding vertex $v_{\mathbf{a}}^1\in G_1$ is not in LO(). Thus, the vertex $v_{\mathbf{a}}^2(h,c_h)$ of G₂ is also included in LO() based on the construction of G₂. But, $v_{\mathbf{a}}^2(h,c_h)\in LO({\mathcal{N}}_2)$ and a(h, c_h) ∉ R imply that the vertex $v_{\mathbf{a}}^1(h,c_h)$ of G₁ is in LO(). Hence, all vertices $v_{\mathbf{a}}^1(p,k)$ with 0 ≤ k < N are in LO(), which means a ∉ R, a contradiction. Thus, R is watershed-monotone to every Γ_i(c_i) for i = 0,1,…, d − 2.

Finally, we calculate the volume vol(R) of the region R.

$$\mathit{vol}(R)=\sum _{\mathbf{x}\in R}\mathit{vol}(\mathbf{x})=\sum _{\mathbf{x}\in LO(S_e^u)}\mathit{vol}(\mathbf{x})-\sum _{\mathbf{x}\in LO(S_e^l)}\mathit{vol}(\mathbf{x})=c(LO({\mathcal{N}}_1))+c(LO({\mathcal{N}}_2)).$$

This proves the lemma.

6.3. Computing a weakly watershed-monotone region of minimum total volume

Based on Lemma 12, we need to compute two such net surfaces ${\mathcal{N}}_1^{\ast }$ and ${\mathcal{N}}_2^{\ast }$ with the total cost $c(LO({\mathcal{N}}_1^{\ast }))+c(LO({\mathcal{N}}_2^{\ast }))$ minimized. As in Section 4, we formulate this problem as computing a minimum-cost closed set in a vertex-weighted directed graph G′ constructed from G₁and G₂.

The directed graph G′ = (V′, E′) contains two vertex-disjoint subgraphs $G_1^{\prime }=(V_1^{\prime },E_1^{\prime })$ and $G_2^{\prime }=(V_2^{\prime },E_2^{\prime })$, which are used to search for ${\mathcal{N}}_1^{\ast }$ and ${\mathcal{N}}_2^{\ast }$, respectively. $G_1^{\prime }$ (resp., $G_2^{\prime }$) is constructed from G₁(resp., G₂) using the same approach as that in Section 4.3 for building $G_i^{\prime }$ from G. The cost w(·) of each vertex in $G_1^{\prime }$ (resp., $G_2^{\prime }$) is set to be the cost of its corresponding vertex in G₁ (resp., G₂). Further, note that ${\mathcal{N}}_2^{\ast }$ must be above ${\mathcal{N}}_1^{\ast }$. To ensure this, a set $E_r^{\prime }$ of directed edges connecting each vertex in $G_1^{\prime }$ to the corresponding vertex in $G_2^{\prime }$ is introduced. Thus, $V^{\prime }=V_1^{\prime }\cup V_2^{\prime }$ and $E^{\prime }=E_1^{\prime }\cup E_2^{\prime }\cup E_r^{\prime }$ (see Figure 9 for an example).

Fig. 9.

Computing an optimal weakly watershed-monotone region R. For a better readability, a 2-D pixel grid Γ is used, (a) Illustrating the construction of the graph G′. The shaded pixel is the given weak watershed kernel pixel c. Only a portion of the directed edges from $G_1^{\prime }$ to $G_2^{\prime }$ is shown. The solid vertices shown make up of a closed setin G′. (b) The extended “upper” surface $S_e^u$ defined by the vertices of $G_2^{\prime }$ in . (c) The extended “lower” surface $S_e^l$ defined by the vertices of $G_1^{\prime }$ in . (d) A weakly watershed-monotone region R corresponding to the closed set .

Then, given any non-empty closed set in G′, we can define two net surfaces and in G₁ and G₂, as in Section 4.4. and have the following property.

Lemma 13

Any non-empty closed set in G′ specifies two net surfaces in G₁ and in G₂ such that is above and the cost w() of is equal to c(LO()) + c(LO()).

By Lemmas 12 and 13, we have the following fact.

Lemma 14

Any non-empty closed set in G′ specifies a feasible weakly watershed-monotone region R ⊆ Γ whose total volume is equal to the total cost of .

Now we show how to construct a closed set in G′ with the same total cost from a given weakly watershed-monotone region R of Γ. For simplicity, we use x(val) to denote a voxel x(d − 1, val) in Γ and $v_{\mathbf{x}}^{\prime i}(\mathit{val})$ to denote the corresponding vertex of x(d − 1, val) in $G_i^{\prime }(i=1,2)$.

Lemma 15

For any weakly watershed-monotone region R of Γ, there is a corresponding closed set in G′ with its cost w() = vol(R).

Proof

For a given weakly watershed-monotone region R ⊆ Γ, we define a closed set in G′, as follows. First, we let = . For each x ∈ , if the intersection of R and the voxel set (x(0), x(1),…, x(N − 1)} (also denote by Col(x, d− 1)) is not empty, say, consisting of voxels x(lw), x(lw + 1), …, x(up), with 0 ≤ lw ≤ up < N, then we put into the vertex $v_{\mathbf{x}}^{\prime 2}(up)$ of $G_2^{\prime }$ and the vertex $v_{\mathbf{x}}^{\prime 1}(lw-1)$ of $G_1^{\prime }$ if lw > 0. The sought closed set is the closure in G′ generated by . We next prove the cost w() equals vol(R).

For each x ∈ , we distinguish two cases: Col(x, d − 1) ∩ R ≠ and Col(x, d − 1) ∩ R = .

Case (1)

Col(x, d − 1) ∩ R ≠ . In this case, all vertices $v_{\mathbf{x}}^{\prime 2}(k)$ of $G_2^{\prime }$ with 0 ≤ k ≤ up and $v_{\mathbf{x}}^{\prime 1}(k)$ of $G_1^{\prime }$ with 0 ≤ k ≤ lw are included in . Let Ch₁(x) (resp., Ch₂(x)) denote the vertices in $G_1^{\prime }$ (resp., $G_2^{\prime }$) corresponding to voxels in Col(x, d − 1). Next, we will show that no vertices $v_{\mathbf{x}}^{\prime 2}(k)$ with up < k < N and $v_{\mathbf{x}}^{\prime 1}(k)$ with lw ≤ k < N are in . Otherwise, assume that $v_{\mathbf{a}}^{\prime 2}(t)=v^{\prime }(a_0,a_1,\dots ,a_{d-2},t)$ is the “top” vertex of Ch₂(a) in (i.e., for any vertex $v_{\mathbf{a}}^{\prime 2}(k)\in C{h}_2(\mathbf{a})\cap \mathcal{C}$, k ≤ t) and t > up. The constructions of G′ and _s indicate that there is no path from any vertex of $G_1^{\prime }$ in to a vertex in $G_2^{\prime }$ whose corresponding voxel is in R. Thus, there must be a vertex $v_{\mathbf{b}}^{\prime 2}=v^{\prime }(b_0,b_1,\dots ,b_{d-1})$ of $G_2^{\prime }$ in _s, from which $v_{\mathbf{a}}^{\prime 2}(t)$ is reachable in $G_2^{\prime }$ (i.e., there is a directed path in $G_2^{\prime }$ from $v_{\mathbf{b}}^{\prime 2}$ to $v_{\mathbf{a}}^{\prime 2}(t)$) and the (d − 1)-th coordinate b_{d − 1} of the voxel b is ≥ t. By characterizing of the property of G′, we then can claim that P: v′(b₀, b₁,…, b_d−2, b_d−1) → v′(a₀, b₁, …, b_d−2, b_d−1) → v′(a₀, a₁, b_2, …, b_d−2, b_d−1) → · · · → v′(a₀, a₁, …, a_d−2, b_d−1) → v′(a₀, a₁, …, a_d−2, t) is a path from $v_{\mathbf{b}}^{\prime 2}$ to $v_{\mathbf{a}}^{\prime 2}(t)$ in $G_2^{\prime }$. Moreover, due to the watershed-monotonicity, all corresponding voxels of the vertices on the path P, except the last one (a₀, a₁, …, a_d−2, t) (since R is not necessarily watershed-monotone to Γ_d−1(c_d−1)), are in R. Thus, a(b_d−1) = (a₀, a₁, …, a_d−2, b_d−1) ∈ R, a(up) = (a₀, a₁, …, a_d−2, up) ∈ R, and the voxel a(t) = (a₀, a₁, …, a_d−2, t) is in between a(b_d−1) and a(up) (i.e., b_d−1 ≥ t> up). Since R is monotone to x_d−1-axis, a(t) ∈ R, a contradiction. Hence, no vertices $v_{\mathbf{a}}^{\prime 2}(k)$ with up < k < N are in . Using a similar argument, we can show that no vertices $v_{\mathbf{a}}^{\prime 1}(k)$ of $G_1^{\prime }$ with lw ≤ k < N are in . Hence, we have w(Ch₂(x) ∩ ) + w(Ch₁(x) ∩ ) = vol(Col(x, d − 1) ∩ R).

Case (2)

Col(x, d − 1) ∩ R = . Let vertex $v_{\mathbf{x}}^{\prime 1}(lw)$ be the “top” vertex in (Ch₁(x)∩) (i.e., for any $v_{\mathbf{x}}^{\prime 1}(k)\in C{h}_1(\mathbf{x})\cap \mathcal{C}$, k ≤ lw). Then, the vertex $v_{\mathbf{x}}^{\prime 2}(lw)$ of $G_2^{\prime }$ is in . We want to show that for any vertex $v_{\mathbf{x}}^{\prime 2}(k)$ of Ch₂(x) in , k ≤ lw. Assume that in there exists a vertex $v_{\mathbf{x}}^{\prime 2}(t)$ of Ch₂(x) with t > lw. Since Col(x, d− 1) ∩ R = , no vertex of $G_2^{\prime }$ in can reach $v_{\mathbf{x}}^{\prime 2}(t)$. Hence, there must exist a path in G′ from a certain vertex $v_{\mathbf{a}}^{\prime 1}$ (with the (d − 1)-th coordinate a_d−1 of the voxel a being ≥ t) of $G_1^{\prime }$ in to $v_{\mathbf{x}}^{\prime 2}(t)$. But this implies that the vertex $v_{\mathbf{x}}^{\prime 1}(t)$ is also in , a contradiction. Thus, we have w(Ch₂(x) ∩ ) + w(Ch₁(x) ∩ ) = vol(Col(x, d − 1) ∩ R) = 0.

Therefore, w() = Σ_x∈ w(Ch₂(x)∩) + w(Ch₁(x)∩) = Σ_x∈ vol (Col(x, d − 1) ∩ R) = vol(R). Thus, the lemma follows.

Hence, we compute a minimum-cost non-empty closed set * in G′, which is used to specify a weakly watershed-monotone region R* in Γ with the minimum volume. Note that |V′| = O(n) and |E′| = O(d · n). By using the minimum s-t cut algorithm in the paper,¹² we have the following result.

Theorem 4

The optimal weakly watershed-monotone region problem is solvable in $O({dn}^{2}log\frac{n}{d})$ time.

6.4. Algorithm for computing an optimal watershed-monotone shell

This section presents our algorithm for solving the d-D optimal watershed-monotone shell problem (d ≥ 3). We solve this problem by using the same framework as the one for the optimal weakly watershed-monotone region problem. However, the construction of the graph G′ = (V′, E′), which is directly constructed from Γ, is in a slightly different way to incorporate the “shell” structure of the target region R.

The graph G′ for this problem contains two disjoint subgraphs $G_1^{\prime }=(V_1^{\prime },E_1^{\prime })$ and $G_2^{\prime }=(V_2^{\prime },E_2^{\prime })$, which are used to search for the inner and outer boundaries, S_I and S_O, of R, respectively. Note that in our algorithm for the optimal weakly watershed-monotone region problem, each subgraph is employed for searching one of the two pieces of the boundary surface of the target region. The geometric constraints between S_O and S_I are enforced by a set of edges between $G_2^{\prime }$ and $G_1^{\prime }$.

We first show the construction of $G_1^{\prime }$. Each voxel x ∈ Γ corresponds to exactly one vertex $v_{\mathbf{x}}^{\prime 1}\in V_1^{\prime }$ with a cost of −vol(x). For a given watershed kernel voxel c(c₀, c₁, …, c_d−1) the directed edges are introduced as follows. Consider every voxel x on each hyperplane x_i = c_i = 0,1, …, d − 1). The i-th column Col(x,i), as defined in Section 6.2, is divided into two sub-columns Col_l(x,i) and Col_u(x,i), such that Col_l(x,i) (resp., Col_u(x,i)) consists of all voxels in Col(x,i) whose i-th coordinates are no bigger (resp., smaller) than c_i. Then, for every k = N − 1, N − 2, …, c_i + 1 (i.e., x(i,k) ∈ Col_u(x,i)), the vertex $v_{\mathbf{x}}^{\prime 1}(i,k)$ has a directed edge to $v_{\mathbf{x}}^{\prime 1}(i,k-1)$ in $E_1^{\prime }$; for every k = 0,1,…, c_i − 1 (i.e., x(i,k) ∈ Col_l(x,i)), the vertex $v_{\mathbf{x}}^{\prime 1}(i,k)$ has a directed edge to $v_{\mathbf{x}}^{\prime 1}(i,k+1)$, as illustrated in Figure 10(a).

Fig. 10.

Computing an optimal watershed-monotone shell. For a better readability, a 2-D pixel grid Γ is used, (a) Illustrating the construction of the graph G′. The shaded pixel is the given watershed kernel pixel c. Only a portion of the edges from $G_1^{\prime }$ to $G_2^{\prime }$ is shown, (b) A watershed-monotone shell R corresponding to the closed set in G′ consisting of all the solid vertices in (a).

The construction of $G_2^{\prime }$ is similar to that of $G_1^{\prime }$. The only difference is the vertex-cost assignment scheme. In $G_2^{\prime }$, each vertex $v_{\mathbf{x}}^{\prime 2}$ associated with a voxel x ∈ Γ has a cost of vol(x). Then, for every voxel x ∈ Γ, we put a directed edge from the vertex $v_{\mathbf{x}}^{\prime 1}$ of $G_1^{\prime }$ to the vertex $v_{\mathbf{x}}^{\prime 2}$ in $G_2^{\prime }$.

We then compute a minimum-cost closed set * ≠ in G′. An optimal watershed-monotone shell in Γ can be specified using R*, as follows. Initially, let R* = ∅. For each voxel x ∈ Γ, if the vertex $v_{\mathbf{x}}^{\prime 2}$ of $G_2^{\prime }$ is in *, but $v_{\mathbf{x}}^{\prime 1}$ of $G_1^{\prime }$ is not, then put x into R* (see Figure 10(b)). The correctness of our algorithm for this problem follows from a similar argument for the optimal weakly watershed-monotone region problem.

Lemma 16

(1) Any watershed-monotone shell R ≠ ∅ in Γ corresponds to a non-empty closed set in G′ whose cost is equal to the volume of R. (2) Any non-empty closed set in G′ defines a watershed-monotone shell R ≠ ∅ in Γ whose volume is equal to the cost of .

Note that the graph G′ thus constructed has 2|Γ| = 2n vertices and O(dn) edges. By using the minimum s-t cut algorithm in the paper,¹² we have the following theorem.

Theorem 5

An optimal watershed-monotone shell in a d-D voxel grid Γ (d ≥ 3) can be computed in $O(d{n}^{2}log\frac{n}{d})$ time.

7. Implementation and Experiments

To further examine the behavior and performance of our LNS algorithm, we implemented it in standard C++ templates. After the implementation, we extensively experimented with 3-D images on physical phantoms, human pulmonary CT data, vascular MR data, and intravascular ultrasound data, and compared with a previously validated slice-by-slice 2-D segmentation approach based on graph search techniques.³² Over the years, graph search techniques have become one of the best understood and most utilized 2-D medical image segmentation tools.²² Our LNS program was tested on an AMD Athlon MP 2000+ Dual CPU workstation with 3.5GB memory running MS Windows XP. In our implementation, we only consider the on-surface cost of voxels.

The experiments showed that our LNS algorithm and software are computationally efficient and produce highly accurate and consistent segmentation results. The average execution times of our simultaneous l-surface (l = 2,3) detection algorithm on images of various sizes are shown in Table 1.

Table 1.

Average execution times (in seconds). Here (l denotes the number of sought surfaces.

Image Size	602×40	802×40	1002×40	1402×40	2002×40
l = 2	3.3	6.1	16.2	85.8	376.1
l = 3	4.8	9.4	22.3	96.1	401.3

An accuracy assessment on images of physical phantom tubes revealed that the overall signed errors for the inner and outer diameters derived from the tube boundaries were (mean ± standard deviation) −0.36 ± 2.47% and −0.08 ± 1.35%, respectively. Figure 11 presents the segmentation results by single surface detection methods versus our LNS algorithm. The synthesized image consists of 3 identical slices stacked together to form a 3-D image. Our LNS approach outperforms both the single surface detection algorithm^30,15 and the 2-D MetaMorphs method.¹³ In comparison, the current MetaMorphs implementation is unable to segment the outer contour.

Fig. 11.

Single-surface methods versus our inter-related surface method, (a) Cross-section of the original synthesized image. (b) Single surface detection³⁰,¹⁵ using a standard edge-based cost function, (c) MetaMorphs method¹³ segmenting the inner border in 2-D. (d) Double-surface segmentation obtained by our LNS approach.

Our LNS approach was tested on segmenting both the inner and outer airway wall surfaces in CT images, in which outer wall surfaces are very difficult to detect due to their blurred and discontinuous appearance and the presence of adjacent blood vessels. The CT images had a nearly isotropic resolution of 0.7 × 0.7 × 0.6mm³ and consisted of 500 – 600 image slices, with 512 × 512 pixels per slice. The currently used 2-D dynamic programming method works reasonably well for the inner wall segmentation but is unsuitable for the segmentation of the outer airway wall. Our new approach produces good segmentation results for both airway wall surfaces in a robust manner. Comparing to manual tracing on 39 randomly selected slices, our LNS technique yielded signed border positioning errors of −0.01 ± 0.15mm and 0.01 ± 0.17mm for the inner and outer wall surfaces, respectively. Figure 12 gives the comparisons on some airway wall segmentation results with a previous slice-by-slice 2-D graph-search based approach.³² Typically, by taking advantage of the 3-D coherence information, our LNS method can follow the 3-D airway wall structures correctly even among small and blurred airway segments, while the 2-D method exhibits large perturbations, and is even completely lost on some image slices.

Fig. 12.

Comparisons on airway wall segmentation results. For a better readability of the 3-D surface renderings, only the single luminal surface results are shown. (a), (b), (e), and (f) are the results yielded by the slice-by-slice 2-D graph-search based approach on two different airway segments. Four consecutive slices and the 3-D surface rendering are shown for each airway segment (10 slices). (c), (d), (g), and (h) are the walls segmented by our 3-D LNS approach.

Our algorithm was also used to simultaneously identify four vascular wall surfaces — the lumen–intima surface, internal elastic lamina (IEL), external elastic lamina (EEL), and adventitia in 3-D MR image data (see Figures 1(a) and 13). Our method successfully identified the four specified surfaces in 44 out of all 48 image slices. In comparison with manual tracing, the mean signed surface positioning errors for the lumen, IEL, and EEL boundaries were (mean ± standard deviation) 0.44 ± 0.37 pixel, −0.29 ± 0.34 pixel, and 0.11 ± 0.31 pixel. Comparing to the results obtained by the 2-D graph-search based approach,³² our method showed higher accuracy and 3-D consistency. Besides, the 2-D approach constantly requires the user to interactively define boundary points for guiding the border detection in difficult locations.

Fig. 13.

Results on MR vascular wall surface segmentation. (a) Four consecutive slices from the original image data. (b) Manually identified lumen, IEL, and EEL surfaces. The outermost adventitia surfaces are not shown. (c) Results produced by the program for our LNS algorithm.

Our 3-D LNS method for intravascular ultrasound (IVUS) image segmentation demonstrated lower surface positioning errors as well as more robust performance indicated by the success rate in comparison to the 2-D graph-search based approach.³² The IVUS images consisted of 1581 image frames approximately 0.5mm apart, with 384 × 384 pixels per frame and in-plane resolution of 0.3 × 0.3mm²/pixel. Some results are given in Table 2.

Table 2.

Comparison results on intravascular ultrasound image segmentation between our 3-D LNS technique and the 2-D graph search approach³².

Method	Success Rate	Positioning Error	Maximum Error
2-D graph search	68%	0.13±0.08mm	2.1mm
3-D LNS	82%	0.09±0.03mm	1.9mm

We have shown that our LNS technique provides a powerful tool for medical image segmentation. However, with the resolution of medical imaging scanners continuously increasing at a rapid pace, the time complexity of our LNS algorithm may constrain its feasibility in segmentation of large image data sets. As future work, we can apply a hierarchical refinement method to alleviate this problem in practice. Several recent works ^{4,3,21,34,17,31} have demonstrated that multi-scale image segmentation using graph cuts can produce impressive segmentation results. We first solves the LNS problem on the coarsest level graph and then solve the problem at successive higher resolution but on a narrow banded graph derived from the optimal solution found at the previous coarser graph. This multi-scale approach makes it possible to achieve high quality segmentation results on large data sets with faster speed, thus allowing our LNS technique to be used in a wider range of medical applications.