Fully Isotropic Fast Marching Methods on Cartesian Grids

Abstract

The existing Fast Marching methods which are used to solve the Eikonal equation use a locally continuous model to estimate the accumulated cost, but a discontinuous (discretized) model for the traveling cost around each grid point. Because the accumulated cost and the traveling (local) cost are treated differently, the estimate of the accumulated cost at any point will vary based on the direction of the arriving front. Instead we propose to estimate the traveling cost at each grid point based on a locally continuous model, where we will interpolate the traveling cost along the direction of the propagating front. We further choose an interpolation scheme that is not biased by the direction of the front. Thus making the fast marching process truly isotropic. We show the significance of removing the directional bias in the computation of the cost in certain applications of fast marching method. We also compare the accuracy and computation times of our proposed methods with the existing state of the art fast marching techniques to demonstrate the superiority of our method.

1 Introduction

A large number of computer vision applications such as segmentation, tracking, optimal path planning etc. use the minimal cost path approach. The Fast Marching Method which is widely used to solve the minimal path problem was first introduced by Sethian [1,10] and Tsitsiklis [11]. Cohen and Kimmel [4,5] later noticed that the minimal cost problem satisfies the Eikonal equation,

$$\Vert \nabla u\Vert =\tau .$$ (1)

For the Eikonal equation 1 defined on a Cartesian Grid, τ(x) would be the traveling cost at a given grid point and u(x), the accumulated cost. Since we solve the Eikonal equation numerically on Cartesian Grids, it is impossible to find the exact solution. Some modifications have been suggested in [6,7] to improve the accuracy of the Fast Marching method. Authors in [6,8,9,11] also suggest using an 8-connected neighbor scheme to improve accuracy. All these techniques use a locally continuous model to estimate the accumulated cost, but assume the traveling cost to be constant (discretized) around each grid point. Only [6] interpolates τ by shifting it to the center of the grid with a nearest neighbor interpolation, but it still assumes a discretized shifted grid for τ. In this paper we propose to use a locally continuous model to estimate τ as well.

For the geometry shown in Figure 1, the Fast Marching Method uses linear approximation to compute the accumulated cost at the point C, but it uses a constant traveling cost τ_C for each of the four grid cells containing the point C. The influence area of the cost function given at a grid point will include all the four quadrants around it. Thus, there is an overlap in the areas of influence of the grid points B and C. This means the value of u_C will vary depending on the direction from which the front is arriving. Ideally, for isotropic fast marching, the accumulated cost should be independent of the direction of the arriving front. For the image shown in Figure 2, we use the traveling cost, τ(x) = I(x), where I(x) is the intensity at each pixel. The accumulated cost in traveling from point A to B should be equal to the cost in traveling from B to A. But, due to the dependence on the direction of marching, there will be a difference in the accumulated costs. Figure 2 compares the minimal path obtained using back propagation from end point B to the source point A with the minimal path obtained by reversing the direction of front propagation. The difference in the two paths highlights the error caused by the directional dependence of the Fast Marching method.

Fig. 1.

Overlap in the influence areas of τ_B and τ_C

Fig. 2.

Image with random noise

In this paper we propose two methods to overcome the above-mentioned shortcomings. The first method uses a linear/bilinear model locally to estimate τ along the direction of the propagating front within each grid cell. Here we use a continuous model to estimate τ and also take the direction of arrival into consideration. We also discuss how the scheme can be made truly isotropic by removing any bias due to the marching direction. We call this method the Interpolated Fast Marching Method and it is discussed in detail in Section 2. In the second method we calculate u on an upsampled grid. In upsampling the grid, τ in the neighborhood of each grid point becomes constant, which eliminates the need to estimate τ using a continuous model. We will use the value of τ from the direction of arriving front. The upsampled version of the 4 and 8-connected neighbor schemes are discussed in Section 3. Finally, in Section 4 we describe a few numerical experiments conducted to highlight the significance of making the fast marching method independent of direction and we test the accuracy of the proposed methods.

2 Interpolated Fast Marching Method

For interpolated Fast Marching scheme we will assume τ to be continuous around each grid point and use linear/bilinear interpolation to estimate the value of the local traveling cost within each grid cell. Here we will derive the equations for the linear and bilinear Interpolated Fast Marching schemes. To estimate the traveling cost in a grid cell, the bilinear scheme will use the value of τ from all the grid points for a given quadrant. Since only 2 neighbors are used in each quadrant to calculate u in a 4-connected neighbor scheme, we only discuss the 8-connected neighbor scheme with bilinear interpolation.

2.1 Linear Interpolation

4-Connected Neighbors Scheme

Consider a front arriving at the grid point C from the quadrant AB and intersecting $\overline{AB}$ at E as shown in Figure 3(a). We will use the linear interpolation of the local traveling cost along the path $\overrightarrow{EC}$ to compute u_C. Thus the accumulated cost at C will be,

$$u_C=\underset{0\le t\le 1}{\min }\left\{u_B(1-t)+u_{A}t+{\int }_0^1\tau \left(p\right)\sqrt{t^2+{(1-t)}^{2}dp}\right\}.$$ (2)

Substituting, τ(p) = τ_C + (τ_A − τ_C)p(1 − t) + (τ_B − τ_C)pt, 0 ≤ p ≤ 1, in (2) we get,

$$u_C=\underset{0\le t\le 1}{\min }\left\{u_B(1-t)+u_{A}t+\sqrt{t^2+{(1-t)}^2}\left(\frac{{\tau }_A+{\tau }_C}{2}+\frac{{\tau }_B-{\tau }_A}{2}t\right)\right\}.$$ (3)

We get the necessary optimality condition to obtain the minimum of u_C by solving $\frac{d{u}_C}{dt}=0$, which yields,

$$u_A-u_B+\sqrt{t^2+{(1-t)}^2}\left(\frac{{\tau }_B-{\tau }_A}{2}t\right)+\frac{2t-1}{\sqrt{r^2+{(1-t)}^2}}\left(\frac{{\tau }_A+{\tau }_C}{2}+\frac{{\tau }_B-{\tau }_A}{2}t\right)=0.$$ (4)

Fig. 3.

Triangulation of Grid cells

8-Connected Neighbors Scheme

The geometry for 8-connected neighbors is shown in Figure 3(b). Using linear interpolation to estimate the local traveling cost along $\overrightarrow{EC}$, the accumulated cost, u_C, will be,

$$u_C=\underset{0\le t\le 1}{\min }\left\{u_B(1-t)+u_{A}t+{\int }_0^1\tau \left(p\right)\sqrt{1+t^2}dp\right\}.$$ (5)

Substituting, τ(p) = τ_C + (τ_B − τ_C)p + (τ_A − τ_B)pt, 0 ≤ p ≤ 1, in (5) we get,

$$u_C=\underset{0\le t\le 1}{\min }\left\{u_{A}t+u_B(1-t)+\sqrt{1+t^2}\left(\frac{{\tau }_B+{\tau }_C}{2}+\frac{{\tau }_A-{\tau }_B}{2}t\right)\right\}.$$ (6)

Again the minimizer of u_C can be obtained by solving $\frac{d{u}_C}{dt}=0$. Thus we have,

$$u_A-u_B+\sqrt{1+t^2}\left(\frac{{\tau }_A-{\tau }_B}{2}\right)+\frac{t}{\sqrt{1+t^2}}\left(\frac{{\tau }_B+{\tau }_C}{2}+\frac{{\tau }_A-{\tau }_B}{2}t\right)=0.$$ (7)

Isotropic Linear Interpolation Scheme

Figure 3(a) and 3(b) show the triangulation of a grid cell for the 4 and 8 neighbor schemes respectively. Depending on the front direction one of the quadrant/octant will be chosen to estimate the accumulated cost. But this will induce a directional bias. To overcome this directional bias, we will have to consider all possible triangulations shown in Figure 3(c). In effect the accumulated cost across a grid cell must be the minimum of the solutions obtained using the 4 and 8 neighbor schemes. This would make the scheme completely unbiased to direction and we call this scheme the Iso-Linear scheme.

2.2 Bilinear Interpolation

8-Connected Neighbors Scheme

The bilinear interpolation to estimate the local traveling cost along $\overrightarrow{EC}$ is given by,

$$\tau \left(p\right)={\tau }_A\left(p\right)\left(pt\right)+{\tau }_B\left(p\right)(1-pt)+{\tau }_C(1-p)(1-pt)+{\tau }_D(1-p)\left(pt\right).$$

It is inherently independent of any directional bias within a grid cell. Substituting, this value of τ(p) for 0 ≤ p ≤ 1, in (5) we get,

$$u_C=\underset{0\le t\le 1}{\min }\left\{u_{A}t+u_B(1-t)+\sqrt{1+t^2}\left(\frac{{\tau }_B+{\tau }_C}{2}+\frac{{\tau }_A-{\tau }_B}{3}t+\frac{{\tau }_D-{\tau }_C}{6}t\right)\right\}.$$ (8)

We will again solve $\frac{d{u}_C}{dt}=0$, which yields,

$$u_A-u_B+\sqrt{1+t^2}\left(\frac{{\tau }_A-{\tau }_B}{3}+\frac{{\tau }_D-{\tau }_C}{6}\right)+\frac{t}{\sqrt{1+t^2}}\left(\frac{{\tau }_B+{\tau }_C}{2}+\frac{{\tau }_A-{\tau }_B}{3}t+\frac{{\tau }_D-{\tau }_C}{6}t\right)=0.$$ (9)

Algebraic manipulations on (4), (7) and (9) will yield quartic equations. We used the Ferrari and Newton methods to solve these quartic equations. We compared the solutions from both techniques and found that they generate equally accurate solutions. Since Newton's method has a quadratic convergence, three iterations were sufficient for convergence. Fixing the number of iterations in each update step also ensures that we have the same computation complexity in each update. This makes the technique suitable to implement on hardware. The solution to Newton's method has fewer (logical and mathematical) operations in comparison to finding the Ferrari (analytic) solution; hence using Newton's method is computationally efficient. We compare the computation times of the two methods on a 500×500 grid in the Table 1. Here we call the 4 and 8-connected neighbor linear Interpolated Fast Marching schemes, Linear-4 and Linear-8 respectively and the 8-connected neighbor bilinear Interpolated Fast Marching scheme, Bilinear-8. The computation times were measured on a laptop with a 1.73 GHz Processor.

Table 1.

Comparison of computation times

	Linear-4	Linear-8	Bilinear-8
Analytic (Ferrari)	1.51s	2.83s	3.23s
Newton's Method	0.51s	0.52s	0.65s

2.3 Marching Forward Loop

We will still follow the main loop as explained in the basic Fast Marching method [10]. But, when a trial point is accepted in the min heap structure we will compute the value of u from both the quadrants/octants which include the newly accepted point and replace the newly calculated u with the minimum of the two solutions and the existing value of u (if the point is marked as trial).

Consider the example in Figure 4(a) where B is the newly accepted point and the accumulated cost at neighbor C is to be computed. As opposed to the basic fast marching technique, u_C does not solely depend on u_A,u_B,u_E and the local traveling cost, τ_C, but it also depends on the costs at all the other 8-connected neighbors. Thus, using the quadrant containing the minimum of u_A and u_E will not necessarily guarantee the minimum solution to (3). Hence we have to consider both the quadrants that contain B. If the front also arrives at C from the other two quadrants, they will be considered when the corresponding neighbors become accepted. The same argument can be extended to the 8-connected neighbor case shown in Figure 4(b). Here we only need to calculate u_C from the two octants containing $\overline{AB}$ and $\overline{FB}$ once point B is accepted. For the front arriving at point C as shown in Figure 2(c), we will consider the possibilities of the front arriving from $\overline{AB}$,$\overline{BD}$ and $\overline{DA}$.

Fig. 4.

B is the newly accepted grid point and u_C is to be computed

We depart from the traditional Fast Marching method only in the update procedure for the accumulated cost, but follow the same main (outer) loop. Thus the parallel algorithm explained in Bronstein et al.[2], can be extended for the implementation on hardware.

3 Upsampled Fast Marching Method

Figure 5 shows that there is no overlap in the influence areas of τ on the upsampled grid. Here the solid circles are the grid points from the original grid. Since the traveling cost is constant in each grid cell, there is no directional bias in the calculation of u. We will compute u on the upsampled grid and then downsample the output on the original grid.

Fig. 5.

No overlap in the influence areas of τ_A, τ_B, τ_C and τ_D

3.1 4-Connected Neighbors Scheme

In the upsampled grid, τ is constant in each quadrant around a grid point. Again the constant traveling cost within each grid cell makes this scheme isotropic. Depending on the direction of the front we will choose the value of τ in calculating u. For example, if the front arrives at E from the north-west then we would use τ_A (Figure 5). At the point G we would use τ_A for a front arriving from the west and τ_B for a front arriving from the east. We would use τ_A to calculate u_A irrespective of the direction of the arriving front. Since the value of τ is constant along the direction of the front at a sub-pixel level, it is not necessary to assume a locally continuous model in interpolating τ. Thus, the accumulated cost at E with the front arriving from the north-west would be,

$$u_E=\underset{0\le t\le 0.5}{\min }\left\{u_{F}t+u_G(0.5-t)+{\tau }_A\sqrt{t^2+{(0.5-t)}^2}\right\}$$ (10)

This minimization leads to the closed form solution,

$$u_E=\{\begin{array}{cc}\frac{(u_F+u_G+\sqrt{\delta })}{2}\hfill & \text{if}\delta \ge 0\hfill \\ \min (u_F,u_G)+\frac{{\tau }_A}{2}\hfill & \text{otherwise}\hfill \end{array}\phantom{\}}$$

where, $\delta =\frac{{\tau }_A^2}{2}-{(u_F-u_G)}^2$.

3.2 8-Connected Neighbors Scheme

As in the case with 4-connected neighbors, τ is constant in each octant around a grid point in the upsampled grid. We note that there will be exactly one point in each octant that corresponds to a point in the original grid. We will use the corresponding value of τ to compute u.

By following the procedure described in Section 2.3, we calculate u only from the two octants that contain the newly accepted point. If F is the newly accepted point, we will calculate u_E in the octants containing $\overline{FA}$ and $\overline{FD}$ (Figure 5). The solution will be the minimum of the two values obtained. Thus, for a front arriving from north-west, the accumulated cost at E will be,

$$u_E=\underset{0\le t\le 0.5}{\min }\left\{u_{A}t+u_F(0.5-t)+{\tau }_A\sqrt{0.5+t^2}\right\}$$ (11)

giving the closed form solution,

$$u_E=\{\begin{array}{cc}{u}_F+\frac{{\tau }_A}{2}\hfill & \text{if}{u}_F\le u_A\hfill \\ u_A+\sqrt{2}\frac{{\tau }_A}{2}\hfill & \text{if}{\tau }_A\le 2\sqrt{2}(u_F-u_A)\hfill \\ u_F+\frac{\sqrt{{\tau }_A^2-4{(u_F-u_A)}^2}}{2}\hfill & \text{otherwise}\hfill \end{array}\phantom{\}}$$

4 Numerical Experiments

We conducted a few experiments to compare the proposed methods to the basic Fast Marching Method (FMM) [10], Tsitsiklis scheme [11], Shifted-Grid Fast Marching (SGFM) [6] and Multi-stencil Fast Marching (MSFM) [7]. We also compare the upsampled 4 and 8-connected neighbor Fast Marching schemes with the upsampled version of the SGFM scheme (upSG).

In the first experiment we pick a random point, marked by the ‘x’ in the images shown in Figure 6, and compute u at every point of the image. We then compute the total cost in propagating a front from each point of the image back to the point marked by the ‘x’. We take the average of the difference (error) across the entire image. The numerical values are listed in the Table 2, under the column labeled Average Back Propagation Error (ABPE). We used the cost function, $\tau \left(x\right)=\frac{1}{{\mid 1+\nabla I\mid }^2}$ for the cardiac image and τ(x) = I(x) for the random noise image.

Fig. 6.

Test Images

Table 2.

Error norms for τ₁ and τ₂, Average Back Propagation Errors and Computation times

	τ 1			τ 2			ABPE		Time (s)
	L 1	L 2	L ∞	L 1	L 2	L ∞	I 1	I 2
FMM	2.46×10–2	6.73×10–4	0.0380	4.37×10–2	2.07×10–3	0.1060	0.0725	0.3901	0.27
Tsitsiklis	2.14×10–2	4.89×10–4	0.0281	3.81×10–2	1.57×10–3	0.0825	0.1007	0.4348	0.26
MSFM	2.36×10–2	6.07×10–4	0.0349	4.23×10–2	1.94×10–3	0.1007	0.0825	0.3572	0.29
SGFM	2.33×10–3	6.32×10–6	0.0051	1.25×10–2	2.14×10–4	0.0580	0.0022	0.0277	0.33
Linear4	1.10×10–2	1.71×10–4	0.0285	1.69×10–2	4.01×10–4	0.0875	0.0122	0.1036	0.51
Linear8	2.25×10–3	6.82×10–6	0.0046	4.46×10–3	3.43×10–5	0.0596	0.0028	0.0355	0.52
IsoLinear	2.25×10–3	6.82×10–6	0.0046	4.03×10–3	3.11×10–5	0.0596	0.0109	0.0911	0.91
Bilinear8	2.74×10–3	9.42×10–6	0.0052	5.01×10–3	4.10×10–5	0.0607	0.0028	0.0101	0.65
Up4	1.79×10–3	7.60×10–6	0.0101	3.14×10–3	2.89×10–5	0.0655	0.0451	0.1919	1.37
Up8	2.99×10–4	1.96×10–7	0.0014	1.54×10–3	7.81×10–6	0.0289	0.0011	0.0221	1.42
UpSG	1.96×10–3	4.15×10–6	0.0035	1.20×10–2	1.94×10–4	0.0566	0.0015	0.0141	1.42

In Figure 7 we present the results of segmenting the left ventricles in a 2D cardiac slice. To segment the image we pick a point on the boundary of the object and compute the saddle points as described in [5]. From each saddle point we then obtain two minimal paths back to the initial point; these paths will give the segmentation of the object. The minimal paths were obtained using a sub-pixel level back propagation scheme. We then choose the saddle point which minimizes the Chan-Vese [3] energy of the obtained segmentation. Images in Figure 7 show the overlay of segmentation curves initialized with 2 different user given points on the boundary. We see that the segmentation curves are not consistent and they depend on the initialization. This is mainly due to the difference in the marching direction in each case and weak image features at certain locations. We highlight certain regions in these images to compare the segmentation obtained from the different methods.

Fig. 7.

A comparison of segmentation

In the images shown in Figure 8, we compare the minimal paths obtained in traveling from point ‘0’ to points ‘1’,‘2’ and ‘3’ with the corresponding paths obtained by reversing the direction. We see that using interpolated FMM gives consistent paths, even in the absence of any strong image feature. The results are in accordance to the Average Back Propagation Errors listed in Table 2. The ABPE for the Tsitsiklis scheme is the highest and accordingly the paths obtained with the Tsitsiklis scheme show a lot of variation. Although the SGFM shows lower average error there are variations in the obtained minimal paths. This is because the interpolation of the cost function in SGFM is equivalent to image smoothing for the τ (τ(x) = I(x)) used in this example. This decreases the corresponding average error, but it also decreases the difference in the geodesic distances of the various paths. Thus with the change in the marching direction, the back propagation takes different paths between two given points.

Fig. 8.

A comparison of tracking

In the next example we compare the accuracy of the various techniques for two cost functions on a 50×50 grid,

$$\begin{array}{cc}\hfill {\tau }_1(x,y)=& 1∕20\sqrt{{\left(sin\frac{x}{20}cos\frac{y}{20}\right)}^2+{\left(cos\frac{x}{20}sin\frac{y}{20}\right)}^2},\hfill \\ \hfill {\tau }_2(x,y)=& 1∕10\sqrt{{\left(sin\frac{x}{20}cos\frac{y}{10}\right)}^2+{\left(cos\frac{x}{20}sin\frac{y}{10}\right)}^2}.\hfill \end{array}$$

The iso-contours of u_analytic are shown in Figure 9. The geodesics from the center (26, 26) of the grid will be straight lines for τ₁ and curved for τ₂. Since, we have the analytic solution for these cost functions, we can compare the L₁, L₂ and L norms for each method. _∞

$$\begin{array}{c}\hfill L_1=\mathrm{mean}\left(\mid u-u_{analytic}\mid \right),\hfill \\ \hfill L_2=\mathrm{mean}\left({\mid u-u_{analytic}\mid }^2\right),\hfill \\ \hfill L_{\infty }=\max \left(\mid u-u_{analytic}\mid \right).\hfill \end{array}$$

The numerical errors in using cost functions τ₁ and τ₂ are listed in Table 2. Notice that the error norms show significant improvement for the proposed methods, especially in the case with curved geodesics (τ₂). The iso-contours of the errors for τ₂ while using FMM, SGFM, Iso-Linear and up8 are shown in Figure 10.

Fig. 9.

Iso-contours

Fig. 10.

Iso-contours of errors for τ₂

We also enlist the computation times for each of these methods on a 500×500 grid in the last column of Table 2. All computation times were measured on a laptop with a 1.73 GHz Processor.

5 Conclusion

In this paper we present techniques to make the fast marching method independent of the marching direction and thus improve the accuracy of the Fast Marching Method. One approach interpolates the local traveling cost along the front and the other computes u on an upsampled grid. We also showed that combining the 8 and 4-connected neighbor schemes further reduces the inaccuracy by considering all possible directions of the arrival of the front. We have compared both our approaches to the existing Fast Marching techniques and we have shown a significant improvement over them. Although both our approaches have higher computation times, they can be implemented efficiently on hardware and they are practical solutions to eliminate the inaccuracies of existing techniques.