AN EFFICIENT NUMERICAL METHOD FOR THE SOLUTION OF THE L₂ OPTIMAL MASS TRANSFER PROBLEM

Abstract

In this paper we present a new computationally efficient numerical scheme for the minimizing flow approach for the computation of the optimal L₂ mass transport mapping. In contrast to the integration of a time dependent partial differential equation proposed in [S. Angenent, S. Haker, and A. Tannenbaum, SIAM J. Math. Anal., 35 (2003), pp. 61–97], we employ in the present work a direct variational method. The efficacy of the approach is demonstrated on both real and synthetic data.

1. Introduction

Optimal mass transport is an important problem with applications in econometrics, fluid dynamics, automatic control, transportation, statistical physics, shape optimization, expert systems, and meteorology [30, 36]. The problem was first formulated by the civil engineer Gaspar Monge in 1781, and concerned finding the optimal way, in the sense of minimal transportation cost, of moving a pile of soil from one site to another. Much later the problem was extensively analyzed by Kantorovich [23], and so is now known as the Monge–Kantorovich (MK) problem.

There are several formulations of the problem [1, 30, 36] of varying degrees of generality. We recall here the formulation of the Monge–Kantorovich problem for smooth densities and domains in Euclidean space. For more general measures, see [1]. Let Ω₀ and Ω₁ be two diffeomorphic connected subdomains of ℝ^d, and let μ₀, μ₁ be Borel measures on Ω₀ and Ω₁, each with a strictly positive density function ${\mu }_0(x)\ge {\mu }_{\text{low}}^0>0\text{ and }{\mu }_1\ge {\mu }_{\text{low}}^1>0$, respectively. Assume

$${\mu }_0({\mathrm{\Omega }}_0)={\mu }_1({\mathrm{\Omega }}_1),$$

i.e.,

$${\displaystyle {\int }_{{\mathrm{\Omega }}_0}{\mu }_0(x)\mathit{\text{dx}}}={\displaystyle {\int }_{{\mathrm{\Omega }}_1}{\mu }_1(x)\mathit{\text{dx}},}$$

so that the same total mass is associated with Ω₀ and Ω₁.

The version of the Monge–Kantorovich problem of interest in this work may be expressed as follows:

$$\text{min}M(u)≔\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}{\mu }_0(x)\rho (u,x)\mathit{\text{dx}},}$$ (1.1a)

$$\text{s.t.}c(u)=\text{det}(\nabla u){\mu }_1(u(x))-{\mu }_0(x)=0,$$ (1.1b)

where u is a C^1,α diffeomorphism from Ω → Ω and ρ(u, x) is a distance function between x and u. Here we treat only the case

$$\rho (u,x)={|u-x|}^2.$$

The constraint c(u) = 0 (the Jacobian equation) is often referred to as the mass preserving (MP) property.

Even with a simple, quadratic distance function, this is a highly nonlinear equality constrained optimization problem. There is extensive analysis as to the existence, uniqueness, and properties of the solution (see, for example, [1, 11, 36] and the references therein). However, while there is a large body of literature which deals with the analysis of the problem, there is a surprisingly small number of papers that deal with the solution of the problem, and even a smaller number of papers that deal with efficient numerical solutions of the problem [8, 4, 2, 29, 10].

Among the papers that deal with the numerical solutions is the paper of Benamou and Brenier [4]. Their paper reconstructs an optimal path from μ₀ to μ₁ by solving an optimization problem with a space-time transport partial differential equation (PDE) as a constraint. Their approach is particularly useful if the transportation path is needed. Its disadvantage is that it enlarges the dimension of the problem by introducing a space-time control problem. An interesting geometric method has also been formulated by Cullen and Purser [9].

A very different approach which is closer to that of the present work has been proposed by Angenent, Haker, and Tannenbaum (AHT) [2]. This approach reconstructs the transformation directly. The idea of the AHT method is to obtain an initial MP transformation u₀(x), and then replace the original problem with the following optimization problem:

$$\text{min}M(s)=\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}{\mu }_0(x){|u_0(s^{-1})-x|}^2\mathit{\text{dx}},}$$ (1.2a)

$$\text{s.t.}c(s)=\text{det}(\nabla s){\mu }_0(s(x))-{\mu }_0(x)=0,$$ (1.2b)

where s ∈ 𝒞^1,α is an MP mapping from μ₀ to itself. It is well-known that the optimization problems (1.1) and (1.2) are equivalent [25]. The iteration starts with s = Id, and since a composition of mass preserving maps is also mass preserving, the authors assume that the MP constraint is valid throughout the optimization process. This allows them to obtain a time dependent PDE with the MP constraint as its invariant, that converges (in functional space) to the solution of the problem. In order to implement their method in a discrete setting, the authors have employed a forward Euler scheme for time stepping and a first order finite difference discretization of the spatial derivatives.

What is essentially done via the AHT methodology is to construct the so-called polar factorization [6, 13, 14, 25] of the initial MP mapping u₀, thus obtaining u = ∇ϕ, where ϕ is some scalar function. There are two main shortcomings to their numerical approach. First, a robust method to obtain an initial MP mapping is needed. The authors have used 1D integration which leads to an MP mapping which is highly irregular. See also [27] for an approach to constructing the initial MP mapping. Thus, the solution of the optimization problem may be far from the initial point, and as a result many iterations are needed to convergence. Second and more important, the AHT method updates the transformation in a space which is tangential to the linearized MP constraint. While such an update is tangential to the MP constraint for a small (infinitesimal) correction, it is not tangential to any finite step used in the discrete setting. Since the MP constraint is never specifically enforced during the discrete AHT minimization process it “drifts,” that is, the final mapping may not be mass preserving and therefore the discrete PDE may not converge to a discrete approximation of the problem.

In this paper we develop a new, straightforward technique that does not suffer from the aforementioned difficulties. First, we modify the objective function in order to obtain better attraction to the MP solution. Second, we employ a conservative discretization of the objective function and the constraint, and finally, we use the state-of-the-art numerical optimization procedure and linear algebra solvers to obtain the solution. The method is illustrated via some examples of medical image registration. These examples are included simply to elucidate the proposed approach on some actual data. It is not our intention to propose optimal mass transport as a general method to medical image registration. See also [21, 20, 39] for a much more extensive discussion of the use of optimal mass transport and elastic image registration.

The rest of the paper is organized as follows. In section 2 we review the mathematical derivation of our approach. In section 3, we discuss the discretization scheme. In section 4 we discuss the solution of the discrete problem. In section 5, we perform numerical experiments in two and three dimensions that demonstrate the effectiveness of our method. Finally, in section 6 we summarize our work.

2. Problem reformulation

We give now the key ideas underlying our approach to solving the L² Monge–Kantorovich problem. Accordingly, suppose that we have an initial transformation w that is not mass preserving, and that we would like to obtain a transformation u which is mass preserving from w. This can be done by projection, that is, by solving the optimization problem

$$\underset{u}{\text{min}}{\Vert u-w\Vert }_{\mathscr{H}}^2,$$ (2.1)

$$\text{s.t.}c(u)=0,$$ (2.2)

where ‖ · ‖_ℋ is some appropriate norm. It is interesting to note that the MK problem is nothing but a projection from the identity transformation w(x) = x with respect to the μ₀-weighted L₂ norm ‖ · ‖_μ0 (see (1.1a)–(1.1b)).

Thus if we simply project to the MP constraint with the μ₀ norm, we should be able to obtain a local minimizer for (2.1). However, such a projection may not be the global minimizer. Nevertheless, while we cannot guarantee global convergence of the projection process, we can increase the chances that the local minimizer of (2.1) is also a global minimizer of the MK problem. To do that, we recall that the global minimizer has the property that u = ∇ϕ [1, 36]; therefore, ∇ × u = 0. Thus we propose to perform the projection by solving the following optimization problem:

$$\mathbf{\text{PROJ\hspace{1em}}}\underset{u}{\text{min}}\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}({\mu }_0(x){|u-w|}^2+\beta {|\nabla \times u|}^2)\mathit{\text{dx}},}$$ (2.3a)

$$\text{s.t.}c(u)=0,$$ (2.3b)

where β > 0. While the term β|∇ × u|² does not change the global minimum, it does give a bias towards a curl-free solution when solving for the projection, and thus has a better chance to converge to the global minimum.

One can check that the conditions for a minimum lead to the following nonlinear system of equations:

$${\mu }_0(u-w)+\beta \nabla \times \nabla \times u+p\text{det}(\nabla u)\nabla {\mu }_0(u)-\nabla \cdot (\text{det}(\nabla u){(\nabla u)}^{-\top }p),$$ (2.4a)

$$\text{det}(\nabla u){\mu }_0(u)=0,$$ (2.4b)

where p is the Lagrange multiplier.

The optimization problem PROJ is a continuous constrained optimization problem. There are many techniques to obtain the solution to such problems. In the following, we will consider a version of sequential quadratic programming (SQP) which is a commonly used methodology for such constrained optimization problems [28]. The advantage of SQP type methods is that they are mesh-independent. That is, the number of iterations to convergence is independent of the mesh size [37].

However, before we consider the explicit solution process, we should first describe two possible approaches. In the “discretize-then-optimize” approach, we discretize the projection (2.3), and then solve a discrete optimization problem. In the “optimizethen-discretize” approach, we first write the Euler–Lagrange equations (2.4) in a continuous setting, and then discretize them in order to obtain a system of discrete nonlinear equations. The two approaches are summarized in [17]. The advantage of the discretize-then-optimize approach is that common optimization algorithms can be used with little modification. This is particularly important if we wish to obtain convergence to a minimum from a possibly distant starting point. Recent work in the field [28, 7] suggests several mechanisms to achieve this goal, which we explore in some detail below. Particular care must be taken so that the inner products and norms are discretized appropriately [22]. We therefore proceed with the discretization of the optimization problem.

We should note that although we use the discretize-then-optimize approach, the discrete nonlinear system and its linearization can be thought of as a discretization of the Euler–Lagrange equations (2.4). As such we need efficient solvers to systems which involve the curl operator and the linearization of the constraints.

3. Discretization

In order to discretize the problem, we define the Lagrangian

$$ℒ(u,p)=\frac{1}{2}{\displaystyle {\int }_{\mathrm{\Omega }}({\mu }_0(x){|u-x|}^2+\beta {|\nabla \times u|}^2)\mathit{\text{dx}}+(p,c(u)),}$$ (3.1)

where p is a Lagrange multiplier.

Certain key applications (for example, in medical image processing) have data which are discretized on a regular grid. We therefore construct our discretization based on a finite volume/difference approach. Rather than working with the deformation field u, we work with the update or displacement υ = u − x. This yields numerically smaller perturbations. Note also that ∇ × u = ∇ × υ, and thus the projection (2.3) does not change.

We divide the domain, Ω, into n₁ × ⋯ × n_d cells, each of size h₁ × ⋯ × h_d, where d is the dimension of the problem. For simplicity we discretize all the components of υ at the nodes of each cell to obtain d grid functions υ̂¹, …, υ̂^d. The Lagrange multiplier p is discretized at the cell centers. In 2D, we denote by ${\widehat{\upsilon }}_{j,k}^i$ the ith grid function discretized at the (x₁)_j, (x₂)_k node, and by ${\widehat{p}}_{j+\frac{1}{2},k+\frac{1}{2}}$ the Lagrange multiplier discretized at the cell center. The discretized quantities are plotted in Figure 1.

FIG. 1.

Discretization of the displacement and Lagrange multipliers.

We note that a staggered discretization for the displacement can also be used [19]. Similar to problems that are derived from computational fluid dynamics and electromagnetics, such a discretization has very nice numerical properties. However, it is less simple to implement and requires careful treatment of boundary conditions. A discussion about the staggered versus the unstaggered discretization can be found in [35].

We now consider the different numerical approximations needed in order to obtain a discrete approximation to the continuous system. A consistent discretization for Δ, ∇× and a consistent discretization of ∇u = I + ∇υ are needed. There are a number of possible discretizations that lead to a well-posed system. Here we derive the explicit discretization in 2D. The extension to 3D is straightforward.

3.1. Discretization of Δ and ∇×

In order to obtain a consistent discretization of the Laplacian, we use a standard discretization (5 point stencil in 2D and 7 point stencil in 3D) with Dirichlet boundary conditions.

To compute the discretization of the curl, we use “short differences” in one direction (see [3, p. 11]) and average them (or the differences) in the other direction. For example, in 2D we obtain

$${(C\widehat{\upsilon })}_{i+\frac{1}{2},j+\frac{1}{2}}=\frac{{\widehat{\upsilon }}_{i,j+1}^1-{\widehat{\upsilon }}_{i,j}^1+{\widehat{\upsilon }}_{i+1,j+1}^1-{\widehat{\upsilon }}_{i+1,j}^1}{2h_1}-\frac{{\widehat{\upsilon }}_{i+1,j}^2-{\widehat{\upsilon }}_{i,j}^2+{\widehat{\upsilon }}_{i+1,j+1}^2-{\widehat{\upsilon }}_{i,j+1}^2}{2h_2}+𝒪(h^2),$$ (3.2)

where we denote by C the curl matrix. It is easy to verify the second order accuracy of this discretization.

3.2. Discretization of the MP constraint

To discretize the MP constraint we use a finite volume discretization, and so integrating over a given region Ω_j

$${\displaystyle {\int }_{{\mathrm{\Omega }}_j}c(\upsilon )\mathit{\text{dx}}={\displaystyle {\int }_{{\mathrm{\Omega }}_j}\text{det}(I+\nabla \upsilon ){\mu }_0(x+\upsilon )\mathit{\text{dx}},}}$$

we see that two approximations are needed. First, we need to approximate det(I+∇υ), and second, we need to approximate μ₀(x + υ). There are a number of options for discretizing the constraint. In our application, μ₀ is typically discretized in cell centers; therefore we choose the domain Ω_j as a cell in our discretization.

In order to approximate μ₀(x + υ) in cell centers, we use linear interpolation; see [26] for details. We denote this approximation by μ₀(x_h + υ̂), where x_h is the discretization of x = [x₁, x₂] on the cell centers, $(i+\frac{1}{2},j+\frac{1}{2})$.

In order to approximate det(I + ∇υ) at the cell centers, we first compute all the derivatives in cell centers as follows:

$${({\upsilon }_{x_1}^k)}_{i+\frac{1}{2},j+\frac{1}{2}}=\frac{{\widehat{\upsilon }}_{i+1j}^k-{\widehat{\upsilon }}_{\mathit{\text{ij}}}^k+{\widehat{\upsilon }}_{\mathit{\text{ij}}+1}^k-{\widehat{\upsilon }}_{\mathit{\text{ij}}+1}^k}{2h}+𝒪(h^2),$$ (3.3a)

$${({\upsilon }_{x_2}^k)}_{i+\frac{1}{2},j+\frac{1}{2}}=\frac{{\widehat{\upsilon }}_{\mathit{\text{ij}}+1}^k-{\widehat{\upsilon }}_{\mathit{\text{ij}}}^k+{\widehat{\upsilon }}_{i+1j+1}^k-{\widehat{\upsilon }}_{i+1j}^k}{2h}+𝒪(h^2),k=1,2.$$ (3.3b)

We also denote the block diagonal matrix

$$\begin{array}{c}\mathbf{\text{GRADv}}=\text{blkdiag}\left(\begin{array}{cc}{\left({\upsilon }_{x_1}^1\right)}_{i+\frac{1}{2},j+\frac{1}{2}}\hfill & {\left({\upsilon }_{x_2}^1\right)}_{i+\frac{1}{2},j+\frac{1}{2}}\hfill \\ {\left({\upsilon }_{x_1}^2\right)}_{i+\frac{1}{2},j+\frac{1}{2}}\hfill & {\left({\upsilon }_{x_2}^2\right)}_{i+\frac{1}{2},j+\frac{1}{2}}\hfill \end{array}\right),\hfill \\ \hfill i=1,\dots ,n_1-1,j=1,\dots ,n_2-1.\hfill \end{array}$$

This matrix is the discrete analog to the matrix ∇υ at each cell center. For the MK problem to be well defined, the matrix I + ∇υ must be SPD everywhere.

Let D_j be the discretization of the jth derivative. The discrete MP constraint for υ then reads

$$(\mathbf{1}+D_1{\widehat{\upsilon }}_1+D_2{\widehat{\upsilon }}_2+(D_1{\widehat{\upsilon }}_1)\odot (D_2{\widehat{\upsilon }}_2)-(D_2{\widehat{\upsilon }}_1)\odot (D_2{\widehat{\upsilon }}_1))\odot {\mu }_0(x_h+\widehat{\upsilon })={\mu }_1(x_h),$$ (3.4)

where ⊙ is the Hadamard product. It is important to further study this discretization and its properties. Typically to study any discretization we study the consistency and the stability of our discretization.

3.2.1. Consistency of the discretization

To study the consistency of the discretization, we employ the following lemma.

LEMMA 1. Let υ̂ = (υ̂¹, … , υ̂^d) be grid functions of a sufficiently smooth field υ, and let GRADv be approximated for the matrix ∇υ. Then, if I + ∇υ has positive real eigenvalues with Re(λ_min) ≥ ρ > 0, the matrix GRADv also has positive real eigenvalues for a sufficiently small h.

Proof. To prove the lemma, we note that the matrix is a block 2 × 2 matrix. This implies that the eigenvalues of the matrix I + GRADv can be computed blockwise, where each of the blocks corresponds to a different cell in the discrete grid. Since the approximation of the derivatives is 𝒪(h²), the eigenvalues of the matrix I + GRADv converge to the eigenvalues of I + ∇υ.

It is important to note that the above lemma does not predict the order of convergence. Unfortunately, very small perturbations in the discrete grid function υ can lead to large perturbations in the eigenvalues of GRADv. Consider, for example, the case in which the approximation to GRADv at some cell is

$$A=I+\mathbf{\text{GRADv}}=\left(\begin{array}{cc}1\hfill & a\hfill \\ 0\hfill & 1\hfill \end{array}\right),$$

and that we have a numerical error of the form εE where

$$E=\left(\begin{array}{cc}0\hfill & 0\hfill \\ 1\hfill & 0\hfill \end{array}\right).$$

Then, it is easy to verify that

$$\lambda (A+\epsilon E)=1\pm \sqrt{\epsilon a}.$$

Thus, for large enough a the eigenvalues of the perturbed system can be very different from the eigenvalues of the continuous one.

Nevertheless, the above lemma is important because it implies that the overall (discrete) problem is consistent for sufficiently small h. It is also important because it gives certain hints about possible difficulties with the numerical solution. It is well known that the continuous problem can be ill-posed if the real parts of the eigenvalues of ∇u are not positive, that is, Re(∇u) ≻ 0 (see [11]). Lemma 1 requires h to be sufficiently small in order to obtain this behavior. If the displacement field is not sufficiently smooth or h is not sufficiently small, then we may encounter negative eigenvalues in the discrete approximation of u. In this case the iteration can stall or even diverge. We therefore propose to check the eigenvalues of I + GRADv throughout the iteration. This can be done in 𝒪(n) work due to the block structure of the system.

There are a number of ways to correct for negative eigenvalues. Denote the Schur decomposition of the gradient matrix at the $i+\frac{1}{2},j+\frac{1}{2}$ cell by

$${(I+\mathbf{\text{GRADv}})}_{i+\frac{1}{2},j+\frac{1}{2}}=V_{i+\frac{1}{2},j+\frac{1}{2}}{\mathrm{\Lambda }}_{i+\frac{1}{2},j+\frac{1}{2}}{V}_{i+\frac{1}{2},j+\frac{1}{2}}^{\top },$$

where all the matrices are 2 × 2 (in 2D). For the sake of concreteness, assume that the first eigenvalue ${\mathrm{\Lambda }}_{i+\frac{1}{2},j+\frac{1}{2}}^1\le 0$. A simple modification of our method is simply to inflate ${\mathrm{\Lambda }}_{\mathit{\text{ij}}}^1$ in such a manner that it is slightly larger than 0, for example, to h. We have taken precisely this approach to avoid vanishing or negative eigenvalues that may lead to divergence of the method.

3.2.2. Stability of the discretization

Although the discretization of the MP constraint is consistent and second order accurate, it is not a stable discretization. It is easy to verify that the derivatives have a nontrivial null-space known as a checkerboard null-space. This is not very surprising. Indeed, consider the simple case of a uniform mass in which one linearizes the perturbation around it. The Taylor expansion yields

$$\text{det}(I+\nabla \delta \upsilon )\approx 1+\nabla \cdot \delta \upsilon .$$

Thus, for the starting point u = x and υ = 0 the first iterate gives a system similar to the Stokes system. It is well known that nonstaggered discretizations of the Stokes equation are not stable [35]. Nevertheless, simple stabilization techniques can be used to obtain a well-posed discrete system. The most common approach is a penalty method [35], where an artificial viscosity is added to the Lagrange multiplier. This additional term stabilizes the system without affecting the overall accuracy of the problem. We return to this issue in the next section.

3.3. Discretization of the objective function

In order to discretize the objective function on a regular grid, two quantities are necessary. First, we need to approximate ∫μ₀|υ|² dx on Ω_j, and second, the approximation of ∫|∇ × υ|² dx is required.

Assuming μ₀ is given in cell centers, we obtain

$${\displaystyle {\int }_{{\mathrm{\Omega }}_k}{\mu }_0{|\upsilon |}^2\mathit{\text{dx}}=h^2\frac{1}{4}{\mu }_{0,(i+\frac{1}{2},j+\frac{1}{2})}{\displaystyle \sum _{\ell =1}^2{({\widehat{\upsilon }}_{\mathit{\text{ij}}}^{\ell }+{\widehat{\upsilon }}_{i+1j}^{\ell }+{\widehat{\upsilon }}_{\mathit{\text{ij}}+1}^{\ell }+{\widehat{\upsilon }}_{i+1j+1}^{\ell })}^2+𝒪(h^2)}.}$$ (3.5)

Summing over the cells, we obtain a second order accurate discretization that can be written in matrix form as

$${\displaystyle {\int }_{\mathrm{\Omega }}{\mu }_0{|\upsilon |}^2\mathit{\text{dx}}={\widehat{\upsilon }}^{\top }{M}_{{\mu }_0}\widehat{\upsilon }+𝒪(h^2),}$$ (3.6)

where M_μ0 is a μ₀ weighted mass matrix.

Since the discretization of the curl is centered at the cell centers, it is straight-forward to use the curl matrix C to obtain

$${\displaystyle {\int }_{\mathrm{\Omega }}{|\nabla \times \upsilon |}^2}\mathit{\text{dx}}=h^2{\widehat{\upsilon }}^{\top }{C}^{\top }C\widehat{\upsilon }+𝒪(h^2).$$ (3.7)

4. Computation of a step

Given the properties of our discretization we now discuss the modification of the Lagrangian and the computation of a step.

In order to overcome the problem of stability in the discrete constraint, we use a well-known strategy utilized in computational fluid dynamics [35] and add a small penalty term to the discrete Lagrangian, defined as

$$ℒ=\frac{1}{2}{\widehat{\upsilon }}^{\top }{M}_{{\mu }_0}\widehat{\upsilon }+\frac{\beta }{2}{\widehat{\upsilon }}^{\top }{C}^{\top }C\widehat{\upsilon }+p^{\top }c(\upsilon )-\frac{\gamma }{2}{\Vert {\nabla }_h\widehat{p}\Vert }^2,$$ (4.1)

where γ is an 𝒪(h²) parameter. The latter term is added to the problem to overcome the stability problem. For a simple constraint c(υ) = ∇ · υ, it is possible to obtain an analytic expression for the optimal γ (see, for example, [35]). However, when the constraint depends on υ, such an optimal expression is not attainable. We therefore set γ = h₁h₂, which now yields a consistent and stable approximation to the Lagrangian.

To solve the problem, we use a version of inexact SQP [7]. The bottleneck in this methodology is the solution of the so-called Karush–Kuhn–Tucker (KKT) system

$$\left(\begin{array}{cc}H\hfill & c_u^{\top }\hfill \\ c_u\hfill & -S\hfill \end{array}\right)\left(\begin{array}{c}\delta \widehat{\upsilon }\hfill \\ \delta \widehat{p}\hfill \end{array}\right)=\text{RHS},$$ (4.2)

where H is an approximation to the second derivatives of the Lagrangian and c_u is the Jacobian of the constraint. Here we used H = M_μ + βC^⊤C. This approximation to the Hessian yields a semipositive definite (SPD) approximation to the (1,1) block and therefore guarantees a descent direction. We have modified standard SQP algorithms and introduced the regularization matrix S = h₁h₂Δ_h ≻ 0 in our numerical procedure.

To solve the KKT system, we have taken a number of steps. First, we have used an augmented Lagrangian approach [15] and generated the modified system

$$\left(\begin{array}{cc}{H}_{\mathit{\text{ag}}}\hfill & c_u^{\top }\hfill \\ c_u\hfill & -S\hfill \end{array}\right),$$ (4.3)

where $H_{\mathit{\text{ag}}}=H+c_u^{\top }{c}_u$. The augmentation helps to deal with the null-space of the curl operator and is commonly used in computational electromagnetics [18, 16].

Next we use a block preconditioner similar to the one discussed in [33]. It is well known that an optimal preconditioner for the KKT system is [5]

$$\left(\begin{array}{cc}{H}_{\mathit{\text{ag}}}\hfill & c_u^{\top }\hfill \\ 0\hfill & S_c\hfill \end{array}\right),$$ (4.4)

where S_c is the Schur complement

$$S_c=c_u{H}_{\mathit{\text{ag}}}^{-1}{c}_u^{\top }+S.$$

The matrix H_ag is derived from an elliptic operator; therefore it is possible to approximate its inverse using standard multigrid methods. To approximate the Schur complement matrix we drop the matrix S, which is 𝒪(h²), and use the pseudoinverse of c_u to obtain

$$S_c^{-1}\approx c_u^{†}{H}_{\mathit{\text{ag}}}{c}_u^{‡}.$$

The computation of c^† times a vector z requires the solution of a system of the form

$$c_u{c}_u^{\top }q=z.$$

Since $c_u{c}_u^{\top }$ is a discretization of an elliptic operator, this can be done using a standard multigrid technique. The numerical properties and performance of this approximation will be studied elsewhere. In our numerical experiments we have noticed that the number of iterations of the generalized minimal residual method (GMRES) [32] that was needed to solve the KKT system was almost mesh independent.

5. Numerical experiments

To test the performance of our method we considered three illustrative examples. First, we used a known deformation field to deform a synthetic image, and we verified that our algorithm could recover the deformation field accurately. We will present results both for 2D and 3D cases. We also tested the algorithm for the registration of 3D brain magnetic resonance image (MRI) datasets as an example application in medical imaging. Regarding this latter application, we certainly do not propose optimal mass transport as a general approach to image registration. Optimal mass transport may be regarded as an intensity-based approach to elastic registration, and as such is closely related to certain optical flow based methods [34, 38]. It makes sense for there to be imagery in which the intensity can be related somehow to some density, which is the case for MRI data. Here the intensity is proportional to the proton density (basically water content) of the given tissue.

5.1. Synthetic examples

Since the solution of the problem is also the solution of the Monge–Ampère equation [11], it is easy to construct an analytic example. In order to do this, we consider the following function:

$$\varphi (x_1,x_2)=\frac{1}{2}(x_1^2+x_2^2)+c\cdot e^{-\frac{1}{2}{(x_1-\frac{1}{2})}^2/{\sigma }_1^2}\cdot e^{-\frac{1}{2}{(x_2-\frac{1}{2})}^2/{\sigma }_2^2}\in {[0,1]}^2,$$

where c, σ₁, and σ₂ are parameters chosen to create a unique deformation field. Differentiating ϕ with respect to x = (x₁, x₂), we obtain u = (u₁, u₂),

$$\begin{array}{c}{u}_1=x_1-c\cdot ((x_1-0.5)/{\sigma }_1^2)\cdot e^{-\frac{1}{2}{(x_1-\frac{1}{2})}^2/{\sigma }_1^2}\cdot e^{-\frac{1}{2}{(x_2-\frac{1}{2})}^2/{\sigma }_2^2},\hfill \\ u_2=x_2+c\cdot ((x_2-0.5)/{\sigma }_2^2)\cdot e^{-\frac{1}{2}{(x_1-\frac{1}{2})}^2/{\sigma }_1^2}\cdot e^{-\frac{1}{2}{(x_2-\frac{1}{2})}^2/{\sigma }_2^2}).\hfill \end{array}$$

We employed a standard 2D phantom image from MATLAB to serve as μ₁(x₁, x₂), and define μ₀(x₁, x₂) by

$${\mu }_0≔\text{det}(\nabla u){\mu }_1(u).$$ (5.1)

We computed μ₀(x₁, x₂) via linear interpolation. The deformed grid, deformation vector field, μ₀, and μ₁ thus obtained are shown in Figure 2.

FIG. 2.

2D Synthetic Example. Two image pairs (a) and (b) with a known deformation, mapping one to the other. (c) shows the deformed grid, and (d) shows the deformation vector field with superimposed gradient contours of the determinant of the Jacobian.

We next inputed the μ₁ and μ₀ obtained in this manner into our solver to find the transformation u using our algorithm.

We also performed the same experiment for a 3D brain MRI dataset. The deformation field from the 2D example was symmetrically extended in the third dimension and the deformed image was computed in the same way. For both experiments we use β = 10². We terminated our algorithm after 100 iterations or when the curl of the solution was 4 orders of magnitudes smaller than its initial size (in the infinity norm).

Table 1 shows the ∞-norm of the error between the known and compute deformation fields at different grid sizes. The tables clearly demonstrate quadratic convergence of our method to the true solution, which is expected from the discretization error used in our numerical approximations.

TABLE 1.

Convergence for the 2D and 3D experiments.

Grid size	Error (infinity norm) in 2D	Error (infinity norm) in 3D
2−3	1.1 × 10−4	1.2 × 10−2
2−4	2.5 × 10−5	3.8 × 10−3
2−5	6.0 × 10−6	9.0 × 10−4
2−6	2.4 × 10−7	2.2 × 10−5
2−7	5.9 × 10−8	5.1 × 10−6
2−8	1.2 × 10−8

To demonstrate the efficiency of our algorithm, we present in Table 2 the number of projection steps used for each of the examples on each level. Note that we have not used a multilevel procedure that could speed up calculations. This was done in order to demonstrate mesh independence of the method. The incorporation of a multilevel method is beyond the scope of the present paper and will be done in future work.

TABLE 2.

Number of projection iterations for the 2D and 3D experiments.

Grid size	# proj iter (2D)	# proj iter (3D)
2−3	56	67
2−4	32	46
2−5	28	34
2−6	27	35
2−7	28	36
2−8	26	35

Table 2 clearly demonstrates the mesh-independence property of the modified SQP algorithm.

5.2. A 3D brain example

We also tested our approach for multimodal registration of a 3D brain atlas to an MRI. Our goal is the identification of cortical structures by mapping a publicly available atlas [24] to the scan of a patient. The MRI of the atlas is a spoiled gradient recalled image acquired on a 1.5-Tesla General Electric Signa System (GE Medical Systems, Milwaukee) with 256 × 256 × 124 voxels and voxel dimension of 0.92 × 0.92 × 1.5 mm. The patient scan is an MPRAGE acquired on a Siemens 3T long bore machine using an 8 channel head coil. The resolution of the scan is 256 × 256 × 144 with voxel dimension 0.54 × 0.54 × 1.0 mm (see Figure 3(b)).

FIG. 3.

Registration results.

The parcellation of the cortex can be encoded by partitioning the boundary between cortex and white matter into anatomical regions [12]. The label map of cortical structures can then be inferred from this partition by propagating the labeling along the boundary to the entire cortex. The pipeline described below will apply this concept for the parcellation of the cortex to the high resolution scan.

The input of the pipeline consists of the atlas, the high resolution scan, as well as a segmentation of the scan into the major tissue classes. In the first step, we coarsely align the atlas to the image data using the B-spline implementation by Rohlfinger [31] with a final spacing of the grid nodes of 2.5 mm. This results in a coarse alignment of the scans. The algorithm has difficulties in mapping the folds of the white matter due to the inherent constraints of the B-spline representation. We then reduce the atlas to the white matter including the parcellation of the cortex along the boundary between gray and white matter (see Figure 2(a)). Afterwards, we refine the alignment of this new atlas to the white matter of the high resolution scan using our optimal mass transport registration approach.

Registration using optimal mass transport is a highly flexible approach that is, unlike B-splines, not constrained to a set of control points. The intensities in the two input datasets are first normalized and rescaled to make sure that both have the same total mass. The white matter registration with the proposed algorithm took just 12 iterations to converge with 2 iterations of the projection to constraint per iteration. The ∇ × u (convergence metric) was reduced to an order of 10⁻³ indicating an optimal map. Figure 2(c) shows the resampled images with 3D views of the corresponding deformation grid in Figure 4. The difference (see Figure 2(d)) between the target indicated in Figure 2(b) and the resampled image shows that our approach accurately aligned the folds.

FIG. 4.

Parecellation results.

After this local alignment, the folds of the atlas should perfectly align with those of the high resolution scan. The parcellation of the folds of the atlas, therefore, also encodes the parcellation of the same region in the high resolution scan. We then complete the cortex parcellation of the high resolution scan by confining the Voronoi diagram of the aligned atlas to the gray matter mask of the high resolution scan. The results in Figure 5 show the corresponding segmentation when applying the deformation map of the B-spline registration and our approach to the label map of [24], and propagating the labels to the cortex via the Voronoi diagram.

FIG. 5.

Deformed grid on white matter slices (left) and 3D volume (right).

6. Summary

In this paper, we have investigated a new method for the computation of the optimal L₂ mass preserving mapping, and derived a novel numerical framework for its efficient computation. We proposed a direct variation approach, and showed how one may obtain a descent direction. A second order accurate discretization to the problem was presented. Further, we presented several illustrative numerical experiments in both two and three dimensions which indicated the effectiveness of our method. There are a number of key directions for future research. In particular, we plan to improve our preconditioners to the KKT system, which is the bottleneck of the computation, as well as use adaptive multilevel mesh refinement. Finally, there are several possible applications of our optimal mass transport technique which we intend to explore. These include texture mappings in computer graphics, and visual tracking in controlled active vision.