Optimal Mass Transport for Shape Matching and Comparison

Abstract

Surface based 3D shape analysis plays a fundamental role in computer vision and medical imaging. This work proposes to use optimal mass transport map for shape matching and comparison, focusing on two important applications including surface registration and shape space. The computation of the optimal mass transport map is based on Monge-Brenier theory, in comparison to the conventional method based on Monge-Kantorovich theory, this method significantly improves the efficiency by reducing computational complexity from O(n²) to O(n). For surface registration problem, one commonly used approach is to use conformal map to convert the shapes into some canonical space. Although conformal mappings have small angle distortions, they may introduce large area distortions which are likely to cause numerical instability thus resulting failures of shape analysis. This work proposes to compose the conformal map with the optimal mass transport map to get the unique area-preserving map, which is intrinsic to the Riemannian metric, unique, and diffeomorphic. For shape space study, this work introduces a novel Riemannian framework, Conformal Wasserstein Shape Space, by combing conformal geometry and optimal mass transport theory. In our work, all metric surfaces with the disk topology are mapped to the unit planar disk by a conformal mapping, which pushes the area element on the surface to a probability measure on the disk. The optimal mass transport provides a map from the shape space of all topological disks with metrics to the Wasserstein space of the disk and the pullback Wasserstein metric equips the shape space with a Riemannian metric. We validate our work by numerous experiments and comparisons with prior approaches and the experimental results demonstrate the efficiency and efficacy of our proposed approach.

1 Introduction

In recent decades, with the fast development of 3D scanning technologies, there has been much research into surface representations for 3D shape analysis. Comparing with other approaches such as volume measurements [27], mathematical morphology [26], medial axis [8], surface based approach offers many advantages including: (1) it offers an accurate shape representation even for local subtle shape changes; (2) it can compute some physically natural measurements, e.g. elasticity and heat diffusion; (3) it has solid mathematical foundations on which one can develop numerically efficient algorithms and achieve global shape analysis, even on shapes with complicated topological structures. In computer vision research, numerous surface based approaches have been proposed to solve various shape analysis problems, such as surface matching [18], [7], [12], [34], [37], [45], [33], anatomical morphometry analysis [50], 3D object recognition and tracking [30], [56] and 3D shape search engine [13]. Even so, a theoretically rigorous and numerically efficient surface based approach would be highly advantageous in this research field. Here we propose to apply the Monge-Brenier optimal mass transport (OMT) theory for shape matching and comparison, focusing on surface registration and a generic shape space model–conformal Wasserstein shape space.

Optimal Mass Transport

Monge raised the classical Optimal Mass Transport Problem that concerns determining the optimal way, with minimal transportation cost, to move a pile of soil from one place to another [9]. Kantorovich [31] has proved the existence and uniqueness of the optimal transport plan based on linear programming. Monge-Kantorovich optimization has been used in numerous fields from physics, econometrics to computer science including data compression and image processing [41]. Recently, researchers have realized that optimal transport could provide a powerful tool in image processing, if one could reduce its high computational cost [16], [54]. However, it has one fundamental disadvantage that the number of variables is O(k²), which is unacceptable to computer vision and medical imaging applications since a high resolution 3D surface normally includes up to hundreds of thousands of vertices.

An alternative Monge-Brenier optimization scheme can significantly reduce the number of variables to be optimized. In late 1980’s, Brenier [11] developed a different approach for a special class of optimal transport problems, where the cost function is a quadratic distance. Brenier’s theory shows that the optimal transport map is the gradient map of a special convex function. Assume the target domain is discretized to n samples, the Monge-Brenier’s approach reduces the unknown variables from O(n²) to O(n), which greatly reduces the computation cost, and improves the efficiency. In our framework, we take Monge-Brenier’s approach. However, our work is based on the newly discovered variational principle [20] which is the underspinning of Monge-Brenier’s approach. Our framework is general and works with any valid measures, μ and ν, defined on two surfaces. Within the scope of this paper, we only consider the area induced measures. As a result, we will use the term OMT-Map and area-preserving map interchangeably. Our parameter domains could be either topological disk (including rectangles and any convex planar domain) (Figure 1 and 2) or topological sphere domains (Figure 3).

Fig. 1.

Comparison of geometric mappings for Armadillo surface model to a planar unit disk: (a) Front view; (b) Back view; (c) Optimal mass transport map result; (d) Conformal mapping result. The results show that conformal mapping has much more area distortions on head and hands areas. The normal information on the original surfaces is preserved and used for rendering. By the shading information on the planar domain ((c) and (d)), the correspondence is illustrated. The hand zoom-in image of (d) shows that the conformal map shrinks the fingers to very tiny areas which may cause numerical instability, while the hand zoom-in image of (c) demonstrates the optimal mass transport method gives a good one-to-one mapping result.

Fig. 2.

Circle-packing texture mappings for conformal parameterization (CFP) (a) and area-preserving parameterization (APP) for the model of a human head, with the planar unit square parameter domain (b). The mappings to the parameter domain results are also shown in (c) and (d), respectively. (e) to (h) are the histograms of angle distortions and area distortions, which demonstrate the accuracy of the Optimal Mass Transport map.

Fig. 3.

Comparison of conformal parameterization (CFP) and area-preserving parameterization (APP) of a Bimba sculpture model, shown in (a) and (b), with the spherical parameter domain. The normal information on the original surfaces is preserved and used for rendering.(g) to (j) are the histograms of angle distortions and area distortions.

Surface Registration

Studying the original surfaces could be extremely difficult when shapes are irregular and very complex, such as human body or human brain cortical surfaces. One effective and common approach is to first parameterize the original 3D domain to some classical parameter domains, such as planar or spherical domain, then register or analyze 3D surfaces through these canonical space [43], [18], [62]. This approach has the advantage of converting complex shapes to simple ones, reducing the computational complexity and improving the efficiency. Conformal geometry based methods have been frequently applied for shape parameterizations [10], [38], [21], [44], [56], [61], [28], [3]. Conformal mapping can keep angle unchanged and preserve local shapes (conformal), but may also produce huge area distortions. In Figure 1, the Armadillo model is mapped onto the planar unit disk. Frame (d) shows the image of a conformal mapping, where the head area shrunk exponentially to the height of the model and hard to be recognized. Other extruding parts, such as hands with fingers shown in the zoom-in image, the exponential area distortions may easily exceed machine precisions, leading to problems and failures of surface matching and registration.

The conformal mapping in (d) pushes forward the area element on the Armadillo model to the planar disk. Then the unique optimal mass transport map is carried out from the disk with the push-forward measure in (d) to the disk with Euclidean measure. The composition of the conformal mapping and the optimal mass transport map is an area-preserving map from the surface to the Euclidean disk. The mapping result is shown in (c), where the head and figures occupy the same areas as those on the original surface. Area-preserving mapping avoids the huge area distortion, thus is more robust and intuitive for processing. Furthermore, this area-preserving mapping is intrinsic to the Riemannian metric, unique, and diffeomorphic. Therefore, the OMT map may help provide practical solutions for general 3D shape analysis tasks, such as surface parameterization, surface matching and comparison.

Conformal Wasserstein Shape Space

Shape space models provide suitable mathematical and computational descriptions for both shape representation and comparisons [32] and they were actively studied in computer vision field (as reviewed in [59]). With the proposed optimal transport theory, here we present its application for modeling shape spaces and measuring shape distances.

Let (M, g) be a Riemannian manifold, (M) is the space of all probability measures defined on M. Given two measures μ, ν ∈ (M), there is an optimal mass transport map T : M → M, the transportation cost of T is defined as the Wasserstein distance between μ and ν, denoted as W(μ, ν). It can be shown that W is a metric of the Wasserstein space , the pair ( (M), W) is called the Wasserstein metric space, which reflects the Riemannian metric of (M, g).

Consider a marked metric surface with the disk topology (s, g), with two markers (p, q), p ∉ ∂M, q ∈ ∂M, there is a unique conformal mapping φ : s → , φ maps p and q to 0 and 1 respectively. The corresponding conformal factor induced by φ is λ : → ℝ. The area element of s is pushed forward to the disk, represented as μ_(s,g):= e^2λ(x, y)dx∧dy. Then we convert a marked metric surface to a probability measure Γ : (s, g) ↦ μ_(s,g).

All the marked metric surfaces with the disk topology quotient the isometry group and the scaling transformation group form the shape space . The mapping Γ : → ( ) is an injective mapping, the pull back metric induced by Γ gives a Riemannian metric in . We call this metric space ( , Γ^*W₂) as the Conformal Wasserstein Shape Space.

The conformal Wasserstein shape space is a novel Riemannian framework to study shape space. This framework has solid theoretic foundation and efficient computational algorithms. It may provide a metric space for shape comparison, shape clustering and classification, shape retrieval and so on.

Advantages

To our knowledge, this work is the first one to take Monge-Brenier theory to study 3D shape analysis problems. It has the following merits:

1. Theoretic soundness: According to convex geometry theorem developed by Brenier [11] and earlier work by Alexandrov [2], the solution exists and is unique. Furthermore, the area of each cell equals to the prescribed measure exactly. When the sampling density goes to infinity, the OMT map converges to the continuous area-preserving map.

2. Generality and efficiency: The method is general for arbitrary dimension, which has the potential to lead to high dimensional parameterizations. For surface case, it can handle both topological disks and topological spheres and achieve bijective surface mapping. Comparing to the conventional Monge-Kantorovich method, our approach reduces the complexity from O(n²) to O(n). It is equivalent to a convex optimization problem, which can be carried out using Newton’s method efficiently. Since the computation is based on classical power diagram, the algorithm can be implemented using any existing numerical software package easily.

3. Flexibility: Our algorithm can take different canonical space as the parameter domain. The algorithm can also fully control the mapping areas of different regions of interest. For example, it can enlarge the regions of interests or regions with more geometric or textural features, and shrink less interesting regions, which are shown in Figure 5. It can improve the visualization experience and help shape analysis.

Fig. 5.

Importance-driven parameterization of a Buddha model.

2 Previous Work

2.1 Optimal Mass Transport

For optimal mass transport, some approaches based on Monge-Kantorovich theory have been proposed. Zhu et al. [65] applied optimal mass transport for flattening blood vessel in an area preserving mapping for medical visualization. Haker et al. [25] proposed to use optimal mass transport for image registration and warping, the method is parameter free and has the unique global optimum. Dominitz and Tannenbaum [16] proposed to use optimal mass transport for texture mapping. The method first starts with an angle-preserving mapping and then refines the mapping using the mass transport procedure derived via a gradient flow. Rehman et al. [54] presented a method for 3D image registration based on optimal mass transport problem. Meanwhile, they stress the fact that the optimization of OMT is computationally expensive and emphasize that it is important to find efficient numerical methods to solve this issue, and it is crucial to extend the results to 3D surfaces.

There are also some works based on Monge-Brenier theory. Our prior work [51], [63] proposed an area-preserving mapping method for brain morphological study and visualization, but they can only compute the maps from the unit disk domain with Euclidean measure to another disk with general measure. Merigot [39] has proposed a multi-scale approach to solve optimal transport problem. de Goes et al. [15] have provided an optimal transport driven approach for 2D shape reconstruction and simplification. Recently they have presented a formulation of capacity-constrained Voronoi tessellation as an optimal transport problem for image processing [14]. This method produces high-quality blue noise point sets with improved spectral and spatial properties. In summary, except our prior work [51], [63], other Monge-Brenier theory based methods were all applied to 2D image matching and registration. By contrast, our work is the first one to apply Monge-Brenier based optimal mass transport method to study 3D shape analysis.

2.2 Surface Registration

There is a vast literature on surface/image registration, a thorough survey on deformable medical image registration can be found in [46], which gives a rigorous treatment for registration problem. Let S and T be source and target images defined in an image domain Ω, a transformation W : Ω → Ω is a diffeomorphism of the domain. Then [46] formulate the registration as an optimization problem with the energy form (T, S ∘ W) + (W), where measures the deformation, measures the regularity of the mapping W. The survey covers methods which minimizing different energies . The elastic body models optimize the elastic deformation energy; the viscous fluid models minimizes the fluid dynamics energy; the diffusion model deforms the harmonic energy (membrane energy); the curvature registration method optimizes the bending energy; the flows of diffeomorphisms finds the geodesic in the shape space. Some other energy terms incorporate the landmark constraints, or the constraints for the mapping, such as the mapping should belong to homeomorphism, volume preserving or rigid motion group.

The survey does not cover methods based on optimal mass transport or conformal mapping. In contrast, our method is based on optimal transport map and conformal mapping. Given two metric surfaces (S₁, g₁) and (S₂, g₂), which are topological disks, first we map them to the planar disk by conformal mappings, φ_k : S_k → , the induced conformal factors are λ_k, k = 1, 2. Then on the disk, there are two measures μ_k = e^2λ_k(x,y)dx ∧ dy. We find an OMT map τ : ( , μ₁) → ( , μ₂), the composition ${\phi }_2^{-1}\circ \tau \circ {\phi }_1$ gives the registration.

From differential geometry, any mapping between two surfaces will induce area distortions and angle distortions. Unless the two surfaces are isometric, one of the two types of distortions is unavoidable. Our registration goal is to minimize both angle and area distortions. Conformal mapping φ_k has 0 angle distortion; optimal mass transport map τ has 0 measure distortion. The work of [16] shows that this type of mapping minimizes both angle and area distortion.

In computer vision and medical imaging research, feature landmarks, such as sulci lines on brain surfaces or extreme points on general surfaces, are usually required to guide surface registration [52], [36], [64], [57], [35], [24]. Kurtek et al. [35] proposes a constrained optimization approach that simultaneously computes dense correspondences and geodesics between surfaces. In this work, if there are landmark constraints, after the optimal mass transport map τ, we add an harmonic map η to enforce the alignment of the landmarks. Although it shares some similar motivation with other landmark constrained surface registration work, our method has a few fundamental distinctions from that of [35]. First, our method is intrinsic while their method considers the embedding; second, our method computes the registration directly while their method finds the deformation path; third, our method can handle non-isotopic surfaces but their method can not.

2.3 Shape Space

A popular Riemannian framework for modeling shape space is to measure the similarity between two shapes by the deformation between them. A deformation process is a path in the shape space, the length of path gives the amount of deformation. Among all paths, the one with the minimal length is the geodesic. The length of the geodesic gives the distance between the shapes.

Shape space is the space of orbits of the reparameterization group acting on the space of immersions [48], [60], [47]. Namely, fix a smooth n − 1 dimensional manifold M, let Γ be Lie group of all diffeomorophisms of M, which is the reparameterization group [49]. The shape space is the space of all smooth immersions quotient by Γ, denoted as . Riemannian metrics measure infinitesimal deformations. Given an immersion f : M → ℝⁿ, and two deformation vector fields on f, h, k : M → ℝⁿ, h, k ∈ T_f( ), one designs a reparameterization invariant metric 〈,〉_f, such that 〈h, k〉_f = 〈h ∘ γ, k ∘ γ 〉_f∘γ, ∀γ ∈ Γ, the construction may involve the metric of the ambient space ℝⁿ, the metric of the immersion f, the covariant derivatives or differential operators on f and so on.

The reparameterization invariant metric constructed in [6], [5] uses the volume form and the mean curvature of the immersion f, the metric in [34] uses the area multiplication factor of f. [35] extends the work in [34] by adding landmark constraints. Instead of considering the whole reparameterization group Γ, only a subgroup fixing the landmarks is applied. The infinitesimal generators of such subgroup are constructed using an elegant technique based on spherical harmonics. [29] represents the embedding f by its area element and normal vector (r, n), the so-called square root normal fields (SRNF), and the reparameterization invariant metric is built on SRNF.

To highlight the pros and cons of our method, we provide item-by-item comparisons between our method and the elastic shape metric methods [29], [34], [35].

• Extrinsic v.s. Intrinsic: The elastic shape metric methods consider the extrinsic embeddings, yet our proposed method only focuses on the intrinsic Riemannian metric.

  1. The elastic shape (extrinsic) metric methods assume there is a deformation between two shapes, which requires two shapes are isotopic. However, the proposed intrinsic method is applicable for non-isotopic shapes.

  2. The elastic shape (extrinsic) metric methods need to embed the surfaces into ℝ³, and focus on designing reparameterization invariant metrics. However, our intrinsic method solely depends on the Riemannian metric and does not need the ℝ³ embeddings. Our method uses normalized conformal mapping and optimal mass transport map, which are unique, thus there is no reparameterization ambiguity. From this point of view, our intrinsic method is simpler and clearer.

• The elastic shape (extrinsic) metric methods use the geodesic length as the distance, and give the deformation sequence in the shape space. The proposed intrinsic metric method uses the cost of optimal mass transport map as the distance, but currently it is unable to provide the deformations.

• Restriction: The elastic shape metric methods can explore the full group of diffeomorphisms, which is powerful for matching and comparison. However, given source and target measures, the proposed optimal mass transport map is unique and can only provide the unique diffeomorphism. We can change the diffeomorphism by designing the target measure. Then the unique optimal mass transport map can be tailored, and we can exhaust the full group of diffeomorphisms.

• Flexibility: The elastic shape metric methods are more flexible. They can freely add more constraints [35]. But the currently proposed optimal mass transport map is difficult to add landmark constraints.

• Applicability: The elastic shape metric methods can handle higher genus surfaces if they are parameterized into a common domain [35]. Our current implementation is on genus zero surface. In theory our method can be generalized to any topology.

• Convergency: The computation of optimal mass transport map is a convex optimization problem guaranteed with the unique global optimum, and can be solved by Newton’s method. However, the optimization problems of the elastic shape metric methods are solved by gradient based method, which has an obvious limitation of converging to a local solution [34].

3 Theoretic Background

This section briefly introduces the theoretic background of conformal mapping and Optimal Mass Transport theory. We refer readers to a classical textbook [22] for conformal geometry, the seminal papers [31] on optimal transport map with Kantorovich’s method, and [20] for more detailed proofs of the proposed method.

3.1 Optimal Mass Transport

Monge [9] raised the optimal mass transport problem in the 18th century.

Problem 1 (Optimal Mass Transport)

Suppose (X, μ), (Y, ν) are metric space with probabilities measures, which have the same total mass ∫_X μdx = ∫_Y νdy. A map T : X → Y is measure preserving, if for any measurable set B ⊂ Y, μ(T⁻¹(B)) = ν(B). Given a transportation cost function c : X × Y → ℝ, find the measure preserving map T : X → Y that minimizes the total transportation cost

$$\mathcal{C}(T):={\int }_{X}c(x,T(x))d\mu (x).$$ (1)

In the 1940s, Kantorovich introduced the relaxation of Monge’s problem and solved it using linear programming method [31].

At the end of 1980’s, Brenier [11] discovered the intrinsic connection between optimal mass transport map and convex geometry.

Definition 1 (Convex Function)

Suppose f : X → ℝ is a function, f is convex if $f(\frac{x_1+x_2}{2})\le \frac{1}{2}(f(x_1)+f(x_2))$. If f is C² continuous convex function, its Hessian matrix is semi-positive definite. $\left(\frac{{\partial }^{2}f}{\partial x_i\partial x_j}\right)\ge 0$.

Definition 2 (Gradient Map)

Suppose f : X → ℝ is a function, the gradient map ∇f : X → Y is defined as x ↦ ∇ f(x).

Theorem 1 (Brenier)

Suppose X and Y are the Euclidean space ℝⁿ, and the transportation cost is the quadratic Euclidean distance c(x, y) = |x − y|². If μ is absolutely continuous and μ and ν have finite second order moments, then there exists a convex function f : X → ℝ, its gradient map ∇f gives the solution to the Monge’s problem. Furthermore, the optimal mass transport map is unique.

3.2 Wasserstein Metric Space

Suppose (M, g) is a Riemannian manifold with a Riemannian metric g.

Definition 3 (Wasserstein Space)

Let (M) denote the space of all probability measures μ on M with finite p^th moment, where p ≥ 1. Suppose there exists some point x₀ ∈ M that ∫_M d(x, x₀)^p dμ(x) < + ∞, where d is the geodesic distance induced by g.

Given two probability μ and ν in , the Wasserstein distance between them is defined as the transportation cost induced by the optimal mass transport map T : M → M,

$$W_p(\mu ,\nu ):=\underset{T_{\#}\mu =\nu }{inf}{\left({\int }_{M}d{(x,T(x))}^{p}d\mu (x)\right)}^{\frac{1}{p}}.$$

The following theorem plays a fundamental role for the current work

Theorem 2

The Wasserstein distance W_p is a Riemannian metric of the Wasserstein space (M). Detailed proof can be found in [55].

3.3 Discrete Optimal Mass Transport

We focus on the Brenier’s approach. Suppose μ has compact support on X, define Ω = supp μ = {x ∈ X|μ(x) > 0}, assume Ω is a convex domain in X. The space Y is discretized to Y = {y₁, y₂, ···, y_k} with Dirac measure $\nu ={\sum }_{j=1}^k{\nu }_j\delta (y-y_j)$.

We define a height vector h = (h₁, h₂, ···, h_n) ∈ ℝ^k, consisting of k real numbers. For each y_i ∈ Y, we construct a hyperplane defined on X,

$${\pi }_i(\mathbf{h}):\langle x,y_i\rangle +h_i=0.$$ (2)

Define a function

$$u_{\mathbf{h}}(x)=\underset{i=1}{\overset{k}{max}}\{\langle x,y_i\rangle +h_i\},$$ (3)

then u_h(x) is a convex function. We denote its graph by G(h), which is an infinite convex polyhedron with supporting planes π_i(h). The projection of G(h) induces a polygonal partition of Ω,

$$\mathrm{\Omega }=\underset{i=1}{\overset{k}{\cup }}{W}_i(\mathbf{h}),$$ (4)

where each cell W_i(h) is the projection of a facet of the convex polyhedron G(h) onto Ω,

$$W_i(\mathbf{h})=\{x\in X\mid u_{\mathbf{h}}(x)=\langle x,y_i\rangle +h_i\}\cap \mathrm{\Omega }.$$ (5)

Note that, this partition is equivalent to a power diagram, denoted as D(h), as explained in [20]. The area of W_i(h) is given by

$$w_i(\mathbf{h})={\int }_{W_i(\mathbf{h})}\mu (x)dx.$$ (6)

The convex function u_h on each cell W_i(h) is a linear function π_i(h), therefore, the gradient map

$$\mathit{grad}{u}_{\mathbf{h}}:W_i(\mathbf{h})\to y_i,i=1,2,\cdots ,k.$$ (7)

maps each W_i(h) to a single point y_i.

The following theorem plays a fundamental role for discrete optimal mass transport theory,

Theorem 3

Given a convex domain Ω ⊂ ℝⁿ, with measure density μ : Ω ∈ ℝ, and a discrete point set Y = {y₁, ···, y_k} with discrete measures ν = {ν₁, ···, ν_k}. Suppose ${\sum }_{j=1}^n{\nu }_j={\int }_{\mathrm{\Omega }}\mu ,{\nu }_j>0$.

Then there must exist a height vector h = {h₁, ···, h_k} unique up to translations, such that the convex function Eqn. 3 induces the cell decomposition of Eqn. 4. And the following area-preserving constraints are satisfied for all cells,

$${\int }_{W_i(\mathbf{h})}\mu (x)dx={\nu }_i,i=1,2,\cdots ,n.$$ (8)

Furthermore, the gradient map grad u_h optimizes the following transportation cost

$$\mathcal{C}(T):={\int }_{\mathrm{\Omega }}{\mid x-T(x)\mid }^2\mu (x)dx.$$ (9)

The existence and uniqueness was first proven by Alexandrov [2] using a topological method; the existence was also proven by Argmstrong [4], the uniqueness and optimality was proven by Brenier [11]. Recently, Gu et al. [20] gives a novel proof for the existence and uniqueness based on the variational principle, which leads to the computational algorithm directly.

Define the admissible space of height vectors $H_0:=\{\mathbf{h}\mid {\sum }_{j=1}^k{h}_j=0\mathit{and}{\int }_{W_i(\mathbf{h})}\mu >0,\forall i=1,\cdots ,k,\}$. Then define the energy E(h),

$$E(\mathbf{h})={\int }_{\mathrm{\Omega }}{u}_{\mathbf{h}}(x)\mu (x)dx-\sum _{i=1}^k{\nu }_i{h}_i.$$ (10)

or equivalently

$$E(\mathbf{h})={\int }_{\mathbf{0}}^{\mathbf{h}}\sum _{i=1}^k{w}_i(\eta )d{\eta }_i-\sum _{i=1}^k{\nu }_i{h}_i+C,$$ (11)

where C is a constant. Consider the shape bounded by the graph G(h), the horizontal plane {x_n+1 = 0} and the cylinder consisting of vertical lines through ∂Ω, the volume of the shape is given by the first term.

The gradient of the energy is given by

$$\nabla E(\mathbf{h})={(w_1(\mathbf{h})-{\nu }_1,\cdots ,w_k(\mathbf{h})-{\nu }_k)}^T,$$ (12)

Suppose the cells W_i(h) and W_j(h) intersects at an edge e_ij = W_i(h) ∩ W_j(h) ∩ Ω, then the Hessian of E(h) is given by

$$\frac{{\partial }^{2}E(\mathbf{h})}{\partial h_i\partial h_j}=\{\begin{array}{ll}\frac{{\int }_{e_{ij}}\mu (x)dx}{\mid y_j-y_i\mid }\hfill & W_i(\mathbf{h})\cap W_j(\mathbf{h})\cap \mathrm{\Omega }\ne \varnothing \hfill \\ 0\hfill & \mathit{otherwise}\hfill \end{array}.$$ (13)

The following theorem lays down the theoretic foundation of our OMT map algorithm.

Theorem 4 (Discrete Optimal Mass Transport [20])

If Ω is convex, then the admissible space H₀ is convex, so is the energy (Eqn. 10). Moreover, the unique global minimum h₀ is an interior point of H₀. And the gradient map (Eqn. 7) induced by the minimum h₀ is the unique optimal mass transport map, which minimizes the total transportation cost (Eqn. 9).

The proof of Theorem 4 is reported in [20]. Due to the convexity of the volume energy Eqn. 10, With this theory, the global minimum can be obtained efficiently using Newton’s method. Comparing to Kantorovich’s approach, where there are O(n²) unknowns, this approach has only O(n) unknowns.

3.4 Conformal Mapping

In order to compute the area-preserving mappings between surfaces, we need to flatten 3D surfaces onto the 2D planar domain first. The most commonly used technique for flattening a surface is conformal mapping.

Suppose (S, g) is a surface with a Riemannian metric g, a diffeomorphism ϕ : (S, g) → ( , dx² + dy²) maps the surface to the planar unit disk . We say ϕ is a conformal (angle-preserving) mapping, if g(x, y) = e^{2λ(x, y)}(dx² + dy²), where the so-called conformal factor λ : S → ℝ represents the area distortion. Our work is based on the following theorems.

Theorem 5 (Riemann Mapping)

Suppose (S, g) is an oriented metric surface, which is of genus zero with a single boundary. Given an interior point p ∈ S and a boundary point q ∈ ∂S, there is a unique conformal mapping ϕ : S → , satisfying ϕ(p) = 0 and ϕ(q) = 1.

Theorem 6 (Spherical Conformal Mapping)

Suppose (S, g) is an oriented metric surface, which is of genus zero and closed. Given an interior point p₁, p₂, p₃ ∈ S, there is a unique conformal mapping ϕ : S → ∪ {∞}, which maps {p₁, p₂, p₃} to {0, 1, ∞}.

The conformal mapping can be computed using either holomorphic differential method or discrete surface Ricci flow as described in [22] and [23]. The theoretic foundation of discrete Ricci flow can be found in [19].

We follow the approach in [16], which gives us an area-preserving mapping, which is also close to shape-preserving as explained in [16]. We map the surface (S, g) onto the planar disk using a conformal mapping. Then we construct an optimal mass transport map from ( , e^2λdxdy) to ( , dxdy). The composition is an area-preserving mapping from the surface to the disk.

3.5 Conformal Wasserstein Shape Space

Combing optimal transportation and conformal mapping theories, we can construct a shape space framework.

We consider all oriented metric surfaces (M, g) with the disk topology, namely M is of genus 0 and with a single boundary ∂M. There are two markers (p, q) ⊂ M, p is an interior point, q is a boundary point. We call (M, g, p, q) as a marked metric surface. The set of all marked metric surfaces is denoted as , := {marked metric surfaces}.

Two marked metric surfaces are equivalent, if there is a normalized isometric diffeomorphism ϕ : (M₁, g₁, p₁, q₁) → (M₂, g₂, p₂, q₂), such that ϕ preserves metrics ϕ^*g₂ = g₁ and preserves markers ϕ(p₁) = p₂, ϕ(q₁) = q₂. The product of the normalized isometry diffeomorphism group and the scaling group is denoted as G, G := {normalized isometries} ⊗ {scaling}.

We define the shape space as

$$\mathcal{S}:=\mathcal{M}/G.$$ (14)

Let (M, g, p, q) ∈ is a normalized marked metric surface, such that its total area is π. In the following discussion, we always omit the markers (p, q), and assume the total area is π. Then according to Riemann mapping theorem, there is a unique conformal mapping ϕ : M → , where is the unit planar disk with Euclidean metric dx² + dy², such that ϕ(p) = (0, 0) and ϕ(q) = (1, 0). Then g = e^2λ(x,y)(dx² + dy²). ϕ push forward the area element on (M, g) to the disk as

$${\mu }_{(M,\mathbf{g})}:=e^{2\lambda (x,y)}dx\wedge dy.$$ (15)

This gives an injective mapping Γ : → ( ), Γ : (M, g) ↦ μ_(M,g). The Wasserstein metric on the Wasserstein space ( ) is pulled back to ,

$$d_{\mathcal{S}}((M_1,{\mathbf{g}}_1),(M_2,{\mathbf{g}}_2)):=W_2({\mu }_{(M_1,{\mathbf{g}}_1)},{\mu }_{(M_2,{\mathbf{g}}_2)}).$$ (16)

We call the metric space ( , ) the conformal Wasserstein shape space. The constructed shape space enjoys numerous advantages such as that it is intrinsic geometric structure and does not have reparameterization ambiguity, etc.

4 Algorithms

This section gives the algorithmic implementation details for optimal mass transport map (OMT-Map) generation using our new variational framework. Based on the OMT-Map algorithm, we introduce surface area-preserving parameterization algorithm on simply connected surfaces, and the computation for conformal Wasserstein distance between surfaces.

4.1 Optimal Mass Transport Map (OMT-Map) Algorithm

Assume Ω is a convex planar domain with measure density μ, P = {p₁, ··· , p_k} is a point set with measure ν = {ν₁, ··· , ν_k}, such that ${\int }_{\mathrm{\Omega }}\mu (d)dx={\sum }_{i=1}^k{\nu }_i$.

According to the discussion in previous section, the OMT-Map can be obtained by minimizing the convex energy in Eqn. 10. In practice, the energy can be optimized using Newton’s method, which requires the computation of the energy gradient using Eqn. 12, and the Hessian matrix using Eqn. 13. The method is straightforward, but the initialization and the step length selection need to be specially addressed.

Initialization

By translating and scaling, P could be inside Ω, P ⊂ Ω. At the beginning, we set each power weight h_i to be 0, namely h = 0, and compute the power diagram D(P, h) and the Delaunay triangulation T (P, h). In this scenario, D(P, h) is a conventional voronoi diagram.

Step Length Selection

Suppose at the k-th step in the optimization, the power weight vector is h^k, and all Voronoi cells W_i(h^k) are non-empty. Then the Hessian matrix H_k in Eqn. 13 is positive definite on the hyper-plane {h|Σ_i h_i = 0}. At the k + 1-step, we set the step length parameter λ as 1, and update the power weight vector

$${\mathbf{h}}^{k+1}={\mathbf{h}}^k-\lambda H_k^{-1}\nabla E({\mathbf{h}}^k).$$ (17)

Then we compute the power diagram D(P, h^k+1). If any Voronoi cell W_i(h^k+1) disappears, then the Hessian matrix H_k+1 will be degenerated. In this case, we shrink the step length parameter λ to be half, $\lambda \leftarrow \frac{1}{2}\lambda$. Then we recompute h^k+1 using the formula in Eqn. 17 and test again. We repeat this procedure, until all Voronoi cells in D(P, h^k+1) are non-empty. Algorithm 1 gives the implementation details.

4.2 Area-preserving Parameterization for Topological Disks

The OMT-Map algorithm can be generalized to compute the area-preserving mappings between surfaces. Suppose S is simply connected surface with a single boundary, namely a topological disk. S is with a Riemannian metric g. By scaling, the total area of (S, g) equals to π. Then according to the Riemann mapping theorem 5, there is a conformal mapping ϕ : (S, g) → ( , dzdz̄), such that g = e^2λ(z)dzdz̄. Then we find a OMT-Map τ : ( , dzdz̄) → ( , e^2λ dzdz̄), then the composition τ⁻¹ ∘ ϕ : (S, g) → ( , dzdz̄) gives the area-preserving mapping.

Algorithm 1.

Optimal Mass Transport Map (OMT-Map)

Input: A convex planar domain with measure (Ω, μ); a planar point set with measure (P, ν), νi > 0, ${\int }_{\mathrm{\Omega }}u(x)dx={\sum }_{i=1}^k{\nu }_i$; a threshold ε.

Output: The unique discrete OMT-Map f : (Ω, μ) → (P, ν).

Scale and translate P, such that P ⊂ Ω.

h ← (0, 0, ··· , 0).

Compute the power diagram D(h),

Compute the dual power Delaunay triangulation T(h),

Compute the cell areas w(h) = (w1(h), ··· , wk(h)).

repeat

Compute ∇E(h) using Eqn. 12.

Compute the Hessian matrix using Eqn. 13.

λ ← 1

h ← h − λH−1∇E(h).

Compute D(h), T(h) and w(h)

while ∃wi(h) == 0 do

h ← h + λH−1∇E(h).

λ ← 1/2λ

h ← h − λH−1∇E(h).

Compute D(h), T(h) and w(h).

end while

until ||∇E|| < ε.

return f : Ω → P, Wi(h) → pi, i = 1, 2, ··· , k.

The smooth surface (S, g) is approximated by a triangular mesh M, with vertex set V = {v₁, v₂, ··· , v_k}. The conformal mapping ϕ can be computed using discrete surface Ricci flow method [61]. Then each vertex v_i ∈ M is mapped to a planar point p_i = ϕ(v_i). The discrete measure ν_i is given by

$${\nu }_i=\frac{1}{3}\sum _{[v_i,v_j,v_k]\in M}\mathit{area}([v_i,v_j,v_k]),$$ (18)

where [v_i, v_j, v_k] is a face adjacent to v_i on the mesh. After normalization, the summation of the discrete measures, Σ_i ν_i, equals to π. Then the OMT-Map τ : ( , dxdy) → (P, ν) can be computed using Algorithm 1. The composition τ⁻¹ ∘ ϕ is a discrete area-preserving mapping, which maps each vertex v_i on the mesh to the centroid of the corresponding cell W_i on the disk, such that the area of W_i equals to ν_i. The implementation details can be found in Algorithm 2.

Algorithm 2.

Topological Disk Area-preserving Parameterization

Input: A triangular mesh M which is a topological disk; three vertices {v0, v1, v2} ⊂ ∂M on the boundary.

Output: The area-preserving parameterization f : M → , which maps {v0, v1, v2} to {1, i, −1} respectively.

Scale M such that its total area equals to π. Compute the conformal parameterization φ : M → , such that the images of {v0, v1, v2} are {1, i, −1} respectively. For each vertex vi ∈ M, define pi = φ(vi), νi to be 1/3 of the total area of the faces adjacent to vi. Set P = {pi}, ν = (νi). Compute the DOTM τ : → (P, μ) using Algorithm 1. Construct the mapping τ−1 ∘ ϕ : M → , which maps each vertex vi ∈ M to the centroid of Wi(h) ⊂ .

4.3 Area-preserving Parameterization for Topological Spheres

Suppose (S, g) is a closed genus zero metric surface, namely a topological sphere, with total area 4π. According to Theorem 6, given three points {p₁, p₂, p₃} ⊂ S, there is a unique conformal mapping ϕ : S → , where is the augmented complex plane {∞}, such that ϕ maps the three points to {0, 1, ∞} respectively, furthermore the original surface metric g = e^2λ dzdz̄.

Consider the unit sphere embedded in ℝ³, it has the induced Euclidean metric h. Let ψ : → be the stereo-graphic projection, then

$$\mathbf{h}=\frac{4\mathit{dzd}\overline{z}}{{(1+z\overline{z})}^2},\mu =\frac{4\mathit{dudv}}{{(1+u^2+v^2)}^2}$$

where μ is measure induced by h, z = u + iv.

Let $\tau :((\widehat{ℂ},\frac{4\mathit{dzd}\overline{z}}{{(1+z\overline{z})}^2})\to (\widehat{ℂ},e^{2\lambda }\mathit{dzd}\overline{z}))$ be the optimal mass transport map, then the composition ψ⁻¹ ∘ τ⁻¹ ∘ ϕ: (S, g) → ( , h) is an area-preserving mapping.

The surface is approximated by a triangle mesh M, the conformal mapping ϕ : S → is obtained by two steps. First, the mesh is conformally mapped to the unit sphere using spherical harmonic mapping method in [21]; secondly, the unit sphere is conformally mapped onto the augmented complex plane using the stereo-graphic projection. Then the discrete point set P consists p_i = ϕ(v_i). The discrete measure ν_i for each vertex is computed using the same formula as Eqn. 18.

The OMT-Map $\tau :(\widehat{ℂ},\frac{4\mathit{dzd}\overline{z}}{{(1+z\overline{z})}^2})\to (P,\nu )$ can be carried out using the same Algorithm 1. The sharp distinction is that the domain Ω here is infinite, the entire complex plane. Some cells are unbounded, but still with finite areas under the spherical measure μ.

In order to use Newton’s method for the optimization, for each cell, we need to compute the spherical area and the spherical edge lengths. Consider a finite polygon G first, suppose its edges are {s₁, ···, s_m} the exterior angles are {θ₁, ···, θ_m}. Because ψ is conformal, so the exterior angles are well preserved on the sphere, and each segment is mapped to curve segment, which is unnecessary to be a geodesic. According to Gauss-Bonnet theorem, ∫_GKdA + Σ_i ∫_si k_gds + Σ_j θ_j = 2π, where K is the Gaussian curvature, k_g is the geodesic curvature. Because Σ_j θ_j = 2π, K = +1, we obtain Area(G) = −Σ_i ∫_si k_gds. where Σ_i ∫_si k_gds can be easily and efficiently computed by spherical geometry. For an infinite cell G, there are two infinite edges, which intersect at the ∞ point. Suppose their intersection angle between two rays is θ, then the exterior angle at ∞ is π − θ. The other part of the computation is similar to the finite cell case. The implementation is quite similar to Algorithm 2 except that we need to compute an additional stereo-graphic projection (ψ) and consider infinite cell G when computing the cell areas w(h).

4.4 Conformal Wasserstein Distance

The OMT-Map algorithm can also be generalized to compute the Wasserstein distance between surfaces. Given two topological disk surfaces (M₁, g₁, p₁, q₁) ∈ , (M₂, g₂, p₂, q₂) ∈ with total area π, where is the normalized marked metric space defined in Eqn. 14. p₁ and p₂ are correspondent interior markers, and q₁ and q₂ are correspondent boundary markers. We first compute the conformal maps ϕ₁ : M₁ → and ϕ₂ : M₂ → , where and are the unit planar disks with Euclidean metric dx² + dy², such that ϕ(p₁) = ϕ(p₂) = (0, 0) and ϕ(q₁) = ϕ(q₂) = (1, 0). Then we construct a convex planar domain (Ω, μ) from , where μ is computed by Eqn. 15. And then we discretize into a planar point set with measure (P, ν), where ν is computed by Eqn. 18. Using (Ω, μ) and (P, ν) as inputs of Algorithm 1, we compute the Optimal Mass Transport map f : Ω → P, W_i(h) → p_i, where p_i ∈ P, i = 1, 2, ···, k. Therefore, the Wasserstein distance between M₁ and M₂ can be computed by

$$\mathit{Wasserstein}(\mu ,\nu )=\sum _{i=1}^n{\int }_{W_i}{(x-p_i)}^2\mu (x)dx$$ (19)

Algorithm 3 gives the implementation details.

Algorithm 3.

Computing Wasserstein Distance

Input: Two topological disk surfaces (M1, g1, p1, q1), (M2, g2, p2, q2). p1 and p2 are correspondent interior markers, and q1 and q2 are correspondent boundary markers.

Output: The Wasserstein distance between M1 and M2.

Scale and normalize M1 and M2 such that the total area of each surface is π. Compute the conformal maps ϕ1 : M1 → and ϕ2 : M2 → , where and are the unit planar disks with Euclidean metric dx2 + dy2, such that ϕ(p1) = ϕ(p2) = (0, 0) and ϕ(q1) = ϕ(q2) = (1, 0). Construct a convex planar domain (Ω, μ) from , where μ is computed by Eqn. 15. Discretize into a planar point set with measure (P, ν), where ν is computed by Eqn. 18. With (Ω, μ) and (P, ν) as inputs of Algorithm 1, we compute the Optimal Mass Transport map f : Ω → P, Wi(h) → pi, where pi ∈ P, i = 1, 2, ···, k. Wasserstein distance between M1 and M2 can be computed by Eqn. 19.

5 APPLICATIONS

This section will first briefly review the numerical efficiency of our algorithm. To demonstrate the flexibility of our algorithm, we will also show an example where the mapping areas can be specifically determined. Then we will mainly focus on two applications of OMT-Map in shape analysis. One is deformable surface matching between surfaces which differed by isometric transformations and the other is conformal Wasserstin shape space that provides a convenient way to cluster human facial expressions.

5.1 Planar and Spherical Area-Preserving Parameterization Results

We performed our area-preserving parameterization on both planar domains and spherical domains. Figure 2 illustrates the circle-packing texture mappings of CFP and APP of a human head model with unit square as the parameter domain. Figure 3 illustrates the comparison of CFP and APP of a kid sculpture model (Bimba) with unit sphere as the parameter domain. In these experiments, all the surfaces are represented as triangular meshes, either acquired by 3D scanners or geometric modeling. The quality of the parameterizations can be evaluated by angle and area distortions. The area distortion is computed as follows. Denote the parameterization as ϕ : M → , for each vertex v_i, the area distortion is defined as ${\epsilon }_i:=log\frac{{\sum }_{j,k}A([\varphi (v_i),\varphi (v_j),\varphi (v_k)])}{{\sum }_{j,k}A([v_i,v_j,v_k])}$, where A(·) represents the area of a triangle, [v_i, v_j, v_k] is the triangle formed by {v_i, v_j, v_k}. Similarly, the angle distortion at a corner angle is given by ${\eta }_{\mathit{ijk}}:=log\frac{\angle \varphi (v_i)\varphi (v_j)\varphi (v_k)}{\angle v_i{v}_j{v}_k}$. Then we plot the histograms of angle distortions and area distortions for both Optimal Mass Transport map and conformal map, which are shown in (e) to (h) in Figure 2, and (g) to (j) in Figure 3. The quantitative results demonstrate the superior accuracy of area-preserving property of Optimal Mass Transport map, while the conformal map method may cause numerical problems because of area distortions at exponential level. Moreover, the experimental results show the excellent performance for visualization by our method. Therefore, the proposed Optimal Mass Transport map is robust and feasible for various practical problems.

Running Time Analysis

Table 1 summarizes the geometric complexities of the triangular meshes and the corresponding running times of OMT-MAP algorithm on results shown in Figure 1, 2 and 3. We can see the running time depends on the resolution of the mesh, the geometric complexity of the mesh and the distribution of the point set P. Comparing the OMT-MAP running time between human head model in Figure 2 and Bimba sculpture in Figure 3, it shows spherical mapping takes more time than planar mapping. It is natural because spherical mapping needs an extra step to map between sphere and planar domain.

TABLE 1.

Geometric complexities and running times.

Model Name	Number of Faces	Running Time(s)	Figure
Armadillo	83.2k	74.5	1
Human head	94.3K	49.9	2
Bimba sculpture	92.2K	78.3	3

5.2 Importance-driven Surface Parameterization

Our OMT-MAP algorithm can also fully control the local areas of different regions of the surface. By adjusting the measure vector μ, our method can control the areas of different local regions, magnifying regions of interest and shrinking unimportant ones. This allows more parameter spaces to be allocated for regions with richer geometric or textural features, and improves the rendering quality and matching accuracy.

Figure 5 demonstrates this merit, where the buddha’s head ((a) and (d)) is magnified by different zooming factors, and the complementary part is shrunk accordingly (e)–(h). Basically, for vertices in the head region, we multiply their measures by the zooming factor, and then normalize the total area to be invariant. The importance-driven mapping results (e)–(h) show more details on the parameter domain than the angle-preserving result (b). Such flexibility controls are particularly useful for visualization or a focused region of interest shape analysis.

5.3 Deformable Surface Matching

In this section, we apply Optimal Mass Transport Map for deformable surface matching. The approach is illustrated by the following commutative diagram:

where S₁ and S₂ are two given surfaces with deformation and f : S₁ → S₂ is the desired matching. We use Optimal Mass Transport Map to compute ϕ_i : S_i → D_i which maps S_i onto the canonical domain D_i. D_i can be domains on plane R² or sphere R³. We call them optimal mass transport parameter domains of the surfaces. Then a planar or spherical mapping g : D₁ → D₂ is constructed for matching. The desired map is induced by $f={\varphi }_2^{-1}\circ g\circ {\varphi }_1:S_1\to S_2$. The OMP-map is intrinsic to the Riemannian metric, unique, and diffeomorphic and useful to compute ϕ₁, ϕ₂. This framework converts a 3D deformable surface matching problem to a 2D planar domain matching problem, or a 3D spherical matching problem, which are much easier than matching on the original surfaces.

Algorithm 4.

Deformable surface registration.

Input: Triangular meshes of surfaces with a simple topology, such as a simply connected domain with one boundary. A template surface as the target surface.

Output: Registered surfaces with a one-to-one correspondence from each surface to the target surface.

Manually or automatically locate some corresponding feature points on S1 and S2 for constraints. Compute the optimal mass transport map ϕi : Si → Di, i = 1, 2, where Di is a unit disk. Compute a constrained harmonic map g : D1 → D2, such that g align the corresponding feature points specified in the first step. The matching is given by $f={\varphi }_2^{-1}\circ g\circ {\varphi }_1:S_1\to S_2$.

Since our Optimal Mass Transport map converts the 3D surfaces to convex planar domain, if the map g is a diffeomorphism, the matching f is also a diffeomorphism. In our framework, that g is diffeomorphism is guaranteed by the following theorem:

Theorem 7 (Rado [42].)

Let (S, g) be a simply connected surface, D be a convex planar domain. f is a harmonic map such that the restriction of f on the boundary f : ∂S → ∂D is a homeomorphism, then f is a diffeomorphism.

5.3.1 Surface Matching by Euclidean Optimal Mass Transport Map

Here we use a simply connected surface with one boundary as an example to show how the OMT-Map algorithm can help compute surface matching. However, the algorithm is able to be generalized to topological sphere surfaces.

For such surfaces, the conformal parameter domain D can be chosen as the unit disk. Given two 3D surfaces S₁ and S₂ with deformations between them, f : S₁ → S₂ is the desired matching. Algorithm 4 show the algorithm details.

5.3.2 Experimental Results

Data Source

To validate the robustness and efficiency of our method, we tested surfaces with large isometric deformations. We chose 7 models that are isometric deformations to each other to study the accuracy and efficiency of our proposed method. The original Armadillo models, the same subject with different motions, are obtained from Aim@SHAPE repository [1] (shown in Figure 6). They form 21 different pairs of surfaces being matched to each other.

Fig. 6.

7 Armadillo models with isometric deformations, which form 21 matching pairs in our experiments.

Figure 7 shows an example surface matching result for Armadillo models with different motions. (a) and (b) are the two models with isometric deformations. We cut a hole at the waist of the models so that they are topologically equivalent to a disk. (c) and (f) are the optimal mass transport map results. Their mapping results are matched using harmonic maps with hard constraints (yellow stars). The colored lines connecting color-encoded circular dots on (a) and (b) show the registered correspondences by OMT map.

Fig. 7.

Surface registration results for Armadillo models with isometric deformations. Their mapping results are registered using harmonic maps with hard constraints (yellow stars). The colored lines connecting color-encoded circular dots on (a) and (b) show the registered correspondences by OMT map.

Performance Evaluation and Comparison

We compared our matching and registration method with conventional conformal mapping method based on Ricci flow theory [57], where the source surface is conformally flattened to a planar disk, then the registration is obtained by a constrained harmonic map between the disk and the target surface. We also compared our work with the Lipman and Funkhouser’s Möbius voting method [36]. The method first randomly samples a triplet on each of two surfaces, and uses Möbius transformations defined by the triplets to map the original surfaces into a complex domain, and finally produces voting points to predict correspondences between the surfaces. We used some performance metrics which were used in prior surface registration studies [40].

Diffeomorphism

One of the most important advantages of our registration method is that, in practice it always generates the mapping between surfaces to be diffeomorphic, even for long tube surfaces that may have numerical problems by conformal mapping, such as the fingers of the Armadillo model show in Figure 1. For each registration, we compute the Jacobian determinant and measure the area of flipped regions. For conformal mapping method, the average ratio from flipped area to the total area of 21 Armadillo pairs is 25.8%. The average flipped area ratio for the Möbius voting method is 4.5%. In contrast, the flipped area ratios for all registrations obtained by our method are exactly zero.

Curvature Difference Maps

Our method to evaluate registration accuracy is to compare the alignment of curvature maps between the registered models [40]. We calculated curvature maps using an approximation of mean curvature, which is the convexity measure. We quantified the effects of registration on curvature by computing the difference of curvature maps from the registered surfaces. For each vertex on the target surface with curvature c₁, we find its correspondent point on the source surface with curvature c₂. Then compute the curvature difference as $\mathit{max}(\frac{\mid c_1\mid }{\mid c_2\mid },\frac{\mid c_2\mid }{\mid c_1\mid })$. In Figure 7(e) shows the average histogram of the curvature difference map of conformal mapping, Möbius voting method and our method computed from all 21 pairs of surface matchings, respectively. The quantitative results indicate that conformal mapping and the Möbius voting method produce less consistent and less accurate correspondences than our method.

Local Area Distortion

Similarly, we evaluated the local area distortion induced by the registrations [40]. For each vertex v on the target surface with its correspondent point p on the source surface, we compute its Jacobian determinant J(v) [57], and represent the local area distortion at v as max(J(v), J⁻¹(v)). J can be approximated by the ratio between the measure μ(v)/μ(p), where $\mu (v_i):=\frac{1}{3}{\sum }_{jk}\mathit{Area}([v_i,v_j,v_k])$ and [v_i, v_j, v_k] is a triangle face adjacent to v_i. Note that if the registration is not diffeomorphic, the local area distortion may go to ∞. Therefore, we add a threshold to truncate large distortions. Figure 7(h) shows the average area distortion histogram of conformal method, Möbius voting method and our method. It is obvious that our registration method produces much less area distortions than the other two methods. From these quantitative empirical evaluations, we observe that our method may outperform previous methods [36], [57] by registration accuracy. Moreover, our method has the advantages that it can handle large area distortion, and guarantees diffeomorphic mappings.

5.4 Wasserstein Distance for Shape Analysis

Wasserstein distance is a Riemannian metric of the Wasserstein space. The Wasserstein distance between two surfaces is a shape metric which can be used for quantifying shape differences. The computational algorithm can be found in Alg. 3. Figure 8 shows the visualization results of Wasserstein distance. (a) and (c) are two face surfaces of different facial expressions. (b) and (d) are the conformal mapping results for (a) and (c), respectively. (e) shows the optimal mapping from (a) to (c), which induces Wasserstein distance by Eqn. 19. For better visualization of (e), we put straight grids on (c), and draw the deformed grids on (e). From the grids deformation, we can clearly see how the surface around the mouth and nose deforms when the facial expression changes from calm to smile.

Fig. 8.

The computation of Wasserstein distance

As noted earlier that Wasserstein distance can be used for quantifying shape differences, we applied Wasserstein distance for facial expressions clustering. Our experimental dataset contains 10 people, each of which has 3 different facial expressions–“sad”, “happy” and “surprise” shown in Figure 9 row 1, 2, 3, respectively. The 3D face surfaces are from Binghamton University 3D Facial Expression Database [58]. For each pair of surfaces in the dataset, we compute the Wasserstein distance. Then we use classical multidimensional scaling (MDS) [53] to embed all the 30 face surfaces in ℝ² based on the Wasserstein distance between each pair of faces. Figure 10 illustrates the visualization results of the MDS embedding. For all the surfaces with “sad” expressions, we mark them as ‘+’ in blue color, and “happy” as ‘x’ in red color, and “surprise” as ‘o’ in green color. We can see that almost all faces with the same expressions are clustered together, and faces with different expressions are divided into different clusters.

Fig. 9.

Face surfaces for expression clustering. The first row is “sad”, the second row is “happy” and the third row is “surprise”.

Fig. 10.

Multidimensional scaling embedding of the Wasserstein distance between each pair of face surfaces in the dataset.

The facial expression clusters verify the idea that physical expressions of emotions can be systematically categorized and support the adoption facial action coding system (FACS) [17] in computer vision and animation research. The experimental results also demonstrated the feasibility and potential of comparing and quantifying shape differences by conformal Wasserstein distance. Whether or not Wasserstein distance provides better accuracy in facial expression clustering than those afforded by other shape distance requires careful validation for each application. More importantly, we anticipate that our approach may serve as novel shape distance for shape analysis. In future we plan to exploit the potential of proposed shape distance for more applications such as face recognition.

6 Conclusion and Future Work

In this paper, we introduce a novel optimal mass transport map algorithm and explore its applications in 3D shape matching and comparison. Based on a newly discovered variational principle, the algorithm is general and theoretically sound. It offers better computational efficiency than prior work while enjoys the flexibility to control mapping region size. It produces diffeomorphic area preserving parameterization results and constructs conformal Wasserstein shape space. Two example applications, deformable surface matching and conformal Wasserstein distance, are studied in our paper. The experimental results show the promise for our algorithm to be a stable and effective solution for 3D shape analysis research.

In the future, we will research and study the deformation/geodesic path between shapes in the conformal Wasserstein shape space. With the deformation sequence, we can construct statistical modeling, such as mean shape, and principal directions of variation by principal component analysis (PCA). Furthermore, we will apply the principle components for broader shape classification and recognition applications.

Fig. 4.

Discrete optimal mass transport map with Brenier’s approach.