Shape analysis via inconsistent surface registration

Abstract

In this work, we develop a framework for shape analysis using inconsistent surface mapping. Traditional landmark-based geometric morphometr- ics methods suffer from the limited degrees of freedom, while most of the more advanced non-rigid surface mapping methods rely on a strong assumption of the global consistency of two surfaces. From a practical point of view, given two anatomical surfaces with prominent feature landmarks, it is more desirable to have a method that automatically detects the most relevant parts of the two surfaces and finds the optimal landmark-matching alignment between these parts, without assuming any global 1–1 correspondence between the two surfaces. Our method is capable of solving this problem using inconsistent surface registration based on quasi-conformal theory. It further enables us to quantify the dissimilarity of two shapes using quasi-conformal distortion and differences in mean and Gaussian curvatures, thereby providing a natural way for shape classification. Experiments on Platyrrhine molars demonstrate the effectiveness of our method and shed light on the interplay between function and shape in nature.

1. Introduction

The use of mathematical mappings for quantifying shape variation began a century ago with the seminal work of D’Arcy Thompson [1], in which a theory of transformations was proposed for studying planar projections of biological shapes, such as fish and human skull, with the aid of a deformed underlying grid. Shape registration, the process of aligning two shapes, is commonly done by computing a mapping that deforms one of them (called the source shape) to match the other one (called the target shape). Establishing a registration between the two shapes makes it easy to compare them and highlight their geometric difference. As the shapes to be compared usually contain certain similar features, it is common to extract a number of feature points (called the landmarks) from each of them and enforce their correspondence under the mapping, thereby reducing geometric mismatch in the resulting shape registration. Given two sets of corresponding landmarks representing two shapes, traditional geometric morphometrics methods such as the Procrustes superimposition method [2] use rigid (translational and rotational), isotropic/anisotropic scaling and shear transformations to align them for quantifying their shape difference. Because of the limited degrees of freedom of these transformations, more advanced non-rigid mappings such as the thin plate splines method [3], surface matching via currents [4,5], large deformation diffeomorphic metric mapping (LDDMM) [6–9] and stationary velocity fields (SVF) [10–13] have been considered. In general, any mapping between two surfaces will unavoidably induce distortion in angle or area (or both). There has been a vast number of works aiming to achieve angle-preserving (conformal) [14–24] and area-preserving (authalic) [25–31] mappings (see [32–34] for detailed surveys on the subject). These methods, however, do not allow for an exact matching of the corresponding landmarks on two surfaces and hence hinder the accurate quantification of shape variation.

In recent years, quasi-conformal theory has emerged as a useful tool for geometry processing and medical imaging [35–42]. In particular, landmark-matching quasi-conformal mapping methods have been developed for image and surface registration [43–45] and applied for the analysis of the cerebral cortex [46], hippocampus [47], colon [48], vertebral bone [49], vestibular system [50], human face [51], insect wing [52,53], teeth [54], etc. However, in the above-mentioned works, the shapes are always assumed to be with a global 1–1 correspondence. In other words, every point on one shape is assumed to correspond uniquely to a point on another shape. For instance, in [54], the occlusal surfaces of the teeth were manually delineated by dental experts, and the quasi-conformal mappings between the teeth assumed a 1–1 correspondence between the segmented tooth boundaries. While the assumption of the global 1–1 correspondence ensures that the entire shapes are considered, it may not always yield the best registration as there may be errors in the steps of data acquisition, surface reconstruction, or manual delineation for certain datasets, such that some parts of one preprocessed surface in fact do not correspond to any position of another preprocessed surface. Having a landmark-matching mapping method which is capable of finding an optimal registration between two surfaces without any assumption of a complete surface correspondence would provide a useful alternative approach for shape analysis, which can be carried out based on the optimal registration results without being affected by the inconsistent parts.

In this work, we propose a new framework for shape analysis based on a recent algorithm for inconsistent surface registration [55]. Unlike the prior quasi-conformal mapping methods, the inconsistent surface registration method does not assume any global correspondence between two shapes, making it particularly suitable for tackling general shape analysis problems. Given two inconsistent surfaces with prescribed landmark constraints, we first apply the method to find an optimal landmark-matching quasi-conformal mapping between them. This gives a 1–1 correspondence between the common regions of the two surfaces, which are automatically determined by the method. The remaining inconsistent regions of the two surfaces are not aligned, and hence, the quantification of the shape variation of two surfaces can be focused on the common regions without being affected by the inconsistency of the overall shapes. In particular, we evaluate the quasi-conformal distortion, mean curvature difference, and Gaussian curvature difference between the common regions of two shapes and use them to construct a dissimilarity measure between the two shapes. Based on the dissimilarity measure between every pair of shapes, we can then perform a cluster analysis to study the shape variation. We demonstrate the effectiveness of our proposed framework by using it to analyse a set of mammalian teeth [56,57].

The rest of the paper is organized as follows. In §2, we review some mathematical concepts relevant to our method. In §3, we describe our proposed framework for shape analysis via inconsistent surface registration. In §4, we describe the tooth dataset used in our work and present our results. We conclude our work and discuss future directions in §5.

2. Mathematical background

(a). Quasi-conformal theory

Here, we review the basics of quasi-conformal theory and refer the readers to [58] for more details. Conformal maps preserve angles and hence the local geometry, and quasi-conformal maps are a generalization of conformal maps. Intuitively, conformal maps send infinitesimal circles to infinitesimal circles, while quasi-conformal maps send infinitesimal circles to infinitesimal ellipses with bounded eccentricity. Mathematically, a map $f:\mathbb{C}\to \mathbb{C}$ is said to be a quasi-conformal map if it satisfies the Beltrami equation

$$\frac{\mathrm{\partial }f}{\mathrm{\partial }\overline{z}}={\mu }_f(z)\frac{\mathrm{\partial }f}{\mathrm{\partial }z},$$ 2.1

for some complex-valued function μ_f(z) with $||{\mu }_f(z)|{|}_{\mathrm{\infty }}<1$. μ_f is called the Beltrami coefficient of f, which captures the quasi-conformal distortion of f (figure 1). In particular, the maximum magnification and the maximum shrinkage of the infinitesimal ellipses are given by |f_z|(1 + |μ_f|) and |f_z|(1 − |μ_f|) respectively, and hence, the aspect ratio of the ellipses, also known as the dilatation, is given by $\frac{1+|{\mu }_f|}{1-|{\mu }_f|}$. It is noteworthy that if μ_f ≡ 0 then the right-hand side of equation (2.1) becomes 0, which implies that f is conformal.

Figure 1.

Infinitesimal circles are mapped to infinitesimal ellipses with bounded eccentricity under any quasi-conformal map f. The maximum magnification, the maximum shrinkage, and the orientation change of the infinitesimal ellipses are |f_z|(1 + |μ_f|), |f_z|(1 − |μ_f|) and arg(μ_f)/2, respectively. (Online version in colour.)

Quasi-conformal maps can also be defined between two Riemann surfaces S₁, S₂ in ${\mathbb{R}}^3$ with the aid of local charts. In our work, S₁ and S₂ are considered to the simply connected open surfaces. Therefore, if a mapping f:S₁ → S₂ can be decomposed into f = ψ⁻¹° h°ϕ, where $\varphi :S_1\to \mathbb{C}$ and $\psi :S_2\to \mathbb{C}$ are two conformal maps and h is a quasi-conformal map on the complex plane, then the quasi-conformal distortion of f can be represented by the quasi-conformal distortion of h.

(b). Differential geometry of surfaces

In our work, curvatures are used for quantifying shape difference. Here, we briefly review the theory of surface curvatures and refer the readers to [59] for details. Let S be a smooth surface in ${\mathbb{R}}^3$, and let p be a point in S. Denote the surface normal at p by N(p). Any normal plane containing N(p) cuts the surface S in a plane curve. One can then evaluate the curvature of the plane curve at p. Considering all possible normal planes at p, the principal curvatures κ₁(p) and κ₂(p) are the maximum and minimum values of the curvature of the resulting plane curve.

Using the principal curvatures, one can obtain the mean curvature H at p, which is given by the average of the principal curvatures:

$$H(p)=\frac{1}{2}({\kappa }_1(p)+{\kappa }_2(p)).$$ 2.2

Note that the mean curvature is an extrinsic measure of curvature which depends on the embedding of the surface in the ambient space. One can also obtain the Gaussian curvature K at p, which is given by the product of the principal curvatures:

$$K(p)={\kappa }_1(p){\kappa }_2(p).$$ 2.3

Unlike the mean curvature, the Gaussian curvature is an intrinsic measure of curvature which is independent of the embedding of the surface in the ambient space.

Figure 2 shows the mean and Gaussian curvatures of five tooth surfaces. Altogether, the mean and Gaussian curvatures capture both the extrinsic and intrinsic geometry of any smooth surface in ${\mathbb{R}}^3$ and hence are useful for shape analysis.

Figure 2.

Five example tooth surfaces in the mammalian molar dataset [56,57], colour-coded by the mean curvature H and the Gaussian curvature K. Each column corresponds to one tooth surface. (Online version in colour.)

3. Proposed shape analysis framework

In this section, we describe our proposed shape analysis framework via inconsistent surface registration. The framework is outlined in algorithm 1, and the details of each step are given in the following sections.

(a). Inconsistent surface registration

(i). Overview

Let S₁, S₂ be two simply connected open surfaces in ${\mathbb{R}}^3$, with corresponding landmarks ${\{p_i\}}_{i=1}^k$ and ${\{q_i\}}_{i=1}^k$. We aim to find two optimal common regions Ω₁ ⊂ S₁ and Ω₂ ⊂ S₂ and an optimal mapping f:Ω₁ → Ω₂, such that f is a bijective landmark-matching mapping satisfying

$$f(p_i)=q_i,i=1,2,\dots ,k.$$ 3.1

In other words, Ω₁ and Ω₂ are the common regions on the two surfaces with a 1–1 correspondence established by f. They contain the landmarks ${\{p_i\}}_{i=1}^k$ and ${\{q_i\}}_{i=1}^k$ and capture the key shape variation between the two surfaces. The remaining regions $S_1\setminus {\mathit{\Omega }}_1$ and $S_2\setminus {\mathit{\Omega }}_2$ are the inconsistent parts of the two surfaces, which will not be taken into account for quantifying the shape variation of the two surfaces. Furthermore, f minimizes the geometric distortion between Ω₁ and Ω₂ in terms of the norm of the Beltrami coefficient |μ|, which provides us with a measure of quasi-conformal dissimilarity of the two common regions Ω₁ and Ω₂. The procedure of the registration is illustrated in figure 3.

Figure 3.

The procedure of the inconsistent surface registration. Given two surfaces S₁, S₂ with landmark constraints, we first conformally parametrize the two surfaces onto the plane. Then, we find an optimal landmark-matching quasi-conformal map h:X → X′ between two optimal subdomains (the common regions) on the two flattened shapes. Finally, we obtain the desired inconsistent registration f:Ω₁ → Ω₂ where Ω₁ ⊂ S₁ and Ω₂ ⊂ S₂ are the two optimal subdomains that correspond to X and X′, respectively. (Online version in colour.)

(ii). Conformal parametrization

Following the approach in [55], we start with flattening the two surfaces S₁ and S₂ using conformal parametrization. This effectively simplifies the registration problem as we then only need to handle a planar registration problem. In particular, we compute two free-boundary conformal parametrizations $\varphi :S_1\to \mathbb{C}$ and $\psi :S_2\to \mathbb{C}$.

Two major advantages of using free-boundary conformal parametrizations are as follows. First, when compared to parametrizations onto a prescribed shape such as the unit disc or a rectangle, free-boundary parametrizations in general have a lower conformal distortion due to the larger flexibility in the flattened shape. Consequently, the discrepancy between the planar registration result and the surface registration result can be reduced. Second, flattening the surfaces onto a standardized shape assumes that the surfaces have a global 1–1 correspondence, which is established via the standardized planar shape. However, in our problem, we do not assume any global 1–1 correspondence between the input surfaces. It is therefore more natural to use free-boundary parametrizations.

Among the existing free-boundary conformal parametrization methods, the least-square conformal map (LSCM) method [15] is used in our framework. LSCM produces a free-boundary conformal flattening map by solving the following energy minimization problem:

$$\underset{u,v}{min}\int \frac{1}{2}||\mathrm{\nabla }u|{|}^2+\frac{1}{2}||\mathrm{\nabla }v|{|}^2-\mathrm{\nabla }u\cdot \mathrm{\nabla }{v}^{\perp },$$ 3.2

where u and v are the coordinate functions. More details of the problem formulation and the computational procedure can be found in [15].

(iii). Optimal planar registration

After obtaining the conformally flattened domains ϕ(S₁) and ψ(S₂), the next step is to find an optimal landmark-matching quasi-conformal map $h:\varphi (S_1)\to \mathbb{C}$ with

$$h(\varphi (p_i))=\psi (q_i),i=1,2,\dots ,k.$$ 3.3

There are infinitely many quasi-conformal mappings that satisfy the landmark-matching constraints. Among them, we search for two optimal subdomains X ⊂ ϕ(S₁) and X′ ⊂ ψ(S₂) and a 1–1 mapping h:X → X′ that minimize the following energy (see [55] for more details):

$$E(X,X^{\prime },h)={\int }_{h(X)\cup X^{\prime }}{(I_1^h-I_2)}^2+{\int }_X(\lambda |{\mu }_h{|}^2+|\mathrm{\nabla }{\mu }_h{|}^2),$$ 3.4

subject to the landmark constraints $h(\varphi (p_i))=\psi (q_i),\hspace{0.17em}i=1,2,\dots ,k$. Here, $I_1:\varphi (S_1)\to \mathbb{R}$ and $I_2:\psi (S_2)\to \mathbb{R}$ are some matching intensities to be prescribed, and $I_1^h:=I_1\circ h^{-1}$ is the deformed image of I₁ under h. μ_h is the Beltrami coefficient of h. The first integral aims at establishing a meaningful pointwise correspondence by minimizing the shape mismatch error between the two subdomains X and X′ in terms of the prescribed intensities, and the second integral is a regularization term that minimizes the local geometric distortion (measured by |μ_h|) and promotes the smoothness of the mapping (measured by $|\mathrm{\nabla }{\mu }_h|$). In practice, we set the matching intensities I₁, I₂ based on the Gaussian curvature of the two surfaces, with an appropriate normalization such that 0 ≤ I₁, I₂ ≤ 1:

$$I_1=\frac{K_1-min{K}_1}{max{K}_1-min{K}_1},I_2=\frac{K_2-min{K}_2}{max{K}_2-min{K}_2}.$$ 3.5

A splitting scheme was used for solving the above optimization problem iteratively in [55], which involves alternately solving the landmark-matching quasi-conformal mapping problem and the intensity-based registration problem. In this work, we modify the splitting scheme by first solving the intensity-based registration problem using the Demons method [60,61], and then, the landmark-matching quasi-conformal mapping problem using the Linear Beltrami Solver [37] at each iteration. This ensures that the intensity mismatch error and quasi-conformal distortion decrease throughout the iterations. Moreover, as our modified splitting scheme always ends with a landmark-matching step, the prescribed landmarks are exactly matched, thereby producing an optimal landmark-matching inconsistent planar registration. In practice, we set the maximum number of iterations to be n_iter = 20. The algorithm is outlined in algorithm 2.

(iv). Final inconsistent shape registration between surfaces

After finding the two optimal subdomains X and X′ and the optimal landmark-matching quasi-conformal map h:X → X′, we can obtain two corresponding common regions Ω₁ ⊂ S₁ and Ω₂ ⊂ S₂ by Ω₁ = ϕ⁻¹(X) and Ω₂ = ψ⁻¹(X′). It is easy to see that all corresponding landmarks in S₁ and S₂ are guaranteed to be in Ω₁ and Ω₂, and they are exactly matched under h.

Finally, the landmark-matching inconsistent shape registration mapping f:Ω₁ → Ω₂ is given by

$$f={\psi }^{-1}\circ h\circ \varphi .$$ 3.6

Since ϕ and ψ are conformal, the quasi-conformal distortion of f is equal to that of h.

(b). Quantifying the dissimilarity between inconsistent shapes

For any simply connected open surfaces S₁, S₂, using the method introduced in§3a, we obtain a landmark-matching quasi-conformal map f:Ω₁ ⊂ S₁ → Ω₂ ⊂ S₂. We can then quantify the difference between S₁ and S₂ using the correspondence between Ω₁ and Ω₂. More specifically, three important shape indices can be considered:

• (i)
  The quasi-conformal distortion μ_f of the mapping f, which captures the effort needed to deform Ω₁ to match Ω₂ in terms of the eccentricity of the infinitesimal circle packing. Note that by quasi-conformal theory, for any quasi-conformal map f, we always have

  $$0\le |{\mu }_f|\le 1.$$ 3.7
• (ii)
  The normalized mean curvature difference $\frac{1}{2}|{\tilde{H}}_1(f)-{\tilde{H}}_2|$ under the optimal registration f, where ${\tilde{H}}_1$, ${\tilde{H}}_2$ are the normalized mean curvature difference of S₁ and S₂ with $0\le |{\tilde{H}}_1|$, $|{\tilde{H}}_2|\le 1$. Note that with the normalization, we always have $|{\tilde{H}}_1(f)-{\tilde{H}}_2|\le 1-(-1)=2$ and hence

  $$0\le \frac{1}{2}|{\tilde{H}}_1(f)-{\tilde{H}}_2|\le 1.$$ 3.8
• (iii)
  The normalized Gaussian curvature difference $\frac{1}{2}|{\tilde{K}}_1(f)-{\tilde{K}}_2|$ optimal registration f, where ${\tilde{K}}_1,{\tilde{K}}_2$ are the normalized Gaussian curvature difference of S₁ and S₂ with $0\le |{\tilde{K}}_1|,|{\tilde{K}}_2|\le 1$. Again, with the normalization, we always have $|{\tilde{K}}_1(f)-{\tilde{K}}_2|\le 1-(-1)=2$ and hence

  $$0\le \frac{1}{2}|{\tilde{K}}_1(f)-{\tilde{K}}_2|\le 1.$$ 3.9

In the global Teichmüller mapping framework [54], the Teichmüller distance, mean curvature difference and Gaussian curvature difference are combined for quantifying the shape difference between the two given surfaces. Note that the global Teichmüller maps give a constant Teichmüller distance defined over the entire domain. While the Teichmüller distance is capable of representing the quasi-conformal distortion of the overall shapes as a scalar quantity, its value may be significantly affected by the correspondence between the inconsistent parts of the two given surfaces under the global Teichmüller mapping. By contrast, with the aid of the inconsistent shape registration, we can focus on the quasi-conformal distortion and the curvature differences between the optimal common regions of the two surfaces. Now, we define the combined shape index δ(S₁, S₂) as follows:

$$\delta (S_1,S_2)=\frac{1}{A({\mathit{\Omega }}_1)}{\int }_{{\mathit{\Omega }}_1}(\alpha |{\mu }_f|+\frac{\beta }{2}|{\tilde{H}}_1(f)-{\tilde{H}}_2|+\frac{\gamma }{2}|{\tilde{K}}_1(f)-{\tilde{K}}_2|),$$ 3.10

where A(Ω₁) is the area of the optimal region Ω₁, and α, β, γ ≥ 0 are the weighting factors for the three quantities with α + β + γ = 1. It follows that

$$0\le \delta (S_1,S_2)\le \frac{1}{A({\mathit{\Omega }}_1)}{\int }_{{\mathit{\Omega }}_1}(\alpha +\beta +\gamma )\le \frac{1}{A({\mathit{\Omega }}_1)}{\int }_{{\mathit{\Omega }}_1}1=1.$$ 3.11

Alternatively, one can obtain a landmark-matching quasi-conformal map g:Ω₂ ⊂ S₂ → Ω₁ ⊂ S₁. This gives us

$$\delta (S_2,S_1)=\frac{1}{A({\mathit{\Omega }}_2)}{\int }_{{\mathit{\Omega }}_2}(\alpha |{\mu }_g|+\frac{\beta }{2}|{\tilde{H}}_2(g)-{\tilde{H}}_1|+\frac{\gamma }{2}|{\tilde{K}}_2(g)-{\tilde{K}}_1|),$$ 3.12

and similarly, we have 0 ≤ δ(S₂, S₁) ≤ 1. Based on the above quantities, we define the dissimilarity d between two inconsistent surfaces S₁ and S₂ by

$$d(S_1,S_2)=min\{\delta (S_1,S_2),\delta (S_2,S_1)\}\in [0,1].$$ 3.13

Note that a dissimilarity matrix for n shapes is an n × n matrix satisfying the following two properties:

• (i)The matrix is symmetric.
• (ii)All diagonal entries of the matrix should be zero, as there is no dissimilarity between a shape and itself.

Now, after computing the pairwise mappings between all n surfaces ${\{S_i\}}_{i=1}^n$, we define an n × n matrix D by D(i, j) = d(S_i, S_j). By the definition of d, we have

$$D(i,j)=d(S_i,S_j)=min\{\delta (S_i,S_j),\delta (S_j,S_i)\}=d(S_j,S_i)=M(j,i)$$ 3.14

for all 1 ≤ i, j ≤ n. Also, since the identity map I:S_i → S_i is conformal, we have

$$D(i,i)=d(S_i,S_i)=\frac{1}{A(\mathit{\Omega })}{\int }_{\mathit{\Omega }}(\alpha \cdot 0+\frac{\beta }{2}|{\tilde{H}}_i-{\tilde{H}}_i|+\frac{\gamma }{2}|{\tilde{K}}_i-{\tilde{K}}_i|)=0,$$ 3.15

for any α, β, γ. Therefore, D satisfies both conditions (i) and (ii) and is a valid dissimilarity matrix.

(c). Cluster analysis based on geometric difference

After obtaining the dissimilarity matrix D that captures the shape difference of the n surfaces ${\{S_i\}}_{i=1}^n$ in terms of the quasi-conformal distortion, the normalized mean curvature difference and the normalized Gaussian curvature difference, we can then cluster the surfaces into different groups according to their pairwise geometric dissimilarity with the aid of clustering algorithms.

Cluster analysis is a well-studied topic in statistics, and many useful algorithms have been proposed. In this work, we consider two widely used clustering methods, namely the hierarchical clustering method and the k-means clustering method. Below, we briefly describe the two methods and refer the readers to [62] for more details.

The hierarchical clustering method starts by linking pairs of surfaces which are close together into binary clusters, i.e. clusters with exactly two objects, based on the dissimilarity measure encoded in D. It then continues to link these binary clusters with each other to form bigger clusters. Ultimately, all n objects are linked together in a binary hierarchical cluster tree. Then, one can cluster all n surfaces into a certain number of clusters by cutting off the hierarchy at different levels in the binary tree.

The k-means clustering method aims to cluster n observations into k clusters, such that each observation belongs to the cluster with the nearest mean. Each observation is required to be a p-dimensional coordinate vector. Starting from our dissimilarity matrix D, we can first apply the multidimensional scaling (MDS) method to construct an optimal projection of the n shapes onto a p-dimensional Euclidean space, so that each shape S_i is represented by a data point ${\mathbf{\text{x}}}_i\in {\mathbb{R}}^p$. In particular, MDS aims to minimize the difference between $||{\mathbf{\text{x}}}_i-{\mathbf{\text{x}}}_j||$ and D(i, j), so that the information of the pairwise distances given by D is optimally translated into a set of coordinates. Now, given the n points ${\{{\mathbf{\text{x}}}_i\}}_{i=1}^n$ and a prescribed positive integer k, the k-means clustering method looks for an optimal grouping of the points into k clusters C₁, C₂, …, C_k, such that the sum of the squared distance between each data point and the centre of its corresponding cluster, i.e. $\sum _{j=1}^k\sum _{\mathbf{\text{x}}\in C_j}||\mathbf{\text{x}}-{\mathit{\mu }}_j|{|}^2$, is minimized. Here, μ_j is the mean of the points in C_j.

4. Experiment

(a). Dataset

To demonstrate the effectiveness of our proposed shape analysis framework, we study the Platyrrhine (New World monkey) molar dataset [56,57], which consists of 50 second mandibular molar specimens of primates publicly available at the biological data archive MorphoSource [63]. The specimens were obtained from the American Museum of Natural History, the Smithsonian Institution National Museum of Natural History, the Harvard University Museum of Comparative Zoology, and the Stony Brook University Museum of Anatomy. The 50 molars were evenly collected from five genera, namely Alouatta (also known as Howler monkeys), Ateles (also known as Spider monkeys), Brachyteles (also known as woolly spider monkeys), Callicebus (also known as titis) and Saimiri (also known as squirrel monkeys). Table 1 records the genus and species of each specimen. The specimens were discretized in the form of triangle meshes, each containing about 5000 vertices and 10 000 faces (see [56,57] for more details of the data acquisition and preprocessing procedure). We further preprocessed the meshes to ensure that they are all simply connected, open and without any non-manifold vertices or edges.

Table 1.

The list of the 50 tooth specimens adapted from the Platyrrhine molar dataset [56,57].

genus	species	specimen ID
Alouatta	Alouatta seniculus	32
	Alouatta seniculus	33
	Alouatta palliata	34
	Alouatta seniculus	36
	Alouatta seniculus	37
	Alouatta seniculus	38
	Alouatta seniculus	39
	Alouatta seniculus	40
	Alouatta palliata	41
	Alouatta palliata	42
Ateles	Ateles belzebuth	3
	Ateles belzebuth	4
	Ateles belzebuth	9
	Ateles geoffroyi	20
	Ateles paniscus	21
	Ateles geoffroyi	22
	Ateles belzebuth	35
	Ateles geoffroyi	43
	Ateles belzebuth	46
	Ateles belzebuth	47
Brachyteles	Brachyteles arachnoides	19
	Brachyteles arachnoides	23
	Brachyteles arachnoides	24
	Brachyteles arachnoides	25
	Brachyteles arachnoides	26
	Brachyteles arachnoides	27
	Brachyteles arachnoides	28
	Brachyteles arachnoides	29
	Brachyteles arachnoides	30
	Brachyteles arachnoides	31
Callicebus	Callicebus donacophilus	2
	Callicebus torquatus	8
	Callicebus torquatus	10
	Callicebus torquatus	11
	Callicebus torquatus	12
	Callicebus torquatus	13
	Callicebus moloch	14
	Callicebus moloch	15
	Callicebus donacophilus	17
	Callicebus donacophilus	18
	Saimiri boliviensis	5
	Saimiri boliviensis	6
	Saimiri boliviensis	7
	Saimiri boliviensis	16
	Saimiri boliviensis	44
	Saimiri boliviensis	45
	Saimiri sciureus	48
	Saimiri sciureus	49
	Saimiri sciureus	50

Note that for each molar in the dataset, the most prominent features are the four cusps. To establish a correspondence between the common parts of two molars, such features should be consistently aligned. Therefore, we extracted the four vertices at the apex of the four cusps as landmarks. The landmark-matching inconsistent shape registration method described in the above section ensures that the four landmarks are exactly matched under the mappings, so that a systematic comparison between the molars can be subsequently performed.

Figure 2 shows five example molar surfaces in the dataset colour-coded by the mean and Gaussian curvatures. Figure 4 shows all 50 molar surfaces in the dataset, each with the four prescribed landmarks labelled.

Figure 4.

The 50 tooth specimens in the Platyrrhine molar dataset [56,57], each with four prescribed landmarks at the four cusps. The specimens are plotted according to the specimen IDs in table 1, with the first row showing specimens 1–6, and the second row showing specimens 7–12, etc. (Online version in colour.)

(b). Implementation

The proposed framework and the relevant algorithms were implemented in Matlab. The linear systems were solved using the Matlab backslash operator (\). The computations were performed on a Windows PC with a quad core processor and 16 GB RAM. To accelerate the computations of the 50 × 50 = 2500 pairwise mappings between all 50 tooth meshes, we used the parfor function in the Parallel Computing Toolbox in Matlab, with all four CPU cores of the PC utilized. The computation of each registration mapping took around 40 s, and the computations of all 2500 pairwise mappings took around 8 h in total.

The cluster analysis was also done using Matlab. For hierarchical clustering, we used the Matlab built-in functions linkage and cluster. For multidimensional scaling, we used the Matlab built-in function mdscale. For k-means clustering, we used the Matlab built-in function kmeans with the parameter Replicates being 100.

(c). Pairwise mapping result

We first consider computing pairwise mappings between the molars using various methods. As shown in figure 5, for rigid transformation methods such as the Procrustes superimposition method [2] and the iterative closest point (ICP) method [64], the two sets of corresponding landmarks cannot be exactly matched, and the remaining parts of the two molars are also misaligned. As for non-rigid transformation methods, we observe that while the thin plate splines (TPS) method [3] is able to match the two sets of landmarks, the induced deformation of the source surface is large and hence the overall shapes are largely mismatched. Also, while the non-rigid ICP method [65] is capable of deforming the source surface to match the overall geometry of the target surface, the landmarks are not exactly matched. In summary, all of the above methods either suffer from being unable to exactly match the landmarks or to match the surface geometry. In contrast to the above methods, our inconsistent surface registration approach is capable of exactly matching all landmark pairs and appropriately deforming the source surface to match the target surface. This demonstrates the effectiveness of our approach for computing pairwise mappings between surfaces.

Figure 5.

Computing a mapping between two molars. Top row (left to right): the source surface with four landmarks, the target surface with four landmarks, the mappings produced by the Procrustes superimposition method [2] and the iterative closest point (ICP) method [64]. Bottom row (left to right): the mappings produced by the thin plate splines (TPS) method [3], the non-rigid ICP method [65], and our inconsistent surface registration method. (Online version in colour.)

One may wonder whether the issue of mismatches in the mapping results produced by the above-mentioned prior methods will be resolved if only the common regions of the two surfaces are considered. Here we make use of the common regions of the two surfaces determined by the inconsistent surface registration method and repeat the mapping experiment in figure 5. As shown in figure 6, there are still mismatches of the landmarks and/or the surface geometry in all mapping results. This suggests that the inconsistent surface registration is important for ensuring a good mapping between the surfaces, and it cannot be replaced with a simple combination of some common region detection methods and the prior mapping methods.

Figure 6.

The mappings between the common regions of the two molars in figure 5 produced by the Procrustes superimposition method [2], the iterative closest point (ICP) method [64], the thin plate splines (TPS) method [3] and the non-rigid ICP method [65]. It can be observed that even only the common regions of the molars are considered, the methods are still unable to exactly match the landmarks and/or the surface geometry. (Online version in colour.)

From the inconsistent surface registration result, one can easily quantify the difference between two shapes. Specifically, because of the 1–1 correspondence between the optimal subdomains of the source and the target surfaces, we can assess the quasi-conformal distortion (in terms of the norm of the Beltrami coefficients |μ|), the normalized mean curvature difference, and the normalized Gaussian curvature difference between them. A comparison between two molars is shown in figure 7. It can be observed that the norm of the Beltrami coefficients |μ| effectively captures the cusp difference between the two molars. The normalized mean curvature difference and the normalized Gaussian curvature difference also highlight subtle shape difference between different parts of the two molars. This suggests that our approach is useful for quantifying shape difference.

Figure 7.

Quantifying the shape difference between two molars using our inconsistent surface registration method. Left column: the normalized mean curvature and the normalized Gaussian curvature of the source molar surface. Middle column: the normalized mean curvature and the normalized Gaussian curvature of the target molar surface. Right column: the norm of the Beltrami coefficients |μ|, the normalized mean curvature difference and the normalized Gaussian curvature difference computed based on the inconsistent surface registration result. (Online version in colour.)

To highlight the difference between our approach and the previous global landmark-matching quasi-conformal Teichmüller mapping method [54], we further consider comparing the two molars that are inconsistently segmented (figure 8). For the previous global quasi-conformal mapping method, the inconsistent parts of the two surfaces will be included in the computation. Hence, the global registration is largely distorted and fails to reflect the actual shape correspondence of the two molars. By contrast, our approach prevents such misalignment by optimally detecting the common regions of the two meshes. With the weaker assumption on the global surface correspondence, our approach is applicable to a wider class of shape analysis problems.

Figure 8.

A comparison between our inconsistent surface registration approach and the global landmark-matching quasi-conformal Teichmüller mapping method [54]. Here, the source mesh (leftmost) and the target mesh (second left) are inconsistently segmented. Our inconsistent surface registration approach successfully identifies and registers the common regions of the two meshes (second right). On the contrary, the landmark-matching Teichmüller mapping method [54] constructs a global 1–1 correspondence between the two meshes including the inconsistent parts, thereby leading to an overall misalignment with a large distortion (rightmost). (Online version in colour.)

(d). Cluster analysis of Platyrrhine molars

After demonstrating the effectiveness of our method for pairwise mappings, we deploy the method to register all 50 × 50 = 2500 pairs of the Platyrrhine molars. This provides us with the quasi-conformal distortion, the normalized mean curvature difference, and the normalized Gaussian curvature difference between the optimal subdomains of each pair of molars. We then proceed to use these quantities to cluster the 50 specimens into different groups.

(i). Binary clustering

We first consider clustering all specimens in the Platyrrhine molar dataset into two groups using the hierarchical clustering method, with some specific choices of the shape index weighting factors α, β, γ for building the dissimilarity matrix D examined.

If only the quasi-conformal distortion is considered (i.e. α = 1, β = 0, γ = 0) or only the normalized mean curvature difference is considered (i.e. α = 0, β = 1, γ = 0) for constructing the dissimilarity matrix, almost everything is clustered into one group. This suggests that simply using the quasi-conformal distortion or the mean curvature is insufficient to distinguish between the shapes of the Platyrrhine molars. However, for α = 0, β = 0, γ = 1, i.e. only the normalized Gaussian curvature difference is considered, we observe that all 10 Saimiri specimens are clustered into one group, while the other 40 specimens are clustered into another group (figure 9 (left)). One may wonder if there is any relationship between the shape of the teeth and their biological functions. Referring to the diets of these species, it is noteworthy that Saimiri are insectivores, i.e. animals that eat mainly insects, while the other four genera are not [66]. This suggests a potential intrinsic geometric difference between the molars of different genera due to a difference in diet.

Figure 9.

The binary clustering results produced by the hierarchical clustering method using our proposed dissimilarity measure, visualized on the multidimensional scaling (MDS) plane. The specimens from different genera are represented using different markers, and the colours of the markers represent the clustering result. The numerical label of each node corresponds to the specimen ID in table 1. For the result shown on the left, the parameters (α, β, γ) = (0, 0, 1) are used for constructing the dissimilarity matrix D. One of the resulting clusters consists of all Saimiri specimens (insectivores), while the other one consists of all non-insectivores. For the result shown on the right, the parameters (α, β, γ) = (0.25, 0.375, 0.375) are used for constructing D. One of the resulting clusters consists of all Alouatta and Brachyteles specimens (folivores), while the other one consists of all non-folivores. (Online version in colour.)

More relationships between function and shape can be observed for some other choices of non-zero α, β, γ. For instance, for (α, β, γ) = (0.25, 0.375, 0.375), all 20 specimens from Alouatta and Brachyteles are clustered into one group, and the other 30 specimens from Ateles, Callicebus and Saimiri are clustered into another group (figure 9 (right)). Interestingly, Alouatta and Brachyteles are folivores, i.e. animals that eat primarily leaves, while the species in the other three genera in the dataset are non-folivores [66]. According to the theory of primate adaptations [67], folivores are characterized by molars with extensive development of shearing crests for reducing the leaves into small particles so as to enhance digestion. It can be observed qualitatively from the images in figure 4 that the Alouatta and Brachyteles teeth have relatively taller cusps and deeper shearing crests when compared with the teeth from the other genera, which is possibly captured by such combination of shape index parameters α, β, γ. Once again, our shape analysis framework sheds light on the interplay between functions and shapes.

(ii). Clustering into five groups

It is also natural to ask if our method is capable of clustering the molars from different genera into different groups. To answer this question, we consider clustering the entire molar dataset into five groups using the k-means clustering method and evaluating the classification accuracy based on their respective genera. Here, the classification accuracy is defined by the percentage of correct cluster label pairs (i.e. two specimens from the same genus are with the same cluster label, or two specimens from different genera are with two different cluster labels) among all distinct pairs of specimens. The multidimensional scaling (MDS) method is first used to construct a projection of the molars onto the two-dimensional plane. Then, the k-means clustering is performed with k = 5.

We vary each of the parameters α, β, γ in the dissimilarity matrix D from 0 to 1, with an increment of 0.05. The maximum classification accuracy is 96.98%, achieved by multiple choices of parameters (α, β, γ) with α ranging from 0.1 to 0.25, β ranging from 0.15 to 0.55, and γ ranging from 0.3 to 0.6. Figure 10 (top left) shows the classification result for one of the optimal sets of parameters (α, β, γ) = (0.2, 0.4, 0.4). It can be observed that all specimens from Alouatta and Brachyteles are correctly clustered, and only one specimen from Ateles and one from Callicebus are wrongly clustered.

Figure 10.

The clustering results produced by the k-means clustering method with k = 5 using different dissimilarity measures, visualized on the multidimensional scaling (MDS) plane. The specimens from different genera are represented using different markers, and the colours of the markers represent the clustering result. The numerical label of each node corresponds to the specimen ID in table 1. Top row (left to right): the clustering results obtained by our method, the Procrustes superimposition method [2], and the Iterative Closest Point (ICP) method [64]. Bottom row (left to right): the clustering results obtained by the Thin Plate Splines (TPS) method [3], the non-rigid ICP method [65] and the global Teichmüller mapping method [54]. (Online version in colour.)

For comparison, we consider clustering the 50 specimens into five groups using the Procrustes method [2], the ICP method [64], the TPS method [3], the non-rigid ICP method [65] and the global Teichmüller mapping method [54]. More specifically, we apply the methods to compute the 50 × 50 pairwise mappings and form dissimilarity matrices. We then apply the k-means clustering on the dissimilarity matrices and evaluate the classification accuracy. The clustering results are shown in figure 10. Table 2 records the performance of different approaches for the clustering experiment, from which it can be observed that our method is more accurate than all other methods. The Procrustes method and the ICP method achieve a moderate classification accuracy possibly because only rigid transformations are considered. The TPS method achieves a relatively low classification accuracy due to the fact that it matches the prescribed landmarks while the overall shapes may be largely mismatched as previously illustrated in figure 5. The non-rigid ICP method performs better than other prior methods but is not as accurate as our proposed method. The better classification achieved by our method can be explained by the fact that the quasi-conformal distortion, the normalized mean curvature difference and the normalized Gaussian curvature difference are all captured by our dissimilarity matrix D. While the Teichmüller method also considers the curvature differences and the quasi-conformal distortion (in terms of the Teichmüller distance), it computes a global registration between two surfaces, and hence, the results are affected by the inconsistent parts of the surfaces. By contrast, our method computes an optimal inconsistent mapping between the surfaces, which effectively excludes the inconsistent parts in the subsequent shape analysis. The higher classification accuracy achieved by our method demonstrates the importance of the inconsistent shape registration for shape clustering.

Table 2.

The performance of different methods for clustering the 50 Platyrrhine molars into five groups, with the MDS dimension being 2. For each method, the 95% confidence interval on the classification accuracy and the leave-one-out cross-validation (LOOCV) accuracy are provided.

method	classification accuracy	LOOCV accuracy
our proposed method	96.29–96.93%	94%
Procrustes [2]	80.94–81.31%	70%
ICP [64]	87.17–88.02%	92%
TPS [3]	74.85–75.48%	62%
non-rigid ICP [65]	91.53–92.06%	84%
Teichmüller [54]	85.28–85.86%	80%

To assess the predictive performance of different methods and test if overfitting occurs, we further perform a leave-one-out cross-validation (LOOCV) on each method. Our method achieves a LOOCV accuracy of 94%, which is the highest among all approaches as shown in table 2. This indicates that our method is highly accurate for predicting new data.

In the above experiments, the MDS dimension is set to be 2 for embedding the shape data into a real vector space. One may wonder whether the results will be different if another MDS dimension is used. As shown in figure 11, our proposed method is capable of classifying the molar surfaces accurately for different MDS dimensions. Table 3 records the classification accuracies for all above-mentioned methods with different MDS dimensions, from which it can be observed that our method achieves the best overall performance among all approaches.

Figure 11.

The clustering results obtained by our proposed method with different multidimensional scaling (MDS) dimensions used, visualized on the respective MDS space. The specimens from different genera are represented using different markers, and the colours of the markers represent the clustering result. The numerical label of each node corresponds to the specimen ID in table 1. (Online version in colour.)

Table 3.

The performance of different methods for clustering the 50 Platyrrhine molars into five groups, with the MDS dimension being 1 or 3.

MDS dimension	method	classification accuracy	LOOCV accuracy
1	our proposed method	85.24–85.65%	78%
	Procrustes [2]	72.74–73.34%	60%
	ICP [64]	83.10–83.50%	76%
	TPS [3]	74.85–75.30%	64%
	non-rigid ICP [65]	87.83–89.32%	64%
	Teichmüller [54]	73.85–74.43%	58%
3	our proposed method	98.81–99.45%	98%
	Procrustes [2]	84.40–84.78%	72%
	ICP [64]	89.54–91.01%	92%
	TPS [3]	76.20–76.46%	60%
	non-rigid ICP [65]	98.38–98.51%	98%
	Teichmüller [54]	85.70–86.12%	80%

Altogether, the experimental results suggest that our proposed framework is capable of revealing the relationship between shape and phylogeny.

5. Conclusion and future works

Over the past several decades, numerous shape analysis methods have been developed using different mathematical tools in differential geometry, complex analysis, geometric measure theory, fluid mechanics, etc. However, most of the existing methods assume that there is a global 1–1 correspondence between every pair of shapes to be compared, which may lead to inaccuracies in the shape analysis of some datasets. In this work, we have proposed a new framework for quantifying shape variation via inconsistent surface registration. Given a set of simply connected open surfaces with prescribed landmarks, we compute an optimal landmark-matching quasi-conformal map between each pair of surfaces. Instead of assuming a global 1–1 correspondence such that every part of one surface corresponds to a certain part of the other surface, our approach searches for the optimal subdomains on the two surfaces that yield the best correspondence. This prevents the mapping results from being affected by the potential segmentation errors in the data. We then quantify the dissimilarity of the surfaces by evaluating the quasi-conformal distortion, the normalized mean curvature difference, and the normalized Gaussian curvature difference using the correspondence between the optimal subdomains. This further enables us to cluster the shapes based on their geometric difference. We have demonstrated the effectiveness of our proposed framework by deploying it on a mammalian tooth dataset. In particular, unlike the prior shape matching methods, our framework allows for the accurate quantification of shape difference between two tooth surfaces, with the prescribed landmarks exactly matched and the common regions optimally aligned. Our method also successfully produces geometry-based clustering results that highly correlate with the underlying biological difference between the specimens.

Note that our proposed framework allows for the exact correspondence of feature landmarks in computing the optimal inconsistent shape registration, based on the assumption that the prescribed landmark positions are precise. In case there are potential manual errors in the landmark labelling step so that an inexact landmark-matching registration is more desirable, we can simply add an intensity-matching step at the end of algorithm ??, thereby producing an optimal landmark-guided registration with a small degree of landmark mismatch allowed. Also, our method can be potentially extended for surfaces with other topology, such as genus-0 closed surfaces and high-genus surfaces. For instance, to handle genus-0 closed surfaces, one may replace the current conformal flattening step with a spherical conformal parametrization step [46] followed by the stereographic projection. For high-genus surfaces, one may replace the current conformal flattening step with a conformal parametrization onto a fundamental domain on the plane. It is also possible to modify the shape dissimilarity measure for different shape analysis tasks. For instance, in case it is desirable to take the amount of common area shared by the two surfaces into account, one may add an extra area difference term $\frac{A(S_1)-A({\mathit{\Omega }}_1)}{A(S_1)}$ in the combined shape index in equation (3.10) and an extra term $\frac{A(S_1)-A({\mathit{\Omega }}_2)}{A(S_2)}$ in equation (3.12). The newly added terms will be zero if and only if the common regions are the two entire surfaces, and hence the new dissimilarity measure will be able to capture the area variation of the surfaces.

In the future, we plan to deploy our inconsistent shape analysis framework to study other biological shapes. In particular, our method is suitable for analysing human faces as each human face contains prominent landmarks such as the eyes, the nose and the mouth, while it is usually hard to precisely segment the face boundary due to three-dimensional scanning errors and potential hair occlusions, etc. We also plan to investigate the combination of our method with other landmark-free anatomical surface mapping methods such as [68] for further improving the accuracy of shape classification.

Data accessibility

The MATLAB codes are available at https://scholar.harvard.edu/files/choi/files/inconsistent_shape_analysis.zip.

Authors' contributions

G.P.T.C. and L.M.L. conceived and designed the research. G.P.T.C. and D.Q. implemented and conducted the simulations and analysed the results. G.P.T.C., D.Q. and L.M.L. wrote the manuscript.

Competing interests

We declare we have no competing interests.

Funding

This work was supported by HKRGC GRF (Reference: 14304715, Project ID: 2130447) (to L.M.L). This work was also supported in part by the Harvard Quantitative Biology Initiative and the NSF-Simons Center for Mathematical and Statistical Analysis of Biology at Harvard, award no. 1764269 (to G.P.T.C.).