Skeletal shape correspondence via entropy minimization

Abstract

Purpose

Improving the shape statistics of medical image objects by generating correspondence of interior skeletal points.

Data

Synthetic objects and real world lateral ventricles segmented from MR images.

Method(s)

Each object’s interior is modeled by a skeletal representation called the s-rep, which is a quadrilaterally sampled, folded 2-sided skeletal sheet with spoke vectors proceeding from the sheet to the boundary. The skeleton is divided into three parts: up-side, down-side and fold-curve. The spokes on each part are treated separately and, using spoke interpolation, are shifted along their skeletal parts in each training sample so as to tighten the probability distribution on those spokes’ geometric properties while sampling the object interior regularly. As with the surface-based correspondence method of Cates et al., entropy is used to measure both the probability distribution tightness and sampling regularity. The spokes’ geometric properties are skeletal position, spoke length and spoke direction. The properties used to measure the regularity are the volumetric subregions bounded by the spokes, their quadrilateral sub-area and edge lengths on the skeletal surface and on the boundary.

Results

Evaluation on synthetic and real world lateral ventricles demonstrated improvement in the performance of statistics using the resulting probability distributions, as compared to methods based on boundary models. The evaluation measures used were generalization, specificity, and compactness.

Conclusions

S-rep models with the proposed improved correspondence provide significantly enhanced statistics as compared to standard boundary models.

1. INTRODUCTION

Establishing correspondence across shape populations is the basis for accurate statistical shape analysis. This is commonly achieved by identifying a set of manually or automatically sampled boundary points also called a Point Distribution Model (PDM). Cootes and Taylor [1] first investigated the correspondence problem of manually defined significance points in a 2D boundary PDM, which is extensible to 3D. Gerig et al. [2] proposed a shape analysis method based on a parametric boundary description using spherical harmonics (SPHARM) which was first presented in Brechbuhler’s dissertation [3]. Styner et al. [4] extended the SPHARM parametric description to provide the implied PDM and this PDM’s use for statistical shape analysis. Since shape models that are PDM based directly depend on the quality of surface positional alignment, the shape statistics are highly related to the alignment and are not able to differentiate shape changes from positional differences.

An alternative, the medial model (m-rep) [5], which employs local thickness information as alignment-independent shape metric, was widely investigated [6-8]. More recently, as an extensively extension to the m-rep a skeletal model called the s-rep [9] has been shown to be particularly powerful [10-12] for describing not just the object boundary but also its interior using spokes, as opposed to PDMs which only describe the boundary of the object.

Cates et al. [13] proposed a method using both geometry entropy and regularity entropy to establish correspondence among the boundary PDMs of a group of similar shapes. Inspired by their experiment, we present a novel approach for improving shape statistics by optimizing the correspondence of interior, skeletal points based on s-rep spokes rather than boundary PDMs. That is, we describe the object as s-rep and shift the spokes on the skeletal sheet to tighten the probability distribution of spoke geometric properties while keeping the spoke distribution in each object interior uniform.

The proposed method is evaluated on both synthetic and real world lateral ventricles. We show the improvement in the performance of statistics using the resulting probability distributions, as compared to methods based on boundary models. We also show the regularity differences of the object interior before and after optimization.

To this end, the paper is structured as follows: section 2 describes the data used in this study; section 3 details the proposed method; the evaluation results are describes in section 4, followed by the conclusion, possible applications and future directions of this study discussed in section 5.

2. DATASETS

We applied our proposed methods to a set of lateral ventricle objects semi-automatically segmented from MRI images in neonate datasets. The overall dataset and segmentation procedures are described in [14]. We selected 30 lateral ventricles for our tests presented here. Furthermore, we generated 80 synthetic lateral ventricles from a template s-rep by displacing a set of spokes by a small (random) distance according to a Gaussian distribution of predefined variance.

3. METHODS

Our correspondence approach is composed of five essential sections (the boldface sections are the main contributions of this paper): 1) S-rep Modeling, 2) Spoke Shifting, 3) Uniformity Properties Calculation, 4) Geometric Properties Calculation, 5) Energy Function and Optimization.

3.1 S-rep Modeling

The s-rep [9] is formed by a folded 2-sided skeletal sheet with spoke vectors proceeding from the sheet to the boundary (see Figure 1 (a)). The skeleton is divided into three parts: up side, down side and fold curve. The object interior can be completely represented by interpolating the discrete s-rep into a continuous skeleton with a continuous field of spokes forming a continuous s-rep whose spokes fill the interior of the object (Figure 1 (b)). Moreover, the s-rep provides an object-related intrinsic coordinate system for the points in the object interior, boundary and even near exterior. S-reps are initially fit to the training objects using the method described in [9] preceded by the deformation of a template s-rep into each object. The deformation is based on a SPHARM object correspondence interpolated via thin plate splines.

Fig. 1.

(a) An example of s-rep for lateral ventricle that is sampled as a folded skeletal points (white balls). Green squares composed the skeletal sheet; (b) object boundary implied by that s-rep.

3.2 Spoke Shifting

We develop a novel spoke sliding mechanism that iteratively shifts the s-rep spoke along its skeletal part with a small step and updates the spoke information using the new interpolated one. Spoke interpolation [7, 15] is used to produce the new spoke at the shifted position. Correspondence is improved by optimizing the geometry entropy of the spokes over the population of objects together with the regularity entropy of the spokes per object summed over all objects. The shifted spokes are used to compute the entropies.

3.3 Uniformity Properties Calculation

Each object is modeled with three regions: an up-side, a down-side and a crest-fold region (see Figure 2). For each region, the uniformity properties are computed separately and then summed across all regions. Given a m n s-rep with m “horizontal” and n “vertical” grid locations, there are k = m × n up-side spokes, k down-side spokes, and f = 2 (m + n − 2) fold spokes.

Fig. 2.

(a) Up side (green) and down side (red) quads; (b) the volume between two corresponding quads (cyan on skeletal, yellow on boundary) is subdivided into (2³)² sub-volumes, the blue dodecahedron (₅) illustrates one of them; (c) fold spokes; (d) the hexahedron between corresponding quad on boundary and the vertical edge on skeletal.

For up or down-side regions, as illustrated in Figure 2 (a) and (b), four neighbor spoke tails compose a skeletal quad and four neighbor spoke tips compose a boundary quad. The line connecting two adjacent quad vertices in the horizontal direction called a horizontal edge, whose length is indicated as $L{\left(h\right)}_e^g$; where g = K for a skeletal quad and g = b for a boundary quad and where $e∊(1,\dots ,E)E=(\mathrm{m}-1)\times \mathrm{n}$ is the horizontal edge number. The same definition applies to vertical quad edges with length $L{\left(v\right)}_w^g$ where index $w∊(1,\dots ,W),W=(n-1)\times m$ is the vertical edge number. Similarly, quad areas are denoted as $A_w^g$. Finally, the volume of the dodecahedron between two corresponding quads is indicated as V_i, where $i\epsilon (1,\dots ,I),I=(m-1)\times (n-1)$ is the quad number. According to this, the quad area and volume property is a M_qav matrix, the horizontal edge length property M_hel is a 2 × E matrix, the vertical edge length property M_vel is a 2 matrix.

The crest side region is formed by 3-tuples of spokes with a common skeletal point, on the fold curve (green line) of the skeletal sheet, as illustrated in Figure 2 (c) and (d). Two neighbor fold spokes (red lines in (c) with their tails on the fold curve) along the direction compose vertical edges $L{\left(V\right)}_t^K$, and their tips on the boundary (blue line) compose vertical edges $L{\left(v\right)}_t^k$. In the complementary direction, the spoke tips on the up side crest and down side crest, which is interpolated from the fold spoke, compose a horizontal edge $L{\left(v\right)}_t^B$. The four spoke tips derived from two neighbor fold spokes compose a boundary quad with area $A_t^B$. The volume of the hexahedron between $L{\left(v\right)}_t^B$ and $L_t^B$ is indicated as V_t, where $t\epsilon (1,\dots ,f),f$, f is the crest quad number M_qav is a 2 matrix; M_hel is a matrix; M_hel is a 2 matrix.

In all cases, each edge of the original quad is linearly subdivided into 2^σ sub-edges, and each quad is divided into (2^σ)² sub-quads, where σ is the interpolation level. In this paper we use σ = 2 in the reported results.

3.4 Geometric Properties Calculation

Spoke directions and skeletal point tuples modulo scaling are of a non-Euclidean nature; they live naturally on abstract spheres. In order to appropriately incorporate this in our computation, we employ composite Principal Nested Spheres (PNS) [16] proposed by Jung and Marron to analyze the s-rep data.

Generally, given the structure $\mathcal{M}={\mathbb{R}}^{d_1}\times {\mathbb{S}}^{d_2}\times {\left({\mathbb{S}}^2\right)}^{d_2}$, where d₁, d₂, d₃ ≥ 1, composite PNS decomposes ℳ into a Euclidean component R^d1 and d₃ + 1 spherical components: one S^d2 and d₃ copies of S². Applying PNS to each of the spherical components yields a column of 2d₃ Euclideanized features (of which d₂ − N will be zero if d₂ > N) from S^d2 and a column of 2d₃ Euclideanized features from the copies of S². The Euclideanized features are further centered via mean subtraction and standard deviation scaling to make them all commensurate. Concatenating these columns yields a feature tuple of length d = d₁ + d₂ + 2d₃. This feature tuple forms the i^th column of the composite Euclideanized geometric properties matrix M_geoof size d × N.

Consider a 3D object in a training set of N observations each represented by the s-rep discussed before. The set of skeletal points forms a PDM that can be aligned to bring its center of gravity to the origin. The tuple of centered points can then be normalized by a scale factor, which is the square root of the sum of squared distances of the centered points. Thus, this PDM is described by a log-transformed scale factor and a tuple of normalized, centered points. The directional component of each spoke abstractly lives on the unit 2-sphere S², and the log-transformed associated length component of each spoke lives on Euclidean space R¹. For the up or down-side, the representation abstractly lives on R^k+1 × S^3k−4 × (S²)^k, and the crest-fold representation lives on R^f+1 × ^3f−4 (S²)^f . Each part is Euclideanized to form its own M_geofor geometry entropy computing.

3.5 Energy Function and Optimization

We define entropy based on Cates’ idea presented in [13] and build our cost function as

$$\underset{x}{\mathrm{argmin}}\left\{E_{geo}-\alpha \sum _{i=1}^N(E_{qav}+E_{hel}+E_{vel})\right\}$$

where E_geois the geometric entropy, $E_{reg}={\sum }_{i=l}^N(E_{qav}+E_{hel}+E_{vel})$ is the regularity entropy with E_qav and E_hel indicates the quad area and volume entropy, the horizontal edge length entropy and vertical edge length entropy respectively, α is the weight used to balance the tightness and regularization. To prevent the smallest modes (those with smallest eigenvalues), which mainly constitute effects of noise, from disturbing the optimization process, eigenvalues with contributions relative to total variance smaller than a prior threshold (ϑ) were removed. For E_geo, ϑ = 0.1 %; for each of the three parts entropy in E_reg, ϑ = % was used. Finally, our objective function was minimized using an iterative, alternate application of the NEWUOA optimizer [17] and the one-plus-one evolutionary optimizer [18].

4. RESULTS

The proposed correspondence method has been evaluated on both synthetic and real world lateral ventricles. In both settings we observed a significant decrease in the geometry entropy measure (E_geo) after optimization, as shown in Figure 3. The red line indicates the cost function. The decreasing of E_geomeans the spokes geometric properties are effectively tightened.

Fig. 3.

Left: the entropies for the crest-fold region (iterations in 10⁴) on the synthetic data set; right: the entropies for the up-side region on the real world data set (iterations in 10⁴). To avoid redundancy, the entropies of the other regions are omitted here as they decrease in a similar way.

From Figure 3 we can also see that the regularity entropy (E_reg) increases for higher iterations. This indicates that the computed, optimized s-reps have a more uniform spoke distribution than the initial s-reps as expected. To further explain the regularity, Figure 4 compares the boundary quad areas (left), the skeletal quad areas (left) and the volumes (right) of the dodecahedron between corresponding boundary-skeletal quads across all the quads of a single s-rep in its initial, aligned, and optimized version. This figure shows all these properties getting more tightly distributed.

Fig. 4.

Gaussian distribution of quad area and volume of the original, Procrustes aligned, and optimized s-reps. Left: Uniformity of quad areas on boundary (dash-dot line, $A_i^B$) and skeletal (solid line, $A_i^K$). Green is the original s-rep, blue is the aligned s-rep, and magenta is the optimized s-rep. Right: Uniformity of volumes (V_i) of the dodecahedron between $A_i^B$ and $A_i^K$ is the index of quad number.

The evaluation of a given correspondence via a statistical shape model defined by this correspondence has been proposed previously in order to gain insight for the correspondence’s applicability for the purpose of shape modeling. We evaluated the proposed method on three standard statistical model metrics: generalization (G(M)), specificity (S(M)) and compactness (C(M)). All three were first introduced by Davies’ in his dissertation [19]. Figure 5 illustrated the shape model performance of the proposed method as compared with SPHARM-PDM method. The proposed method obtains significantly improved results, even if we only consider only the boundary points/PDM implied by the s-reps (spoke tips). It is noteworthy that the proposed method does not optimize directly over the locations of the boundary points.

Fig. 5.

Comparison of correspondence quality between the s-rep implied PDM (spoke tips) and the SPHARM implied PDM applied to set of lateral ventricles. Left: generalization; middle: specificity; right: compactness in both variance and dimension.

5. CONCLUSION

In this work we proposed a novel group-wise optimization of skeletal properties to establish an enhanced s-rep correspondence. Each object’s interior is thereby modeled by a skeletal model of a quadrilateral skeletal node-grid with spoke vectors to the boundary. The spokes of the skeleton are shifted so as to tighten the probability distribution on those spokes’ geometric properties while sampling the object interior regularly. Entropy is used to measure both the probability distribution tightness and sampling regularity. This method yields models with significantly improved model properties as measured via generalization, specificity and compactness. In the future, we will explore the fitting procedure to obtain better performance on shape statistical analyses such as classification, hypothesis testing, and probability distribution estimation, as well as a variety of medical image analysis methods dependent on these statistical analyses.