OPTIMIZATION-DRIVEN STATISTICAL MODELS OF ANATOMIES USING RADIAL BASIS FUNCTION SHAPE REPRESENTATION

Abstract

Particle-based shape modeling (PSM) is a popular approach to automatically quantify shape variability in populations of anatomies. The PSM family of methods employs optimization to automatically populate a dense set of corresponding particles (as pseudo landmarks) on 3D surfaces to allow subsequent shape analysis. A recent deep learning approach leverages implicit radial basis function representations of shapes to better adapt to the underlying complex geometry of anatomies. Here, we propose an adaptation of this method using a traditional optimization approach that allows more precise control over the desired characteristics of models by leveraging both an eigenshape and a correspondence loss. Furthermore, the proposed approach avoids using a black-box model and allows more freedom for particles to navigate the underlying surfaces, yielding more informative statistical models. We demonstrate the efficacy of the proposed approach to state-of-the-art methods on two real datasets and justify our choice of losses empirically.

1. INTRODUCTION

Statistical Shape Modeling (SSM) is a family of techniques used to quantify shape variability given a population of anatomies. SSM has advanced the understanding of different diseases and disorders [1-6]. SSM relies on representing shape variation in a compact way to perform subsequent statistical analyses. The two most popular shape representations used for building SSMs are deformation fields and landmarks. Deformation fields encode implicit transformations between population samples and a pre-defined (or learned) atlas. Landmarks are explicit points spread on shape surfaces that correspond across the population and serve as proxies to the surface for subsequent statistical analysis [7]. Shape analysis used to be performed by manually placing landmarks and performing statistics on them, but this was labor-intensive, not reproducible, and required extensive domain expertise (e.g., radiologists). Modern computational methods allow the automatic computation of deformations or placement of landmarks, shifting the SSM field to data-driven characterization of population-level variabilities that is objective, reproducible, and scalable. These methods include minimum description length – MDL [8], particle-based shape modeling – PSM [9] for landmarks, and frameworks based on Large Deformation Diffeomorphic Metric Mapping for deformation fields (e.g., [10]).

Landmark-based representations, also known as point distribution models (PDMs), have been favored for analyzing surface variability due to their simplicity, computational efficiency, and interpretability for statistical analyses [7]. However, such scalability suffers when considering that some applications require thousands of densely placed landmarks to capture shape structure faithfully when representing intricate shape surfaces with localized or convoluted shape features that may live between landmarks. The state-of-the-art PDM method, PSM [9], produces dense particle placements. However, it provides a uniform surface sampling, and thus fail to adapt to the underlying geometrical intricacies at different scales.

Image2SSM [11] first proposed the use of radial basis function shape representations (RBF-shapes) for the SSM task. RBF-shapes are continuous surface representations instead of simply discrete surface proxies. They allow particles to self-adapt to poorly represented surface areas while building the statistical model. Despite its benefits, the black-box nature of such a deep network optimization undermines the user’s ability to understand and control the process of building SSMs, not to mention the computational cost of parameterizing the PDM construction with a deep network. Furthermore, parameterizing landmarks using a network limits particles manipulation at both training and inference times, leading to poor surface adherence and configuration congruency.

To address these limitations, this paper proposes a non-deep optimization-based solution that leverages RBF-shapes to build SSMs. It has the advantage of being self-adaptive to the underlying geometry and can further reduce the number of required particles for intricate surfaces. Additionally, not using a black box model deep learning approach gives our method much more control over the particle spread and placement since they are not parametrized with a deep neural network. We also introduce the simultaneous use of correspondence and eigenshape losses to improve configuration consistency while emphasizing dominant modes of variation. Additionally, enhancing traditional optimization techniques contributes to the advancement of the deep learning variants for PDM construction (e.g., [12-15]) by providing stronger shape priors to train the networks. We show that our method performs similarly or better than PSM [9] and Image2SSM [11] on real datasets in correspondence metrics and two-way surface-to-surface distance. We also show how the correspondence and eigenshape losses function to build better PDMs. The proposed method will be integrated into the open-source package for building PDMs, ShapeWorks [9].

2. METHODS

A statistical shape model should balance an accurate geometrical representation of the individual surfaces in the given cohort while preserving landmark configuration congruency across the surfaces, referred to as correspondence, encoding shape variations in a compact set of parameters. The proposed method optimizes four losses using stochastic gradient descent to achieve such a balance by explicitly emphasizing and controlling the different characteristics of an SSM. The objective is to optimize for a set of landmarks (or control points in RBF literature). These control points are fit to a cohort of shapes in the form of segmentations $𝒮={\{{\mathbf{S}}_i\}}_{i=1}^I$. We optimize for a set of $J$ control points $𝒫={\{{\mathbf{P}}_i\}}_{i=1}^I$ for each input shape, where the $i-\text{th}$ shape PDM is denoted by ${\mathbf{P}}_i=[{\mathbf{p}}_{i,1},{\mathbf{p}}_{i,2},\cdots ,{\mathbf{p}}_{i,J}]$ and ${\mathbf{p}}_{i,j}\in {\mathbb{R}}^3$. The RBF-shape can then reconstruct a full surface using these points and their surface normals by interpolating between them. This provides online information to the model regarding how well each part of the surface is represented, leading to adaptive updates.

This section will first explain how the RBF-shape can estimate reconstruction error at each region and how each of the four losses is involved in the optimization.

2.1. Radial Basis Function Surface Representation

RBF-shape is an implicit surface representation based on radial basis functions, which can represent intricate shapes by leveraging both surface control points and normals to inform shape reconstructions [16,17]. It uses a set of control points at the zero-level set and a pair of off-surface points (aka dipoles) with a signed distance $s$ and $-s$ along the surface normal of each control point. This is illustrated in Fig. 1.

Fig. 1.

A set of control points and their respective dipoles can define an implicit signed distance function within a narrow band that can be efficiently queried for surface representation accuracy at any narrow band point.

We refer to the set of control points and their dipoles as $\widetilde{{\mathbf{P}}_i}$ for shape $i$, where $\widetilde{{\mathbf{P}}_i}=[{\mathbf{P}}_i,{\mathbf{P}}_i^{+},{\mathbf{P}}_i^{-}]$ with ${\mathbf{p}}_{i,j}^{\pm }={\mathbf{p}}_{i,j}\pm s{\mathbf{n}}_{i,j}$. The shape’s implicit function, which maps a point $\mathbf{x}\in {\mathbb{R}}^3$ to its signed distance to the surface can then be written as:

$$f_{\widetilde{{\mathbf{P}}_i},{\mathbf{w}}_i}(\mathbf{x})=\sum _{j\in \widetilde{{\mathbf{P}}_i}}{w}_{i,j}\varphi (\mathbf{x},{\tilde{\mathbf{p}}}_{i,j})+{\mathbf{c}}_i^T\mathbf{x}+c_i^0$$ (1)

where $\varphi$ can be any RBF basis function (e.g., the thin plate spline $\varphi (\mathbf{x},\mathbf{y})={(\mid \mathbf{x}-\mathbf{y}{\mid }_2)}^2\log (\mid \mathbf{x}-\mathbf{y}{\mid }_2)$, the biharmonic $\varphi (\mathbf{x},\mathbf{y})=\mid \mathbf{x}-\mathbf{y}{\mid }_2$ or the triharmonic $\varphi (\mathbf{x},\mathbf{y})={(\mid \mathbf{x}-\mathbf{y}{\mid }_2)}^3$) and ${\mathbf{c}}_i\in {\mathbb{R}}^3$ and $c_i^0\in \mathbb{R}$ encode the linear component of the surface. We obtain ${\mathbf{w}}_i=[w_{i,1},w_{i,2},\dots ,w_{i,3J},c_i^0,c_i^1,c_i^2,c_i^3]\in {\mathbb{R}}^{3J+4}$ by solving a system of equations formed by Eq 1 over $\mathbf{x}\in \widetilde{{\mathbf{P}}_i}$ and regularization factors to form a fully determined system, fully described in [16, 17]. The function $f$ provides an implicit shape representation that is resolution agnostic and can be queried to construct segmentations and surface meshes at arbitrary resolution. This representation excels at adapting to the underlying surface geometry, leading to better shape representation using fewer control points.

2.2. SSM Optimization

Control points for each surface are initialized using the same set of points sampled uniformly at random. During the optimization, the four losses ensure that the control points (1) live on the surface, (2) maintain the same configuration across all shapes, (3) push the population-level variability to dominant modes of variation, and (4) are spread according to the reconstruction error of the RBF-shape representation. Together, these losses promote a balance between the geometric representation of the individual surfaces and maintaining consistent placement of control points across the population while resulting in a compact statistical model.

Surface loss:

The surface loss aims to force control points to lie on the surface. For each shape $i\in I$, we define this loss as

$$L_{{\mathbf{D}}_i}^{surf}({\mathbf{P}}_i)=\sum _{j=1}^J\mid {\mathbf{D}}_i({\mathbf{p}}_{i,j})\mid ,$$ (2)

where ${\mathbf{D}}_i({\mathbf{p}}_{i,j})$ is the true distance to the closest surface point to ${\mathbf{p}}_{i,j}$, obtained by querying the signed distance transform.

Correspondence loss:

The notion of control points correspondence across the shape population can be quantified by the Frobenius norm of the covariance of the control points. In simple terms, this loss attempts to collapse each control point to its mean across all shapes. It is written as

$$L_{\mu }^{corres}({\mathbf{P}}_1,\dots ,{\mathbf{P}}_K)={\Vert \sum _{k=1}^K({\mathbf{P}}_k-\mu ){({\mathbf{P}}_k-\mu )}^T\Vert }_F$$ (3)

where $\mu ={\sum }_{i=1}^I{\mathbf{P}}_i$ is the mean control points, and ${\Vert \cdot \Vert }_F$ the Frobenius norm.

Eigenshape loss:

This loss pushes control points to align with the most dominant modes of variability. In [9], this loss is the one used as the correspondence loss (derived from [18]). This objective does not improve correspondence of the control points, but rather improves compactness by aligning control points to the principal components. This observation aligns with the source claims about this loss in [18], which described this objective as being equivalent to minimizing the Mahalanobis distance of the system. However, due to the eigenvalue scaling inherent in the Mahalanobis distance definition, missed correspondences get captured by the principal components and are not corrected by this loss, making it a poor loss to enforce correspondence. This shortcoming is shown empirically in our results section, hence our choice of using the Frobenius norm loss to enforce correspondence. Nonetheless, including this loss is imperative to improving the compactness of the resulting shape model, which is a desirable property in the analysis that does not severely interfere with good sampling and correspondence.

Given a minibatch of size $K$, the eigenshape loss is formulated using the differential entropy $H$ of the samples in the minibatch, assuming a Gaussian distribution.

$$\begin{gathered} L_{\mu }^{eigen}({\mathbf{P}}_1,\dots ,{\mathbf{P}}_K)=H(\mathbf{P}) \\ =\frac{1}{2}\log \mid \frac{1}{3JK}\sum _{k=1}^K({\mathbf{P}}_k-\mu ){({\mathbf{P}}_k-\mu )}^T\mid \end{gathered}$$ (4)

where $\mathbf{P}$ here indicates the random variable of the shape space and $\mid \cdot \mid$ is the matrix determinant. Both the correspondence and eigenshape loss are triggered starting from the second epoch as soon as a mean particle set $\mu$ is available.

Sampling loss:

This loss encourages the control points to adapt to the underlying geometry. We query the distance at $R$ randomly sampled points ${\mathbf{B}}_i=[{\mathbf{b}}_{i,1},\dots ,{\mathbf{b}}_{i,R}]$ that lie within a narrow band at $\pm s$ distance from the surface. The control points are then guided to poorly represented areas by minimizing their distance to the surrounding narrow band points, scaled by the queried approximate violation intensity.

To define surrounding narrow band points, let ${\mathbf{K}}^i\in {\mathbb{R}}^{R\times M}$ be the matrix containing the pairwise distances between each narrow band point ${\mathbf{b}}_{i,r}$ and each control point ${\mathbf{p}}_{i,j}$, such that, for the $i-\text{th}$ shape, the $r$, $j-\text{th}$ element is $k_{r,j}^i={\Vert {\mathbf{b}}_{i,r}-{\mathbf{p}}_{i,j}\Vert }_2$. We use a soft minimum function to allow each narrow band point to influence all nearby control points, written as $\text{softmin}({\mathbf{K}}^i)$, where the $r$, $j\text{th}$ element of $\text{softmin}({\mathbf{K}}^i)$ is computed as $\text{exp}(-k_{r,j}^i)∕{\sum }_{j^{\prime }=1}^J\text{exp}(k_{r,j^{\prime }}^i)$. This soft minimum is normalized over ${\mathbf{P}}_i$. Finally, the RBF approximation squared error at the narrow band points is captured by ${\mathbf{e}}_i\in {\mathbb{R}}_{+}^R$, where $e_{i,r}={[f_{\widetilde{{\mathbf{P}}_i},{\mathbf{w}}_i}({\mathbf{b}}_{i,r})-{\mathbf{D}}_i({\mathbf{b}}_{i,r})]}^2$. Let ${\mathbf{E}}_i={\mathbf{e}}_i{1}_M^T$, where $1_M$ is a ones-vector of size $M$. The sampling loss is then written as

$$\begin{gathered} L_{{\mathbf{B}}_i,{\mathbf{D}}_i,{\mathbf{w}}_i}^{sampl}({\mathbf{P}}_i,{\mathbf{N}}_i)= \\ \text{mean}(\text{softmin}({\mathbf{K}}_i)\otimes {\mathbf{K}}_i\otimes (c{\mathbf{E}}_i+1)) \end{gathered}$$ (5)

where $\otimes$ indicates the Hadamard (elementwise) multiplication of matrices, $c$ is a weighting variable, and mean computes the average over all the matrix elements. Adding 1 to ${\mathbf{E}}_i$ gives each narrow band point at least a unit pull.

Total loss:

We optimize minibatches of size $K$ at a time with a total loss written as follows:

$$\begin{gathered} L_{ℐ,𝒟,\partial 𝒟}({𝒫}_K,{𝒩}_K)=\sum _{i=1}^K(\alpha L_{{\mathbf{D}}_i}^{surf}({\mathbf{P}}_i)\phantom{)} \\ +\phantom{(}\beta L_{{\mathbf{B}}_i,{\mathbf{D}}_i,{\mathbf{w}}_i}^{sampl}({\mathbf{P}}_i,{\mathbf{N}}_i)) \\ +\gamma L_{\mu }^{eigen}({\mathbf{P}}_1,\dots ,{\mathbf{P}}_K) \\ +\zeta L_{\mu }^{corres}({\mathbf{P}}_1,\dots ,{\mathbf{P}}_K) \end{gathered}$$ (6)

where $\alpha$, $\beta$, $\gamma$, and $\zeta \in {\mathbb{R}}_{+}$ are hyperparameters of each losses and ${𝒫}_K$, ${𝒩}_K$ are the control points and their surface normals of the samples in the minibatch.

3. RESULTS

Our results demonstrate that the proposed method outperforms Image2SSM [11] in both two-way surface-to-surface distance and correspondence metrics while improving slightly over PSM [9]. Additionally, we showcase the effect of the correspondence loss and the eigenshape loss on a single major axis variation ellipsoid toy example to justify our choice of loss functions.

3.1. Datasets and Implementation Details

We perform tests on two real datasets. First, a dataset of signed distance transforms from 40 proximal femur CT scans devoid of pathologies. The second dataset consists of 814 left atrium signed distance transforms, a challenging dataset due to size and high shape variability.

We used the PyTorch 1.12.1 Autograd functionality to automatically backpropagate our losses using the SGD optimizer. As a preprocessing step, a set of uniformly sampled control points is used to initialize the optimization on a single shape. These control points then serve as the initialization for all shapes during the optimization of all control points. This preprocessing is not strictly necessary as the main optimization can be initialized with the same random set of points, but it reduces the total number of iterations needed by leveraging this reasonable prior. This is in contrast to the PSM initialization, which utilizes a multi-scale particle splitting routine to initialize the shape model.

At each epoch, 10,000 narrow band points are sampled from the narrow band of each surface to accelerate computation. We used the biharmonic kernel for the basis function, but the results are observed to be agnostic to this choice. The hyperparameters we used are learning_rate = 1, α = 0.1, β = 1.0, γ = 0.01, ζ = 0.01, and c = 10 for femurs; changing ζ = 0.1, γ = 0.001, ζ = 0.001 for left atria; and α = 0.01, and c = 1 for ellipsoids.

3.2. Shape Model Quality

We evaluate our approach against the state-of-the-art Image2SSM [11] and PSM [9] in building a model with 128 particles. Fig. 2 shows that our approach matches or outperforms Image2SSM and PSM in the maximum two-way surface-to-surface distance of the reconstruction while yielding a slightly poorer mean distance than PSM. It resolves the worst surface misrepresentations but fails to capture the general surface as closely without an explicit snap-to-surface routine like PSM uses.

Fig. 2.

The two-way surface-to-surface distance between Image2SSM, PSM, and our approach.

Fig. 3 shows conventional metrics of compactness (percentage of variance captured), specificity (ability to generate realistic shapes), and generalization (ability to represent unseen shape instances) [19] for femurs and left atria compared to Image2SSM and PSM for the first ten modes of variation, where it yields comparable compactness but better specificity and generalization.

Fig. 3.

Shows the compactness (higher is better), specificity (lower is better), and generalization (lower is better) for ten modes on femurs and left atria.

3.3. Correspondence and Eigenshape Losses

For simplicity, we show the effects of the correspondence and eigenshape loss on a population of ellipsoids where only the major axis $x$ varies. Fig. 4(a) shows how a missed correspondence is not only not fixed by the correspondence loss, but is captured in its main mode of variation. Fig. 4(b) shows how even with perfect correspondence, courtesy of the correspondence loss, the percentage variance of the main mode and compactness are worse than 4(a) due to the lack of eigenshape loss. Fig. 4(c) shows how we can achieve a compact model with good correspondences using both losses. All in all, this shows not only that the eigenshape loss is inadequate to resolve correspondences, but it can also generate spurious variability in the main modes of variation.

Fig. 4.

By changing $\gamma$ and $\zeta$, we show the results of running our optimization with (a) only correspondence loss, (b) only eigenshape loss, and (c) both. The red circle highlights two control points that are swapped for 6 out of 20 shapes.

4. CONCLUSION

We demonstrate an approach for improving PDM construction using a traditional optimization technique. Our proposed losses offer new insights regarding the development of correspondence models that better capture variability while being faithful to the surface representations. Future work involves fully exploring the effect of existing losses for correspondence and how they interact against the eigenshape loss, as well as furthering RBF-shape’s ability to capture surface error to enable surface-informed non-uniform sampling.