ACCELERATION CONTROLLED DIFFEOMORPHISMS FOR NONPARAMETRIC IMAGE REGRESSION

Abstract

The analysis of medical image time-series is becoming increasingly important as longitudinal imaging studies are maturing and large scale open imaging databases are becoming available. Image regression is widely used for several purposes: as a statistical representation for hypothesis testing, to bring clinical scores and images not acquired at the same time into temporal correspondence, or as a consistency filter to enforce temporal correlation. Geodesic image regression is the most prominent method, but the geodesic constraint limits the flexibility and therefore the application of the model, particularly when the observation time window is large or the anatomical changes are non-monotonic. In this paper, we propose to parameterize diffeomorphic flow by acceleration rather than velocity, as in the geodesic model. This results in a nonparametric image regression model which is completely flexible to capture complex change trajectories, while still constrained to be diffeomorphic and with a guarantee of temporal smoothness. We demonstrate the application of our model on synthetic 2D images as well as real 3D images of the cardiac cycle.

1. INTRODUCTION

Image regression serves a prominent role in many medical image analysis frameworks. It may be used as a statistical representation of change over time, either cross-sectionally, or for subject specific change trajectories to be integrated into a longitudinal modeling scheme. Regression is also essential in estimating continuous trajectories, allowing sampling at arbitrary time points for age matching, or matching based on disease progression or other clinical scores. Regression can also be an used for enforcing time consistency across observations, which is an essential filtering routine for measurements with significant noise [1, 2].

The most widely adopted image regression method is based on a geodesic path on the manifold of diffeomorphisms [3, 4]. While linear in the space of diffeomorphisms, the model is nonetheless capable of capturing large nonlinear changes while guaranteeing desirable anatomical properties such as topological consistency and invertibility. However, it is generally hypothesized that geodesic models are only accurate over short time intervals. The geodesic model may not be a good approximation of highly accelerated and saturating growth characterized by early childhood development from birth, and is not capable of modeling more complex non-monotonic changes, as observed in cardiac imaging for example. In this case, more flexible or higher order models are needed when the geodesic constraint is too restrictive.

Higher order models have also been proposed, such as the extension of kernel regression to images [5], which has the flexibility to model arbitrary image change over time. However, since kernel regression is based on weighted averaging, a relatively dense sampling across time is required, as there is no underlying growth model to inform trajectories between observations. This makes kernel regression more suited to large cross-sectional populations and limits its applicability for subject-specific modeling with sparse sampling over time. Another promising higher order regression model proposed extending splines to the manifold of diffeomorphisms [6]. Results demonstrate that the model captures non-monotonic shape change, but is limited to capturing one inflection and therefore cannot model generic trajectories. The method further assumes a fixed initial image, and experimentation appears limited to 2D slices.

In this paper, we develop a flexible nonparametric diffeomorphic image regression model. The main idea is to parameterize flows of diffeomorphisms by acceleration rather than velocity, first introduced in the context of shape regression in [7]. As a consequence, image trajectories are defined to be twice differentiable by the relationship between acceleration, velocity, and position from classical mechanics, guaranteeing smooth image evolution. Following the work of [8], we adopt a sparse parameterization of diffeomorphisms based on a set of control points in the ambient space, which decouples deformation vectors from the voxel grid of the image. In addition to extending [7] to images, we further improve the model in several ways, including estimation of the initial image, estimation of the optimal location of the control points, and we introduce a new regularity term on initial velocity. We show that in a discretized setting the acceleration controlled model can be defined by a finite number of parameters, and we present an efficient solution to model estimation based on a regularized least-squares criterion. Finally, we validate our model on synthetic 2D images as well as a dynamic sequence of 3D CT of the cardiac cycle.

2. METHODOLOGY

We define an acceleration field at location x and time t as

$$a_i\left(x,t\right)=\sum _{i=1}^{N_C}{K}^V\left(x,c_i\left(t\right)\right){\alpha }_i\left(t\right)$$ (1)

where a sparse set of N_C control points c_i(t) carry impulse vectors α_i(t) and smooth kernel operator K induces the structure of the reproducing kernel Hilbert space V (K is a Gaussian in practice with standard deviation ${\sigma }_V^2$). The relationship between impulse and acceleration is analogous to the relationship between momenta and velocity in the large deformation diffeomorphic framework [9]. A flow of diffeomorphisms of the ambient space is parameterized by the time varying acceleration fields defined by the second order ODE

$$\ddot{\varphi }\left(x\left(t\right),t\right)=a\left(x\left(t\right),t\right)$$ (2)

with initial velocity v₀ = v(t₀) and initial position x₀ = x(t₀). Solving Eq. 2 generates a flow of diffeomorphisms starting from identity ϕ(0) = Id, which carries a particle at initial position x(t₀) to x(T). Given an initial distribution of control points c_i(0), the full trajectory of the control points can be computed by solving Eq. 2, giving the continuous path c_i(t). Likewise, the physical coordinates of image voxels also evolve according to Eq. 2, starting from an initial image configuration I₀. For notational convenience, let α(t), v₀, and c₀ be the concatenation of the α_i(t)’s, v_i(0)’s, and c_i(0)’s.

Given a set of image observations $I_{t_i}$ in time range t₀ to T, model estimation consists of determining time varying impulse α(t), initial velocity v₀, initial control point positions c₀, and initial image I₀ which minimizes

$$E\left(\alpha \left(t\right),v_0,c_0,I_0\right)=\sum _{i=1}^{N}d{\left({\varphi }_{t_i}\circ I_0,I_{t_i}\right)}^2+{\gamma }_A{\int }_{t_0}^T{\Vert a\left(t\right)\Vert }_V^2+{\gamma }_{v_0}{\Vert v_0\Vert }^2$$ (3)

where d is a distance metric between images, normalized cross-correlation in our implementation, γ_A weights the regularity of the time-varying acceleration, and ${\gamma }_{v_0}$ weights the regularity of initial velocity. Other reasonable choices for d are root mean squared error or mutual information.

The proposed model is nonparametric given that there are essentially an infinite number of parameters, since we must solve for a continuous function of time α(t). However, in practice we discretize time into N_t time-steps, resulting in N_C ∗ N_t impulse vectors to be estimated. We implement a gradient descent scheme; given current estimates of model parameters, each iteration goes as follows. First, solve equation Eq. 2 to obtain the continuous trajectory of control points c(t). Next, time-varying acceleration vectors can be computed at physical image coordinates by Eq. 1, and the trajectory of image coordinates can be computed by solving Eq. 2. A continuous image sequence can be constructed by interpolation in the initial image I₀, which can be used to compute residuals with respect to d at time points corresponding to observations $I_{t_i}$. All gradients are computed using autograd in PyTorch [10]. For solving the second order ODE Eq. 2, we use the velocity Verlet algorithm given initial velocity v₀, which is a 2nd order leapfrog integration scheme. Our implementation is available at https://github.com/jamesfishbaugh/acceleration-diffeos.

3. EXPERIMENTAL VALIDATION

Synthetic Bull’s-eye:

We first validate our image regression model on a 2D synthetic image sequence of an evolving bull’s-eye. The outer ring first decreases in size linearly, and then linearly increases to its original size. The inner circle grows exponentially, and then decreases exponentially to its original size. The sequence is perfectly symmetric, i.e. t₁ = t₅ and t₂ = t₄. The images are blurred with a small Gaussian kernel to create smooth edge transitions. The synthetic sequence is shown in the top row of Fig. 1.

Fig. 1.

Top) Synthetic bull’s-eye image sequence. Middle) Images estimated from geodesic regression. Bottom) Images estimated with our nonparametric image regression model.

We estimate a geodesic model using deformetrica [11] as well as a model using our proposed acceleration controlled method. For both methods, we choose σ_V = 10 pixels and choose regularity parameters to normalize the contribution of the data-matching term between methods. The geodesic model is unable to capture non-monotonic shape change and is therefore a poor fit, with an R² = ‒0.42 indicating that a simple voxelwise average over time would be a better fit. In contrast, our nonparametric model has the flexibility to match the observations, achieving an R² = 0.99. Indeed, with synthetic noise free observations our method tends towards interpolation. In practice, this has important implications regarding over-fitting of noisy observations, which we will discuss in the conclusion.

4D Cardiac CT:

Next, we explore the ability of our model to capture complex shape change and motion during the cardiac cycle. We obtain a dynamic 4D cardiac CT sequence from http://www.osirix-viewer.com image library (MAGIX), which contains ten 3D volumetric images captured during the cardiac cycle. For memory and computational efficiency, we crop to a smaller region, but we do not perform any preprocessing such as motion correction, noise reduction, or intensity normalization. The top row of Fig. 2 shows several axial slices of the cardiac sequence.

Fig. 2.

Top) Several frames of axial slices from 3D CT of the cardiac cycle. Estimated dynamic heart sequence by geodesic (middle) and acceleration controlled (bottom) regression. The acceleration controlled model captures complex non-monotonic change while the geodesic model shows relatively little change in order to stay close to the average. It is highly encouraged to view animations of geodesic (https://goo.gl/Y7W53K) and acceleration controlled (https://goo.gl/ZHFSJy) models for a more intuitive understanding of the results.

As with the synthetic example, we use a common value of σ_V = 12 mm for a geodesic model as well as our acceleration controlled model, and choose regularity weights to allow for a fair comparison between methods. Both models are estimated fully in 3D, though we only show axial slices here for visualization purposes. The middle row of Fig. 2 shows the trend estimated by geodesic regression. Overall, the model contains very low energy deformation, as the best the geodesic model can do to minimize sum of square error is to stay close to the average. The bottom row shows the estimated acceleration controlled model, which captures complex patterns of contraction and expansion of internal structures of the heart. The differences between models are difficult to convey with static images, it is recommended to zoom the figure and use the grid overlay as a visual guide which helps inform the magnitude of structural change. Readers are highly encouraged to view animations of geodesic (https://goo.gl/Y7W53K) and acceleration controlled (https://goo.gl/ZHFSJy) models for a more intuitive understanding of the results.

Fig. 3 shows the typical error in reconstruction for a couple of time points in the form of difference images. All difference images are scaled to the same intensity values (and contrast artificially enhanced for visualization purposes), with bright areas denoting the highest disagreement with the ground truth observation. In general, the acceleration controlled model matches the observations more accurately over the time range, which is most notable around the exterior of the heart and at the edges of the internal heart structures.

Fig. 3.

Difference images between ground truth observations and images estimated by geodesic (top) and acceleration controlled regression (bottom).

4. CONCLUSION

Geodesic regression models have become ubiquitous in the analysis of image time-series. However, when the observation time window is large or shape change is non-monotonic, the geodesic constraint may be too restrictive to accurately model the underlying structural change. In this paper, we propose a higher order nonparametric image regression model based on parameterizing diffeomorphic flow by acceleration, which defines growth trajectories by the integration of a second order ODE as in classical mechanics. As a consequence, our model is completely flexible to capture complex non-monotonic change patterns while still constrained to diffeomorphic flow and temporally smooth evolution. However, due to our model’s flexibility, avoiding overfitting and model selection becomes quite challenging as one can increase the model fit by either decreasing the weights on regularity, or by lowering the convergence criteria of gradient descent. There is some preliminary work on model selection in the case of shape observations [12], but parameter selection and model selection remain open issues. Finally, we showed on synthetic and real 3D medical images that our acceleration controlled model accurately captures complex structural changes.