Spatially Adaptive Log-Euclidean Polyaffine Registration Based on Sparse Matches

Abstract

Log-euclidean polyaffine transforms have recently been introduced to characterize the local affine behavior of the deformation in principal anatomical structures. The elegant mathematical framework makes them a powerful tool for image registration. However, their application is limited to large structures since they require the pre-definition of affine regions. This paper extends the polyaffine registration to adaptively fit a log-euclidean polyaffine transform that captures deformations at smaller scales. The approach is based on the sparse selection of matching points in the images and the formulation of the problem as an expectation maximization iterative closest point problem. The efficiency of the algorithm is shown through experiments on inter-subject registration of brain MRI between a healthy subject and patients with multiple sclerosis.

1 Introduction

Medical image registration is used in a variety of applications, from atlas construction to intraoperative navigation. The goal is to find a transform T that maps an image I onto another image J. A valuable property of the transform is diffeomorphism which guarantees invertibility and topology preservation [9].

Different models can be chosen for T [4]. Dense field models, as returned by diffeomorphic demons [9], are very flexible. However, models with fewer parameters are typically more robust. Log-euclidean polyaffine transforms (LEPT) [2,6] are compact and diffeomorphic transforms. They are built by composing affine transforms in the log-domain, and have been successfully used to register predefined anatomical structures [6].

The nature of tissues suggests that higher order deformations could also be modeled by LEPT with more degrees of freedom. However, predefining affine regions at a lower scale would be too cumbersome or impossible.

This paper introduces a registration method that adaptively fits a LEPT based on matching points sparsely selected in the image. The main contribution is the extension of the expectation-maximization iterative closest point (EM-ICP [7]) to use LEPT. This scheme accounts for matching ambiguities due to regularities in the image and naturally integrates regularization as a prior on T.

The rest of this paper is organized as follows. Section 2 introduces the elements of the algorithm. Section 3 shows results on brain inter-subject registration. Section 4 concludes and presents some directions for future work.

2 Methods

The proposed method, summarized in Table 1, aims at optimizing the parameters of a LEPT to map image J onto I. The location of the anchors of the affine components are first defined. The parameters of the affine components are then estimated in a multi-scale approach. At each scale, corresponding points are selected and their prior matching probabilities are recorded. The optimization then alternates between updating the posterior matching probabilities and optimizing the transform parameters. The image structure tensor is used in the definition of these probabilities to account for matching ambiguities.

Table 1.

Summary of the proposed method

1:	Define the anchor locations ak in I.
2:	for r = 0 to R − 1
3:	Compute the structure tensor 𝒮 of image I at scale r.
4:	Compute the LEPT weights w(x) at scale level r.
5:	Select the K best matches C = {(si;mij, πij)} between I and T (r−1) ○ J.
6:	for i = 1 to Nit do
7:	E-Step Update the probabilities: C (r) ← Update(C (r−1); T (r−1)).
8:	M-Step Optimize the affine corrections δL (r) ← Optimize(T (r); C (r)).
9:	Composition of the transform: $\text{exp}(L_k^{(n)})\leftarrow \text{exp}(L_k^{(r-1)})\text{exp}(\delta L_k^{(r)})$.
10:	Interpolation of the transformation: T (n) ← Interpolate(L (n), w(x)).
11:	end for
12:	end for

2.1 Log-Euclidean Polyaffine Transforms

Log-euclidean polyaffine transforms (LEPT) are defined as a weighted composition of N affine transforms in the logarithm domain [2]. In other words, under the action of an LEPT, the point x is transformed to x + T(x) with

$$T(\mathit{x})=\text{exp}\left(\sum _{k=1}^K{w}_k(\mathit{x})L_k\tilde{\mathit{x}}\right),$$ (1)

where exp(.) is the exponential-map, L_k ∈ ℝ^3×4 is the principal logarithm of an affine matrix in homogenous coordinates (with the last line ignored), w_k(x) are the weights which depend on the anchor location a_k, and x̃ = [x, 1]^T. LEPT have remarkable properties. They are invertible and their inverse is a LEPT [2].

In this paper, the location of the anchors are determined a priori based on the local intensity heterogeneity of the image. More precisely, a measure of heterogeneity h_I(x) is computed at every x (see Sec. 2.3). The anchors are defined as the K-means centroids of the point cloud spanned by all x s.t. h_I(x) > h̄_I(x), where h̄_I(x) is a sliding mean of heterogeneity. Hence, the anchors will tend to concentrate more in areas with high local contrast. Given K anchors a_k, we want to optimize the 12K parameters corresponding to L_k.

2.2 Block Matching

Block matching is used to establish a dense correspondence between points in image I and J as in [6]. The correlation coefficient is used as the similarity measure between blocks. This coefficient is invariant under linear transformations of the intensities. Let B_s (resp. B_m) be the block centered at s_i (resp. m_j) in image I (resp. J), the correlation coefficient is:

$$\rho ({\mathit{s}}_i,{\mathit{m}}_j)=\frac{\text{Cov}(B_s,B_m)}{\sqrt{\text{Var}(B_s)\text{Var}(B_m)}}.$$

For each block in I, the correlation coefficient ρ_max(s_i) of its best match in J is compared to the mean of all other correlation coefficients for this block, ρ̄(s_i). The N points in I with the highest ratio ρ_max/ρ̄ are recorded. The prior matching probabilities are given by a normal distribution of the mean squared error (MSE_n = 2−2ρ) between normalized blocks of zero mean and unit variance:

$${\pi }_{ij}=\frac{1}{\sqrt{2\pi {\sigma }_e^2}}\text{exp}(-\frac{2-2\rho ({\mathit{s}}_i,{\mathit{m}}_j)}{2{\sigma }_e^2}).$$

Block matching is not robust to noise and suffers from the aperture problem. To deal with this issue, M matches in J are recorded for each of the N selected points in I, and we will choose N ≫ K. The remaining of this paper explains how the LEPT can be robustly estimated based on these K × M matching pairs.

2.3 Image Structure Tensor

The location of the matches can be ambiguous due to the regularity of the intensity profile around the point. Line-like profiles give rise to first-order ambiguity and surface-like profiles give rise to second-order ambiguity. This problem can be addressed by anisotropically weighting the error with the structure tensor [5].

The structure tensor at x_k, S(x_k) ∈ ℝ³ is defined as the autocorrelation of the intensity gradient ∇I(x) in a neighborhood Ω_k:

$$S({\mathit{x}}_k)=E_{{\Omega }_k}\{\mathbf{\nabla }I(\mathit{x})\mathbf{\nabla }I{(\mathit{x})}^T\}=\frac{1}{\left|{\Omega }_k\right|}\sum _{\mathit{x}\in {\Omega }_k}\mathbf{\nabla }I(\mathit{x})\mathbf{\nabla }I{(\mathit{x})}^T.$$

The normalized structure tensor, 𝒮(x) = S(x)/∥S(x)∥₂, is a positive semidefinite matrix whose maximum eigenvalue is 1 with a corresponding eigenvector maximally aligned with the gradient in Ω_k.

The norm of the weighted error, e′ = 𝒮e, is ∥e′∥² = e^T 𝒮^T 𝒮e and is more affected by the component of e parallel to the gradient of the image. Consequently, matching ambiguities raised by regular structures have less impact on the overall error than errors made in a direction aligned with the image gradient.

Besides weighting the errors, the structure tensor is also used to define anchor locations. Indeed, its highest eigenvalue λ₃ is significantly higher than zero only for heterogenous areas, making it natural to define h_I(x) = λ₃(x).

2.4 Transformation Estimation

Given the set C of matching pairs (s_i, m_j) at each iteration, with prior probabilities π_ij, we propose to estimate the transform T by maximizing the joint log-likelihood of C and T [7,1]:

$$T^{*}=\underset{T}{\text{arg max}}E\{\text{log}P(C,T)\}=\underset{T}{\text{arg max}}E\{\text{log}P(C|T)\}+E\{\text{log}P(T)\}.$$ (2)

The first term of the likelihood tends to honor the correspondences established by block matching. The second term is a prior on T, which does not depends on C. This term favors, a priori, some transforms over the others, based on intrinsic properties of the transforms only. This is a statistical interpretation of the regularization energy used by many authors [6,9,4]. Here, we will define the prior as

$$\text{log}P(T)=-{\lambda }^2\sum _{k,l}{a}_{k,l}\parallel L_k-L_l{\parallel }^2+\mathrm{c}\mathrm{s}\mathrm{t},$$

where ∥.∥² is the Frobenius norm and $a_{k,l}={\sum }_{\mathit{x}}{w}_k(\mathit{x})w_l(\mathit{x})(\frac{1}{{\sum }_{\mathit{x}}{w}_k(\mathit{x})}+\frac{1}{{\sum }_{\mathit{x}}{w}_l(\mathit{x})})$ is an overlapping coefficient. This expression states that transforms are more likely if nearby affine components (affine components that share a common affecting area) are close to each other. Interestingly, this term is equivalent to the regularization energy defined in [6].

An efficient method to optimize (2) is EM-ICP [7]. This algorithm consists in alternatively optimizing the criterion for C considering T fixed (E-step) and for T considering C fixed (M-step). Provided that enough matching pairs are selected, EM-ICP is very robust to noise, hence being well suited to cope with the block matching issues. The E-step simply results in computing the matching probabilities:

$$\overline{{(C)}_{ij}}=\frac{{\pi }_{ij}p({\mathit{s}}_i|{\mathit{m}}_j,T)}{{\sum }_k{\pi }_{ik}p({\mathit{s}}_i|{\mathit{m}}_k,T)},$$ (3)

where the expression given in [7] is now extended to account for the matching ambiguities:

$$p({\mathit{s}}_i|{\mathit{m}}_j,T)=\text{exp}(-\parallel 𝒮({\mathit{m}}_j)(T*{\mathit{s}}_i-{\mathit{m}}_j){\parallel }^2/2{\sigma }_n^2).$$

.

The M-step then consists in optimizing the parameters of T:

$$T^{*}=\underset{T}{\text{arg max}}\sum _{i=1}^{n_s}\sum _{j=1}^M\overline{{(C)}_{ij}}\text{log}p({\mathit{s}}_i|{\mathit{m}}_j,T)-{\lambda }^2{\sum _{k,l=1}^K{a}_{k,l}\parallel L_k-L_l\parallel }^2.$$ (4)

Unlike affine transforms, this equation does not have an obvious solution for the parameters L_k, due to the exponential map of (1). Therefore, we propose a first order approximation in which T(x) is approximated by:

$$T(\mathit{x})\approx 𝕀+\sum _{k=1}^N{w}_k(\mathit{x})L_k\tilde{\mathit{x}}.$$

In that case, optimizing (4) amounts to the quadratic programming (QP):

$$l^{*}=\underset{l}{\text{arg min}}{(Hl-D)}^{T}S(Hl-D)+{\lambda }^2{l}^{T}Al,$$ (5)

where:

• l ∈ ℝ^12N is the vector of the elements of L_k taken row-wise and concatenated,

• H ∈ ℝ ^{3n_s × 12N} is the interpolation matrix. Each 3 × 12 block [H]_ij corresponds to point s_i = (x_i, y_i, z_i) and the j^th anchor a_j and is equal to I₃ ⊗ (w_j(s_i)s̃_i)^T, where I₃ is the 3 × 3 identity matrix and ⊗ stands for the Kronecker product,

• D ∈ ℝ^3n_s is the vector obtained by taking, for each s_i, the barycenter of its matches m̅_i weighted by $\overline{{(C)}_{ij}}$ (the equivalence between criterion (4) and the use of barycenters is justified in [7].),

• S ∈ ℝ^{3n_s × 3n_s} is the block diagonal matrix of the structure tensor. Each 3 × 3 block on the diagonal corresponds to the structure tensor at point s_i,

• A ∈ ℝ^{12K × 12K} is the prior matrix with (A)_k≠l = −a_k,l and (A)_kk = ∑_l a_k,l.

In practice, the weights w_i and overlapping coefficients a_k,l are thresholded out, so that H, S and A are sparse. Taking the derivative of (5) w.r.t. l and setting it to 0 yields the linear system:

$$(H^{T}SH+{\lambda }^{2}A)l=H^{T}SD.$$ (6)

Interestingly, the regularization term is optimized simultaneously with the similarity term, unlike [6] where an ad-hoc one step gradient descent is performed on the regularization energy. Alternating the estimation of matching probabilities (3) and the estimation of parameters (6) until convergence results in a globally optimal LEPT that best fits the observations of the block matching.

2.5 Weights: the Kriging Estimator

In [6,2], the weights w_k(x) are simply a normalized Gaussian function of the distance between the point and the anchor. Here we use the Kriging estimator (KE) to define these weights, as in [8]. The KE has the advantage of adapting the weights to the spatial distribution of anchors in a statistically sound way.

Let us interpret LEPT as random fields of matrix logarithms L(x) for which L_k are observations at locations a_k. In this interpretation, the weights are the coefficients of a linear estimator of L at x. KE is a best linear unbiased estimator for the field L(x) given a (presumably valid) model of its spatial correlation encoded in a variogram: γ(x, y)=E {|L(x) − L(y)|²/2}. Given (γ_a(x))_i=γ(a_i, x), the weights w(x)=(w₁(x)…w_K(x))^T are obtained by solving the linear system:

$$[\begin{array}{c}\mathit{w}(\mathit{x})\\ \mu \end{array}]=\Gamma {(\mathit{x})}^{-1}{\gamma }_a(\mathit{x}),\text{with}\Gamma (\mathit{x})=\left[\begin{array}{cc}(\gamma ({\mathit{a}}_i,{\mathit{a}}_j))& \mathbf{1}\\ {\mathbf{1}}^T& 0\end{array}\right]\in {ℝ}^{(K+1)\times (K+1)}$$

where μ is the Lagrange multiplier ensuring unbiasedness of the estimate. In this paper an exponential isotropic variogram is used: γ(x, y) = 1 − e^{−∥x−y∥/t}.

3 Experiments and Results

The method was tested on a dataset of ten brain T1-MRI (resolution: 256×256×l76). A synthetic experiment was first carried out. Inter-subject registration was then performed. Unless otherwise mentioned, parameter values are: K=500, N=(20+5r)³ at scale r, R=5, M=20, N_it=5, t=40, λ²=0.3, σ_e=σ_n=5. For block matching, blocks of 5³ voxels and searching region of 9³ voxels are used.

3.1 Synthetic Experiments

The knowledge of a ground true deformation helps understanding how the algorithm behaves. A synthetic field $T(\mathit{x})=\text{sin}(\frac{\pi \mathit{x}}{50})\text{cos}(\frac{\pi y}{50}){(3,3,3)}^T$ was applied to the image. Both adaptive LEPT and diffeomorphic demons accurately recovered T within the brain volume, with mean absolute error of 0.21 and 0.12 respectively (Fig. 1) and were invertible (min. jacobian of 0.67 and 0.65 respectively).

Fig. 1.

(Left) LEPT (displayed as checkpoints) are able to accurately recover the synthetic field (deformed grid) within the brain volume (Middle) Influence of N on the recovering accuracy (the x — axis is logarithmic) (Right) Influence of K on the recovering accuracy.

The registration was then performed for different number N of sparse matches and K of anchors. The evolution of the accuracy with N (Fig. 1 (Middle)) suggests that the method is not sensitive to the number of selected matches as long as this number is high compared to K. For lower N, the affine transforms can no more be robustly estimated and the performances collapse. The evolution of the accuracy with K (Fig. 1 (Right)) tells us that no loss of accuracy is observed between K = 700 and K = 300, suggesting that the number of selected matches is the bottleneck here. For lower K, the performance decreases, but the method does not diverge, since all affine components can still be robustly estimated.

3.2 Inter-Subject Registration

The method was then applied to register ten multiple-sclerosis (MS) patients to one healthy subject. Each brain was skull stripped and affinely registered to the subject. Images were manually segmented by an expert in a validated protocol, providing an external validation criterion for the registration. More precisely, the Dice’s coefficient between the subject label image and the label image of the patients after alignment were computed for each tissue (Fig. 2).

Fig. 2.

Dice’s coefficient with a 95% CI. Adaptive LEPT aligns structures better than diffeomorphic demons. The difference is especially significant for low contrast structures such as the putamen and the insluae.

As a whole, adaptive LEPT aligns tissues better than diffeomorphic demons. For high contrast regions (e.g.. lateral ventricles), both methods achieve comparable results. However, diffeomorphic demons tend to excessively favor these regions at the cost of a very poor alignment of low contrast structures (e.g. putamen). For these structures, the difference of performance between the two algorithms is strongly significant.

Fig. 3 depicts the aligned contours for one slice, along with the deformed grid. Again, the Jacobian of the field never fell under 0 (min. jacobian of 0.12). However, a strong pinching effect appear in the lobe regions. These regions are typically subject specific and, while good pairs of local matches can be detected, they may be misleading in the transform estimation. Finally, note that the non-rigid deformation of the structures (as seen e.g. by the bending of the lateral ventricles) would not be captured if a single affine region was defined for them as in the previous LEPT registration framework.

Fig. 3.

(Left to right) Subject with labeled tissues, projected labels of the patient’s tissues after diffeomorphic demons registration and after adaptive LEPT registration, patient image with the deformed grid overlaid.

4 Conclusion and Future Work

This paper introduces a registration algorithm that adaptively fits a LEPT based on a set of sparse matches. Results on inter-subject registration show that LEPT are able to capture the local affine deformations occurring at small scales. In a future work, we want to investigate an adaptive way of incrementally defining the anchor locations such as it is done in [3].