NON-RIGID REGISTRATION GUIDED BY LANDMARKS AND LEARNING

Abstract

Registration methods frequently rely on prior information in order to generate anatomical meaningful transformations between medical scans. In this paper, we propose a novel intensity based non-rigid registration framework, which is guided by landmarks and a regularizer based on Principle Component Analysis (PCA). Unlike existing methods in this domain, the computational complexity of our approach reduces with the number of landmarks. Furthermore, our PCA is invariant to translations. The additional regularizer is based on the outcome of this PCA. We register a skull CT scan to MR scans aquired by a MR/PET hybrid scanner. This aligned CT scan can then be used to gain an attenuation map for PET reconstruction. As a result we have a Dice coefficient for bone areas at 0.71 and a Dice coefficient for bone and soft issue areas at 0.97.

1. INTRODUCTION

The registration of medical scans often relies on prior information to produce anatomical meaningful transformations [1]. Techniques, such as [2, 3] constrain the registration to a manifold. To capture anatomical meaningful deformations, the manifold is generally trained on a large set of scans. In this paper, we develop a non-rigid registration algorithm based on technology only requiring small data sets for training.

Our registration approach is guided by landmarks as well as PCA on deformation fields. Non-rigid landmark based methods, such as Fischer et al. [4], use the Lagrange multipliers to integrate the landmarks into the energy function capturing the registration problem. Hereby each Lagrange multiplier expands the dimensionality. This is inconsistent to the intuitive idea that landmarks should simplify the registration. We instead propose to fixate the deformation maps at those locations treating the correspondences as Dirichlet boundary conditions. Thus, the computational burden reduces with an increasing number of landmarks.

The deformation of the remaining image space is then defined by the minimum of an energy functional consisting of an image similarity term and two regularizers. The first regularizer enforces the smoothness of the deformation field, the second regularizer is based on the outcome of a PCA on the deformation maps. Compared to manifold learning techniques, PCA based non-rigid registration methods, such as [5], require less training samples as PCA is based on the assumption that the distribution of the deformation fields is Gaussian. We thus do not constrain the resulting deformation to the space defined by the PCA model, such as in [5], but instead only use the PCA model to guide the registration. We furthermore reduce the variation to be captured by PCA by defining a model that is invariant to translations.

We use our algorithm to register a skull CT scan to MR scans which were acquired by a MR/PET hybrid scanner. MR/PET hybrid scanners can help to detect tumors in an early stage and give information about the activity of the tumor, improve the treatment of patients in radiation therapy or assist medicines in differential diagnostics. Unlike for PET/CT hybrid scanners, where the attenuation map is constructed from the CT scan, the MR scan from a MR/PET hybrid scanner cannot be directly used for attenuation correction of PET due to the image inhomogeneity in MR scans. The skull CT scan is the average of various CT scans which we will call an atlas CT. The registration process of the atlas CT to the MR scan is the focus of our application.

2. METHODS

We now construct a non-rigid registration algorithm to map each voxel x ∈ Ω_M of the image domain Ω_M ⊂ ℝ^d of the moving image M to a voxel x − u(x) ∈ Ω_F in the image domain Ω_F ⊂ ℝ^d of the fixed image F with the transformation u(·). We compute the optimal mapping u^* by solving

$${\mathit{u}}^{\ast }:={\text{argmin}}_{\mathit{u}}\mathcal{D}(F,M,\mathit{u})+\alpha ·\mathcal{R}(\mathit{u})+\beta ·\mathcal{P}(\mathit{u})$$ (1)

where 𝒟(·) is the image matching term, ℛ(·) the regularizer enforcing smoothness on u, ℘(·) is the regularizer based on our PCA and α, β ∈ ℝ are weighting factors.

2.1. Landmarks

To follow the intuitive idea mentioned in the introduction, our framework now efficiently embeds the landmarks into the registration framework by treating the landmarks as Dirichlet boundary conditions and determining u^* for the remaining image domain. In other words, let Ω_L ⊂ Ω_M be the set of landmarks in the moving image and u_L(x) be the corresponding deformation mapping landmark x ∈ Ω_L to its corresponding landmark in Ω_F. We then solve (1) for Ω_M\Ω_L using the constraint that u^*(x) = u_L(x) for x ∈ Ω_L. As we cut off the landmarks, the remaining image domain has no longer rectangular shape. As consequence, the respective matrices are asymmetric now. We address this problem in Section 2.3 where we specify the implementation of our registration framework.

2.2. Regularization based on PCA

The definition of ℘(·) is based on our PCA of deformation fields which identifies the major modes of variation within a sampling set. Let 𝒰 be a Hilbertspace. A functional PCA decomposition finds mutually orthogonal modes v_i of normal length that optimize

$$\begin{array}{l}{\mathit{v}}_i^{\ast }(\mathrm{x}):={\text{argmax}}_{{\mathit{v}}_i}{\mathcal{E}}_{\mathit{PCA}}({\mathit{v}}_i)\\ ={\text{argmax}}_{{\mathit{v}}_i}\sum _{j=1}^m{\langle {\mathit{w}}_j-\overline{\mathit{w}},{\mathit{v}}_i\rangle }^2\end{array}$$ (2)

with $\overline{\mathit{w}}(\mathrm{x}):=\frac{1}{m}{\sum }_{i=1}^m{\mathit{w}}_i(x)$, w_i are the m sampling functions, i.e. the deformation maps of the training set, and 〈x, y〉 := x^Ty is the inner product induced by the Hilbertspace 𝒰. Accordingly, the values of ℘(u) relate to the residual of u with respect to the PCA space:

$$\mathcal{P}(\mathit{u}):={\Vert \mathit{u}-\left(\overline{\mathit{w}}+\sum _{i=1}^m{\mathit{v}}_i\langle {\mathit{v}}_i^{\ast },\mathit{u}-\overline{\mathit{w}}\rangle \right)\Vert }_{\mathcal{U}}$$ (3)

where ||·||_𝒰 is the L²-Norm.

In the remainder of this Section, we determine the major axis v_i fulfilling (2) by incorporating the constraint by a Langrangian multiplier ensuring the normal length.

$${\mathit{v}}_i^{\ast }={\text{argmax}}_{{\mathit{v}}_i}\sum _{j=1}^m{\langle {\mathit{w}}_j-\overline{\mathit{w}},{\mathit{v}}_i\rangle }^2-{\lambda }_i(\langle {\mathit{v}}_i,{\mathit{v}}_i\rangle -1)$$ (4)

From this, we calculate the Gâteaux derivative dℰ_PCA(v_i; η) at v_i in the direction η ∈ 𝒰 to determine the extremal points what results in

$$\langle \mathit{\eta },\sum _{j=1}^m({\mathit{w}}_j-\overline{\mathit{w}})\langle {\mathit{w}}_j-\overline{\mathit{w}},{\mathit{v}}_i\rangle \rangle =\langle \mathit{\eta },{\lambda }_i{\mathit{v}}_i(\mathrm{x})\rangle$$ (5)

For details of the calculation of the Gâteaux derivative, see [6].

Due to the inconsistence of rigid registration, we want to make our model robust to translations of rigid alignment. Therefore d constant global rigid translation functions t_j with

$${\mathit{t}}_j(\mathrm{x})={(0,\dots ,0,\underset{\text{j-th}\text{entry}}{\underbrace{1}},0,\dots ,0)}^T\in {ℝ}^{d}j=1,\dots ,d$$ (6)

are introduced. Our translation invariant sampling functions $\stackrel{\sim }{{\mathit{w}}_i}$ are now defined as :

$${\stackrel{\sim }{\mathit{w}}}_i(\mathrm{x})={\mathit{w}}_i(\mathrm{x})-\sum _{j=1}^d{\mathit{t}}_j(\mathrm{x})\langle {\mathit{t}}_j,{\mathit{w}}_i\rangle i=1,\dots ,n$$ (7)

These new sampling functions w̃_i are orthogonal to t_j. We recompute w̄ and ${\mathit{v}}_i^{\ast }$ from (2) by replacing w_i with w̃_i. The modes ${\mathit{v}}_k^{\ast }$ are orthogonal to t_j as each ${\mathit{v}}_k^{\ast }$ is a linear combination of w̃_i. This means, that t_j are orthonormal to our PCA basis and we can augment this basis with the additional modes ${\mathit{v}}_{m+j}^{\ast }={\mathit{t}}_j$. Hence, ℘(u) is translation invariant.

We also generate a translation invariant PCA model based on the derivatives of the deformations. The first derivative of a deformation is already translation invariant and the Laplacian Δu of a deformation field is invariant to global affine transformations. This Laplacian is the basis for the curvature PCA model. The mean w̄ and the Eigenmodes v_i are computed on the Laplacian of sampling functions Δw by similar proceeding as described above with

$${\mathcal{P}}_{\mathrm{\Delta }}(\mathit{u})={\Vert \mathrm{\Delta }\mathit{u}(\mathrm{x})-\left(\mathrm{\Delta }\overline{\mathit{w}}+\sum _{i=1}^m{\mathit{v}}_i(\mathrm{x})\langle {\mathit{v}}_i,\mathrm{\Delta }\mathit{u}-\mathrm{\Delta }\overline{\mathit{w}}\rangle \right)\Vert }_{\mathcal{U}}^2$$ (8)

The registration in Section 3 is performed using ℘ or ℘_Δ as additional regularizer of (1).

2.3. Numerical Solution

Similar to [4], we determine the solution to (1) by solving

$${\nabla }_{\mathit{u}}(\mathcal{D}(F,M,\mathit{u})+\alpha \mathcal{R}(\mathit{u})+\beta \mathcal{P}(\mathit{u}))=0$$ (9)

where ∇_u is the gradient with respect to u. We solve this equation on a discrete image grid where u is a vector capturing the displacement of each voxel in Ω_M. Furthermore, we now assume without loss of generality that w̄(x) = 0:

$${\nabla }_{\mathit{u}}\mathcal{P}(\mathit{u})=2\left(\mathit{u}-\sum _{i=1}^m{\mathit{v}}_i\langle {\mathit{v}}_i,\mathit{u}\rangle \right)-2\stackrel{\sim }{\mathit{w}}.$$ (10)

By simplifying ℛ(u) to an inner product u^TAu, the above equation simplifies to

$${\nabla }_{\mathit{u}}\mathcal{D}(F,M,\mathit{u})+\alpha \mathit{Au}+\beta {\nabla }_{\mathit{u}}\mathcal{P}(\mathit{u})=0$$ (11)

In practice, we will solve this problem with the semi-implicit gradient descent which updates u according to solving the linear equation system

$$(\mathit{I}+s·\alpha ·\mathit{A})·{\mathit{u}}^{(i+1)}={\mathit{u}}^{(i)}s·{\nabla }_{\mathit{u}}\left(\mathcal{D}(F,M,{\mathit{u}}^{(i)})+\beta \mathcal{P}({\mathit{u}}^{(i)})\right)$$ (12)

with u⁽ⁱ⁾ being the mapping at the i-th iteration, I is the identity matrix and s is the step size. For more details see [6].

We determine the initial mapping via A·u⁽⁰⁾ = y, where y is a zero vector except for the entries corresponding to landmarks. There it has the values u_L. Thus u⁽⁰⁾ correctly aligns the landmarks between the scans and fulfills the smoothness constraints but ignores the image information. Alternatively, we fix u⁽ⁱ⁺¹⁾ at the landmarks and update u⁽ⁱ⁺¹⁾ at all other voxels in the image domain. We do so by simply setting the rows associated with landmarks in A and ∇_u (𝒟(F,M,u⁽ⁱ⁾) + ℘(u⁽ⁱ⁾)) to zero. Now, (I + s · α ·A) is asymmetric. We could either resymmetrize the matrices or apply methods not requiring symmetry like the stabilized bi-conjugate gradient method. The second kind of solution is employed in our experiments. The approach thus can guarantee proper alignment in the vicinity of the landmarks but is a little bit slower than updating the entire deformation field.

We outlined our numerical solution for the semi-implicit gradient descent (12) as it is easier and more clearly to illustrate the integration of Dirichlet boundary conditions into this scheme. Due to former analysis of optimization schemes, we implemented for our experiments the Newton scheme after Broyden, Fletcher, Goldfarb and Shanno [7]. The iteration step via Newton holds

$${\mathit{u}}^{(i+1)}={\mathit{u}}^{(i)}s{\left({\mathit{H}}_{\mathcal{D}(F,M,\mathit{u}(i))}+\alpha \mathit{A}+\beta {\mathit{H}}_{\mathcal{P}}\right)}^{-1}\left({\nabla }_{\mathit{u}}\mathcal{D}(F,M,\mathit{u}(i)+\alpha \mathit{A}{\mathit{u}}^{(i)}+\beta {\mathit{H}}_{\mathcal{P}}{\mathit{u}}^{(i)}\right)$$ (13)

where H_{𝒟(F,M,u(i))} is the Hessian of 𝒟(F,M,u(i)) and H_℘ is the Hessian of ℘. To avoid solving the large and densely filled Hessian H, we employ an approximation based on Sherman-Morrison-Woodbury [8]. We make use of mutual information [9] as distance measure 𝒟 in conjunction with the curvature regularizer ℛ [10].

3. EXPERIMENTS

We perform two different types of experiments. First we consider images of a PET/CT scanner and compare different kind of registration methods. Then we apply out body scans of a MR/PET hybrid scanner and align an atlas CT to the MR scan.

3.1. Influence of landmarks on a PET/CT registration

To illustrate the influence of landmarks, we visually compare the outcome of different registration approaches to align a CT to the corresponding PET scan (Fig. 1). There were 7 landmarks specified: the superior and inferior renal capsules of the kidneys (4 landmarks), the lower tip of the liver (1 landmark) and the diaphragm (2 landmarks). Although the non-rigid registration shown in picture (c) is not state-of-the-art, we just want to illustrate the influence of landmarks as only non-rigid registration with landmarks (d) does match the CT scan with the PET scan as expected.

Fig. 1.

(a) CT, (b) PET/CT rigid, (c) PET/CT non-rigid, (d) PET/CT non-rigid with landmarks

3.2. Atlas registration in a MR/PET application scenario

As the MR scan cannot be directly used for attenuation correction, an atlas CT is registered to a MR scan enabling attenuation correction of PET scans acquired in MR/PET hybrid scanners. The quality of this CT-MR registration is the focus of our experiment. We have 34 skull CT scans of 34 different patients and all in all 25 T1 weighted and 17 T2 weighted MR scans of those patients. For efficiency reasons we down-sample the data to an isotropic resolution of 1.95mm and a volume size of 128 × 128 × 71 to keep computational time down. Non-anatomical structures, such as the table, are masked and ignored. To determine the quality of the registered atlas CT, the CT scan of each patient is used as groundtruth employing the following four measures: the Root Mean Square Error in Hounsfield unit (HU) (RMSE), the Mean Absolute Error in HU (MAE), the Dice coefficient of thresholded bone areas (thresholded at 600HU) (BDICE) and the Dice coefficient of thresholded bone and soft issue areas (thresholded at −200HU) (STDICE).

As the employed Newton scheme [7] is a Quasi-Newton scheme, the inverse of the Hessian has to be calculated. This is either done exactly or with the approximation [8] as mentioned in Section 2.3. We compare the accuracy of our algorithm with respect to 5 different implementations : without PCA regularization (standard), with PCA regularization ℘(·) on the deformation map (PCA exact) as well as PCA regularization on the curvature ℘_Δ(·) (curv PCA exact), and finally both PCA implementations with approximated Hessian (PCA approx and curv PCA approx). The algorithms with additional PCA regularization were employed with several number of eigenmodes m to test their impact. In Table 1 the results are listed for m = 10. Our results were fairly insensitive for m > 10. The average E and standard deviation σ is summarized in Fig. 2.

Fig. 2.

Table of Results

All in all, PCA regularized implementations compared to the standard implementation clearly enhance the results. For example, the Dice coefficient for bone areas for the PCA approximate for T1 weighted images is 0,69, i.e. that the first 10 eigenmodes cover 69% of the bone areas. The corresponding Dice coefficient for bone and soft issue areas is 0.97, so the content of the remaining eigenmodes is irrelevant. As we achieve the coverage for bone and soft issue areas of 97% even with the approximate algorithm, we keep the computational work low by increasing simultaneously the robustness of the registration.

4. CONCLUSION

In this paper we have proposed a novel method how to integrate prior information to a non-rigid registration. The algorithm is guided by landmarks and the outcome of a PCA on deformation maps. Unlike previous approaches, we reduced the computational complexity with our method treating the landmarks as Dirichlet boundary conditions. In addition, we developed PCA model based regularizers that are invariant to global translations. By this means, we demonstrated on the one hand the great impact of non-rigid registration with landmarks and on the other hand we illustrated that our additional PCA regularizers help to increase the robustness of an atlas registration.