Mid-Space-Independent Deformable Image Registration

Abstract

Aligning images in a mid-space is a common approach to ensuring that deformable image registration is symmetric – that it does not depend on the arbitrary ordering of the input images. The results are, however, generally dependent on the mathematical definition of the mid-space. In particular, the set of possible solutions is typically restricted by the constraints that are enforced on the transformations to prevent the mid-space from drifting too far from the native image spaces. The use of an implicit atlas has been proposed as an approach to mid-space image registration. In this work, we show that when the atlas is aligned to each image in the native image space, the data term of implicit-atlas-based deformable registration is inherently independent of the mid-space. In addition, we show that the regularization term can be reformulated independently of the mid-space as well. We derive a new symmetric cost function that only depends on the transformation morphing the images to each other, rather than to the atlas. This eliminates the need for anti-drift constraints, thereby expanding the space of allowable deformations. We provide an implementation scheme for the proposed framework, and validate it through diffeomorphic registration experiments on brain magnetic resonance images.

1. Introduction

The computation of a set of dense spatial correspondences among images – a.k.a. image registration – is a central step in most population and longitudinal imaging studies. Linear transformation is often not sufficient to account for cross-subject anatomical variation or temporal changes in an individual anatomy, thereby making deformable image registration (Sotiras et al., 2013) a necessary part of most analysis pipelines. The importance of registration accuracy in neuroimaging is evident from the literature; for instance, inaccurate alignment has been shown to lead to incorrect diagnosis (Reuter et al., 2014) and ineffective radiotherapy (Castadot et al., 2008) of tumors, and the inability to detect early effects of Alzheimer’s disease (Cuingnet et al., 2011; Fischl et al., 2009).

In deformable registration, the choice of the reference space in which the images are compared affects the outcome, making the resulting deformation field dependent on this choice. Choosing the native space of one of the input images (say, the first image) as the reference breaks the symmetry of pairwise registration, meaning that reversing the order of the input images will produce different spatial correspondences. Such an inverse-inconsistency has been shown to be related to biased errors introduced into the estimation of Alzheimer’s disease effects (Fox et al., 2011; Hua et al., 2011; Thompson and Holland, 2011; Yushkevich et al., 2010), daily dose computation (Yang et al., 2008) and auto recontouring (Ye and Chen, 2009) in radiation therapy, the quantification of lesion evolution in multiple sclerosis (Cachier and Rey, 2000; Rey et al., 2002), and the measurement of longitudinal changes (Reuter et al., 2012). Local volume changes in the deformation field and discretization artifacts are two major contributors to registration asymmetry. Pairwise registration has been proposed to be symmetrized by minimizing the average of two cost functions, each using one input image as the reference space (Cachier and Rey, 2000; Christensen and Johnson, 2001; Tagare et al., 2009; Trouvé and Younes, 2000), which unfortunately results in the non-uniform integration of the image mismatch measure in the native spaces of the input image (Aganj et al., 2015b).

In a different approach to achieve symmetry in pairwise registration, both images are deformed and compared in an abstract reference space chosen to be “in between” the native spaces of the images, known as the mid-space (Ashburner and Ridgway, 2013; Avants and Gee, 2004; Beg and Khan, 2007; Chen and Ye, 2010; Joshi et al., 2004; Lorenzen et al., 2004; Lorenzen et al., 2006; Lorenzi et al., 2013; Noblet et al., 2008, 2012; Škrinjar et al., 2008; Yang et al., 2008; Ye and Chen, 2009). Since both images are treated equally, mid-space registration is invariant with respect to the ordering of the images. Such approaches essentially minimize their cost functions with respect to two transformations that take the two input images to the mid-space. However, without additional constraints, this increases the degrees of freedom of the problem twofold, compared to the end result of pairwise registration that is the one transformation taking one input image to the other. Furthermore, if the images are compared in the mid-space, the optimization algorithm is given the liberty to update the mid-space so as to decrease the cost function without necessarily changing the resulting image-to-image transformation. For example, the algorithm can shrink the regions with mismatching image intensities to make the deformed images look more similar in the mid-space, without necessarily making them more similar in their native spaces. To alleviate these issues, additional constraints are used prevent the mid-space from drifting away from the native spaces of the two images. These anti-drift constraints, which are different from those regularizing the transformations, define the mid-space. They typically either restrict the space of possible pair of transformations (resulting in fewer degrees of freedom), or penalize those values of the two transformations that move the mid-space away from the native spaces. The most common such constraints, proposed in the mid-space registration and atlas construction literature, are restrictions on the two transformations to have opposite displacement fields (Aljabar et al., 2008; Bhatia et al., 2004; Bouix et al., 2010; Fonov et al., 2011; Guimond et al., 2000; Miller et al., 1997; Noblet et al., 2012; Studholme and Cardenas, 2004; Yang et al., 2008) or velocity fields (Ashburner and Ridgway, 2013; Grenander and Miller, 1998; Lorenzi et al., 2013). In large deformation models, geodesic averaging of the deformations has also been proposed, which preserves the desired properties of the transformations (Avants and Gee, 2004; Joshi et al., 2004; Lorenzen et al., 2006). The anti-drift constraints, however, can have the side effect of restricting the final image-to-image transformation, thereby causing the exclusion of some legitimate results (see Section 2.1 for examples). Furthermore, the choice of these constraints may affect the results by biasing the registration algorithm towards favoring a particular set of transformations.

Unbiased atlas construction techniques can constitute mid-space registration, as the images are deformed to the atlas space (Ashburner and Ridgway, 2013; Hart et al., 2009; Joshi et al., 2004). In an atlas construction approach to image registration, the desired output is the deformation field, but not the auxiliary atlas. Consequently, one can analytically solve for the atlas in the cost function, leading to an implicit-atlas cost function that is minimized with respect to the image-to-atlas transformations. To that end, it was initially proposed to compare the deformed images to the atlas in the mid-space (Geng et al., 2009; Joshi et al., 2004). A better-justified generative model, however, progresses from the atlas to the images and compares the deformed atlas to the images in the native image spaces (Allassonnière et al., 2007; Ma et al., 2008; Sabuncu et al., 2009). Taking advantage of this native-space atlas construction resolves the issue of susceptibility to shrinkage-type problems, leading to a proper implicit-atlas cost function for mid-space registration (Ashburner and Ridgway, 2013). Nevertheless, the registration still remains a function of two transformations taking the images to a mathematically defined mid-space.

In this work, we derive the key fact that implicit-atlas registration has a data term that is inherently independent of the mid-space, and only depends on the overall image-to-image transformation. This implies that the individual image-to-atlas transformations are redundant and unnecessary to keep, and that anti-drift constraints are indeed not needed. We also show how to analytically solve the common Tikhonov regularization terms with respect to one of the image-to-atlas transformations. These lead us to a new cost function that, in contrast to the existing mid-space approaches, can be minimized directly with respect to the image-to-image transformation, with no anti-drift constraints. The proposed cost function is general and can be used with any deformation field parameterization, such as the displacement and the velocity fields.

This article extends our preliminary conference version (Aganj et al., 2015a). In particular, we propose a new regularization term in addition to the data term (Section 2), evaluate our method more comprehensively on 3D brain magnetic resonance images (Section 3), propose the extension of our framework to group-wise deformable registration ( Appendix A), and provide further details on the derivations and the implementation of the method ( Appendix B & Appendix C).

2. Methods

2.1. Mid-space based registration

We begin with a brief overview of mid-space based pairwise deformable registration. An extension of our framework to group-wise registration is suggested in Appendix A. Let I₁, I₂:Ω → ℝ be the two d-dimensional input images to be registered, where Ω ⊆ ℝ^d.¹ The goal of pairwise deformable registration is to compute the regular transformation T: Ω → Ω that makes overlapping regions of I₁ and I₂ ∘ T locally correspond to each other; a task that is often accomplished by minimizing a cost function with respect to T. In one popular approach, the image-to-image transformation T is parameterized as $\mathrm{T}={\mathrm{T}}_2\circ {\mathrm{T}}_1^{-1}$, where T₁ and T₂ deform I₁ and I₂ to a mid-space (see Section 1 for references). The deformed images may be compared in the mid-space by minimizing a data term such as the common sum of squared differences (SSD) term, ∫_Ω((I₁ ∘ T₁)(y) − (I₂ ∘ T₂)(y)² dy (Geng et al., 2009; Joshi et al., 2004).² The mid-space data term is by definition invariant with respect to the ordering of the images; i.e., swapping I₁ and I₂ will swap T₁ and T₂ in the produced set of transformations.

With no additional constraints, the dimensionality of the mid-space registration problem (solving for T₁ and T₂) is twice as large as the standard asymmetric problem (solving for T). Another drawback of this approach is that the mid-space can drift arbitrarily far away from the native spaces of the images due to large changes in T₁ and T₂, for instance through combination with a transformation S, as T₁ ∘ S and, T₂ ∘ S, which decreases the mid-space cost function without changing the final $\mathrm{T}=({\mathrm{T}}_2\circ \mathrm{S})\circ {({\mathrm{T}}_1\circ \mathrm{S})}^{-1}={\mathrm{T}}_2\circ \mathrm{S}\circ {\mathrm{S}}^{-1}\circ {\mathrm{T}}_1^{-1}={\mathrm{T}}_2\circ {\mathrm{T}}_1^{-1}$. An example of this phenomenon is the situation where the optimization algorithm updates T₁ and T₂ in order to shrink the regions where the deformed images do not match, resulting in a decrease in the mid-space cost function, without necessarily changing the end result, T. To avoid this issue, additional constraints are often employed to keep the mid-space “close” to the native image spaces. Such constraints reduce the degrees of freedom and to some extent prevent the mid-space drift, however, at the expense of limiting our ability to model all possible transformations T. Examples of such anti-drift constraints and their limitations are as follows:

• The transformations may be constrained to have zero-sum displacement fields, i.e. u₁(y) + u₂(y)=0 where u_i(y) ≔ T_i(y) − y, resulting in the constraint T₁(y) + T₂(y) = 2y (or similarly the alternative constraint ${\mathrm{T}}_1^{-1}(\mathrm{x})+{\mathrm{T}}_2^{-1}(\mathrm{x})=2\mathrm{x}$). Now let us consider the 2D case where the true transformation T is a 180° rotation about the origin, i.e. T(x) = −x. By definition, T₂(y) = (T ∘ T₁)(y), which leads to T₁(y) + T₂(y) = 0. This, however, cannot be simultaneously satisfied along with the constraint T₁(y)+T₂(y)=2y by any T₁ and T₂.

• The transformations may be forced to be the inverses of each other, i.e. ${\mathrm{T}}_2={\mathrm{T}}_1^{-1}$, either explicitly, or through opposite-sign velocity fields. Given that $\mathrm{T}={\mathrm{T}}_2\circ {\mathrm{T}}_1^{-1}$, such a constraint leads to T = T₂ ∘ T₂, meaning that T₂ is the functional square root (half iterate) of T. However, Kuczma et al. (1990) have given examples of a 2D diffeomorphic function that does not have any functional square root (the authors’ Example 11.10.1), and a 1D diffeomorphic function that does not have a diffeomorphic functional square root (the authors’ Example 11.4.1). Accordingly, by using a transformation T₂ to parameterize the solution as T = T₂ ∘ T₂, the space of all diffeomorphic transformations T will not be covered (even if T₂ is not restricted to be diffeomorphic).

Since there is no unique way to define the mid-space, the registration results, T, can depend on the choice of the anti-drift constraints imposed on T₁ and T₂. In fact, as seen above, some legitimate values of T can never be reached due to such constraints.

2.2. Implicit-atlas based registration

2.2.1. Implicit-atlas data term

Another approach to mid-space based pairwise registration is through constructing an unbiased atlas from the images (Ashburner and Ridgway, 2013; Hart et al., 2009; Joshi et al., 2004). In atlas construction, the observed images are assumed to be instances generated from a template image (atlas) with some geometric and intensity variation. Therefore, the problem reduces to finding an atlas, A: Ω → ℝ, and regular transformations T₁ and T₂, where ${\mathrm{T}}_1^{-1}$ and ${\mathrm{T}}_2^{-1}$ take the atlas from the mid-space to the native spaces of the images, in such a way that the deformed versions of the atlas resemble the observed images. The SSD data term to be minimized in such an optimization is sum of the (asymmetric) subject-atlas distance terms, as follows:

$$\stackrel{\sim }{\mathrm{D}}({\mathrm{I}}_1,{\mathrm{I}}_2,\mathrm{A},{\mathrm{T}}_1,{\mathrm{T}}_2):={\int }_{\mathrm{\Omega }}{\left({\mathrm{I}}_1(\mathrm{x})-(\mathrm{A}\circ {\mathrm{T}}_1^{-1})(\mathrm{x})\right)}^2\text{dx}+{\int }_{\mathrm{\Omega }}{\left({\mathrm{I}}_2(\mathrm{z})-(\mathrm{A}\circ {\mathrm{T}}_2^{-1})(\mathrm{z})\right)}^2\text{dz}.$$ (1)

Comparing the images to the atlas in the physically meaningful native image spaces complies with the generative model assumption that the image is generated as a deformed version of the atlas (not vice versa), and that the Gaussian noise is added to the deformed atlas (Allassonnière et al., 2007). By applying the change of variables x = T₁(y) and z = T₂(y) in each integral, the above data term can be written as:

$$\stackrel{\sim }{\mathrm{D}}({\mathrm{I}}_1,{\mathrm{I}}_2,\mathrm{A},{\mathrm{T}}_1,{\mathrm{T}}_2)={\int }_{\mathrm{\Omega }}\left[{\left(({\mathrm{I}}_1\circ {\mathrm{T}}_1)(\mathrm{y})-\mathrm{A}(\mathrm{y})\right)}^2{\mathrm{J}}_1(\mathrm{y})+{\left(({\mathrm{I}}_2\circ {\mathrm{T}}_2)(\mathrm{y})-\mathrm{A}(\mathrm{y})\right)}^2{\mathrm{J}}_2(\mathrm{y})\right]\text{dy},$$ (2)

where J₁(y) ≔ det ∂T₁(y) and J₂(y) ≔ det ∂T₂(y) are the Jacobian determinants of the transformations. The Â(y) minimizing the cost function is derived as (Ma et al., 2008):³

$$\widehat{\mathrm{A}}(\mathrm{y})=\frac{{\mathrm{J}}_1(\mathrm{y})({\mathrm{I}}_1\circ {\mathrm{T}}_1)(\mathrm{y})+{\mathrm{J}}_2(\mathrm{y})({\mathrm{I}}_2\circ {\mathrm{T}}_2)(\mathrm{y})}{{\mathrm{J}}_1(\mathrm{y})+{\mathrm{J}}_2(\mathrm{y})}.$$ (3)

In an explicit-atlas scheme, the transformations and the atlas can be iteratively computed from Eq. (1) and Eq. (3) (Hart et al., 2009).⁴ However, the atlas can indeed be eliminated from Eq. (2) by substituting A(y) with Â(y), leading to an implicit-atlas data term (Ashburner and Ridgway, 2013):

$$\stackrel{\sim }{\mathbf{D}}\left({\mathbf{I}}_{\mathbf{1}},{\mathbf{I}}_{\mathbf{2}},\widehat{\mathbf{A}},{\mathbf{T}}_{\mathbf{1}},{\mathbf{T}}_{\mathbf{2}}\right)={\int }_{\mathbf{\Omega }}\frac{{\left(({\mathbf{I}}_{\mathbf{1}}\circ {\mathbf{T}}_{\mathbf{1}})(\mathbf{y})-({\mathbf{I}}_{\mathbf{2}}\circ {\mathbf{T}}_{\mathbf{2}})(\mathbf{y})\right)}^2}{{\mathbf{J}}_{\mathbf{1}}^{-\mathbf{1}}(\mathbf{y})+{\mathbf{J}}_{\mathbf{2}}^{-\mathbf{1}}(\mathbf{y})}\mathbf{dy}.$$ (4)

As we will see in Section 2.3, keeping both T₁ and T₂ is redundant, which leads to further simplification of the data term by making it independent of the mid-space.

2.2.2. Regularization term

For the regularization term, we use the common Tikhonov integral that penalizes the deviation of the Jacobians of the transformations from the identity matrix. We want the following energy term to remain minimal:

$$\stackrel{\sim }{\mathbf{R}}({\mathbf{T}}_{\mathbf{1}},{\mathbf{T}}_{\mathbf{2}}):={\int }_{\mathbf{\Omega }}{\Vert \mathbf{\partial }\left({\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathbf{x})\right)-𝕀\Vert }_{\mathbf{F}}^{\mathbf{2}}\mathbf{dx}+{\int }_{\mathbf{\Omega }}{\Vert \mathbf{\partial }\left({\mathbf{T}}_{\mathbf{2}}^{-\mathbf{1}}(\mathbf{z})\right)-𝕀\Vert }_{\mathbf{F}}^{\mathbf{2}}\mathbf{dz},$$ (5)

where 𝕀 stands for the d×d identity matrix, and ||·||_F is the Frobenius norm. Note that we regularize ${\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathbf{x})$ and ${\mathbf{T}}_{\mathbf{2}}^{-\mathbf{1}}(\mathbf{z})$ and compute the integrals in the physically-meaningful native image spaces, as opposed to regularizing T₁(y) and T₂(y) in the abstract mid-space. In Section 2.3.2, we will solve the minimization of R̃ with respect to one of the transformations, and rewrite it independent of the mid-space.

2.3. Independence from the mid-space

2.3.1. Mid-space-independent data term

Here we demonstrate that the implicit-atlas based data term is inherently independent of the mid-space. In other words, we can find the optimal transformations T between the two images, without solving for T₁ and T₂ that define the mid-space.

Equation (4) is a cost function of both T₁ and T₂. Even so, further simplification reveals that the two transformations are redundant. The change of variables $\mathbf{y}={\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathbf{x})$, with $\mathbf{dy}=\mathbf{dx}/\left({\mathbf{J}}_{\mathbf{1}}\circ {\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathbf{x})$, reveals:

$$\stackrel{\sim }{\mathbf{D}}\left({\mathbf{I}}_{\mathbf{1}},{\mathbf{I}}_{\mathbf{2}},\widehat{\mathbf{A}},{\mathbf{T}}_{\mathbf{1}},{\mathbf{T}}_{\mathbf{2}}\right)={\int }_{\mathbf{\Omega }}\frac{{\left({\mathbf{I}}_{\mathbf{1}}(\mathbf{x})-\left({\mathbf{I}}_{\mathbf{2}}\circ {\mathbf{T}}_{\mathbf{2}}\circ {\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathbf{x})\right)}^2}{\mathbf{1}+\frac{\left({\mathbf{J}}_{\mathbf{1}}\circ {\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathbf{x})}{\left({\mathbf{J}}_{\mathbf{2}}\circ {\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathbf{x})}}\mathbf{dx}.$$ (6)

Recall that the output transformation T is computed as $\mathbf{T}(\mathbf{x})=({\mathbf{T}}_{\mathbf{2}}\circ {\mathbf{T}}_{\mathbf{1}}^{-\mathbf{1}})(\mathbf{x})$, the Jacobian determinant of which is $\mathit{J}(\mathit{x}):=\mathit{det}\mathit{\partial }\mathit{T}(\mathit{x})=\left({\mathit{J}}_{\mathbf{2}}\circ {\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathit{x})/\left({\mathit{J}}_{\mathbf{1}}\circ {\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}\right)(\mathit{x})$. It now becomes clear that the above integral is, remarkably, only a function of T, and that the data term can be written independently of the individual T₁ and T₂, as:

$$\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}):={\int }_{\mathit{\Omega }}{\left({\mathit{I}}_{\mathbf{1}}(\mathit{x})-({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})(\mathit{x})\right)}^{\mathbf{2}}\frac{\mathit{J}(\mathit{x})}{\mathbf{1}+\mathit{J}(\mathit{x})}\mathit{dx}.$$ (7)

With the data term being independent of T₁ and T₂, the mid-space disappears, eliminating the problem of the mid-space drift as well. We no longer need to enforce any anti-drift constraints, hence naturally avoiding limiting the space of possible transformations T by the particular choice of such constraints, and in fact widening the space of allowable transformations. This is while reducing the degrees of freedom of the optimization to half of those of the unconstrained problem of solving for both T₁ and T₂.

Equation (7) might seem like a non-uniform integral in the native space of I₁. However, this data term originates from Eq. (1), which is the sum of two integrals taken uniformly in the native spaces of images, and non-uniformly only in the mid-space. Therefore, the pitfalls associated with non-uniform integration in native spaces (Aganj et al., 2015b) are not expected to arise.⁵ We have shown that Eq. (1) can be written independently of not only the atlas, as in Eq. (4), but also the mid-space itself, as in Eq (7).

2.3.2. Mid-space-independent regularization term

Contrary to the data term, the regularization in Eq. (5) depends on both T₁ and T₂, and therefore on the mid-space. This is expected, since taking the mid-space far away from the image spaces would cause distortion in both transformations, thereby increasing R̃(T₁,T₂).

Using T₂ = T ∘ T₁, we first rewrite R̃(T₁,T₂) in terms of T₁ and T:

$$\begin{array}{l}\stackrel{\sim }{\mathit{R}}({\mathit{T}}_1,\mathit{T}\circ {\mathit{T}}_{\mathbf{1}})={\int }_{\mathit{\Omega }}{\Vert \mathbf{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dx}+{\int }_{\mathit{\Omega }}{\Vert \mathit{\partial }\left(({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}\circ {\mathit{T}}^{-\mathbf{1}})(\mathit{z})\right)-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dz}\\ ={\int }_{\mathit{\Omega }}{\Vert \mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dx}+{\int }_{\mathit{\Omega }}{\Vert (\mathit{\partial }({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}})\circ {\mathit{T}}^{-\mathbf{1}})(\mathit{z})(\mathit{\partial }{\mathit{T}}^{-\mathbf{1}}\circ {\mathit{T}}^{-\mathbf{1}})(\mathit{z})-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dz}.\end{array}$$ (8)

The change of variables z = T(x) leads to:

$$\stackrel{\sim }{\mathit{R}}({\mathit{T}}_{\mathbf{1}},\mathit{T}\circ {\mathit{T}}_{\mathbf{1}})={\int }_{\mathit{\Omega }}\left({\Vert \mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}+{\Vert \mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)\mathit{\partial }{\mathit{T}}^{-\mathbf{1}}(\mathit{x})-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{J}(\mathit{x})\right)\mathit{dx}.$$ (9)

Using ${\Vert \mathit{X}\Vert }_{\mathit{F}}^{\mathbf{2}}=\mathit{tr}({\mathit{X}}^{\mathit{T}}\mathit{X})$ and then completing the square, one can verify that the above is equal to:

$$\begin{array}{l}\stackrel{\sim }{\mathit{R}}({\mathit{T}}_{\mathbf{1}},\mathit{T}\circ {\mathit{T}}_{\mathbf{1}})={\int }_{\mathit{\Omega }}\mathit{tr}\left[{(\mathit{\partial }\mathit{T}(\mathit{x})-𝕀)}^{\mathit{T}}{\left(𝕀+\frac{1}{\mathit{J}(\mathit{x})}\mathit{\partial }\mathit{T}(\mathit{x})\mathit{\partial }\mathit{T}{(\mathit{x})}^{\mathit{T}}\right)}^{-\mathbf{1}}(\mathit{\partial }\mathit{T}(\mathit{x})-𝕀)\right]\mathit{dx}\\ +{\int }_{\mathit{\Omega }}\mathit{tr}\left[\left(\mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)-\mathit{W}(\mathit{x})\right)\left(𝕀+\mathit{J}(\mathit{x})\mathit{\partial }{\mathit{T}}^{-\mathbf{1}}(\mathit{x})\mathit{\partial }{\mathit{T}}^{-\mathbf{1}}{(\mathit{x})}^{\mathit{T}}\right){\left(\mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)-\mathit{W}(\mathit{x})\right)}^{\mathit{T}}\right]\mathit{dx},\end{array}$$ (10)

where W(x) ≔ (J(x)𝕀 + ∂T(x)^T)(J(x)𝕀 + ∂T(x)∂T(x)^T)⁻¹∂T(x). Diffeomorphism (J(x)>0) implies that the middle factor in each of the two traces in Eq. (10) is positive definite, and consequently the integrands of the two integrals are nonnegative. Since both the data term, D(I₁,I₂,T), and the first integral in Eq. (10) are independent of T₁, the cost function (i.e. weighted sum of data and regularization terms) reaches its minimum with respect to T₁ when the second integral in Eq. (10) equals zero, which happens for a T̂₁ satisfying $\mathit{\partial }\left({\widehat{\mathit{T}}}_{\mathbf{1}}^{-\mathbf{1}}\right)=\mathit{W}$. We will not need the value of T̂₁ though, because T alone will provide us with the complete solution to pairwise registration. Thus, we proceed by assigning T₁ =T̂₁, thereby reducing the regularization term to its first integral:

$$\mathit{R}(\mathit{T}):={\mathit{min}}_{{\mathit{T}}_{\mathbf{1}}}\stackrel{\sim }{\mathit{R}}({\mathit{T}}_{\mathbf{1}},\mathit{T}\circ {\mathit{T}}_{\mathbf{1}})={\int }_{\mathit{\Omega }}\mathit{tr}\left[{(\mathit{\partial }\mathit{T}(\mathit{x})-𝕀)}^{\mathit{T}}{\left(𝕀+\frac{\mathbf{1}}{\mathit{J}(\mathit{x})}\mathit{\partial }\mathit{T}(\mathit{x})\mathit{\partial }\mathit{T}{(\mathit{x})}^{\mathit{T}}\right)}^{-\mathbf{1}}(\mathit{\partial }\mathit{T}(\mathit{x})-𝕀)\right]\mathit{dx}.$$ (11)

The middle factor in the trace is what distinguishes the proposed R(T) from the standard asymmetric Tikhonov regularization, ${\int }_{\mathit{\Omega }}{\Vert \mathit{\partial }\mathit{T}(\mathit{x})-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dx}$.

2.3.3. Mid-space-independent cost function

Image registration can now be independent of the mid-space by minimizing a weighted sum of the proposed data and regularization terms, as:

$$\widehat{\mathit{T}}={\mathit{argmin}}_T\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T})+\mathit{\lambda }\mathit{R}(\mathit{T}),$$ (12)

where λ is a manually determined positive constant. In the continuous case, this cost function is symmetric, as one can verify that D(I₁, I₂, T) = D(I₂, I₁, T⁻¹) and R(T) = R(T⁻¹). In Section 2.4, we will discuss how to alleviate any possible bias due to discretization artifacts.

2.4. Implementation

We minimize the cost function in Eq. (12) via gradient descent with line search. Similar to the diffeomorphic demons framework (Vercauteren et al., 2009), we minimize in alternative steps the data term and the regularization term, using the compositive and additive schemes, respectively.⁶

For the data term, we update T by composing it with a transformation S, which is chosen to be the exponential of the update field to preserve the diffeomorphism. To that end, at each iteration we fix T and compute the variation of D(I₁, I₂, T ∘ S) with respect to S, with S currently assumed to be the identity transformation (Id(x) = x). Following the chain rule for the derivative, we have:

$${\frac{\mathit{\delta }\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}\circ \mathit{S})}{\mathit{\delta }\mathit{S}}|}_{\mathit{S}=\text{Id}}=\frac{\mathit{\partial }}{\mathit{\partial }\mathit{S}}(\mathit{T}\circ \mathit{S}){\frac{\mathit{\delta }\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}\circ \mathit{S})}{\mathit{\delta }(\mathit{T}\circ \mathit{S})}|}_{\mathit{S}=\text{Id}}=\mathit{\partial }{\mathit{T}}^{\mathit{T}}\frac{\mathit{\delta }\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T})}{\mathit{\delta }\mathit{T}}.$$ (13)

The notation (x) has been dropped for brevity. We use the Lemma in Appendix B to compute the variation of the data term as follows:

$${\frac{\mathit{\delta }\mathit{D}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}\circ \mathit{S})}{\mathit{\delta }\mathit{S}}|}_{\mathit{S}=\text{Id}}=-\frac{\mathbf{2}({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})\mathit{J}}{{(\mathbf{1}+\mathit{J})}^{\mathbf{2}}}\left(\mathit{\nabla }{\mathit{I}}_{\mathbf{1}}+\mathit{J}\mathit{\nabla }({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})\right)+{({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})}^2\mathit{\nabla }\left[{\left(\frac{\mathit{J}}{\mathbf{1}+\mathit{J}}\right)}^{\mathbf{2}}\right].$$ (14)

The last term of Eq. (14) involves the Hessian of the deformation field, which can introduce large discretization error that possibly outweighs the accuracy that this term brings about. In our experiments, ignoring this second-derivative term improved the optimization of the cost function.

As for the regularization term, we derive its variation in Appendix C as:

$$\frac{\mathit{\delta }\mathit{R}(\mathit{T})}{\mathit{\delta }\mathit{T}}=-\mathit{div}\left[\mathbf{2}\mathit{JP}\left(𝕀-{\mathit{P}}^{\mathit{T}}\mathit{\partial }\mathit{T}\right)+{\Vert {\mathit{P}}^{\mathit{T}}\mathit{\partial }\mathit{T}\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{C}\right],$$ (15)

where P ≔(J𝕀 + ∂T∂T^T)⁻¹ (∂T − 𝕀), the divergence is defined as div X ≔ (∇^TX^T)^T, and C ≔ J∂T^−1T is the cofactor matrix of ∂T. Keeping the transformation diffeomorphic is especially important for the proposed method, given that the Jacobian appears in both the data and regularization terms. Therefore, we also include the following symmetric logarithmic barrier term to keep the Jacobian determinant away from zero:

$$\mathit{L}(\mathit{T}):={\int }_{\mathit{\Omega }}{(\mathit{log}\mathit{J}(\mathit{x}))}^{\mathbf{2}}\left(\mathbf{1}+\mathit{J}(\mathit{x})\right)\mathit{dx},$$ (16)

where γL(T) is added to R(T) (and to the regularization terms of the other methods in Section 3). Note that L(T) = L(T⁻¹). In our experiments, the best optimization performance was obtained with γ = 10⁻⁶, which is the value used in the reported results. We calculate the variation of L(T) using the Lemma in Appendix B, as:

$$\frac{\mathit{\delta }\mathit{L}(\mathit{T})}{\mathit{\delta }\mathit{T}}=-\mathit{C}\mathit{\nabla }\left[\mathbf{2}\left(\mathbf{1}+\frac{\mathbf{1}}{\mathit{J}}\right)\mathit{log}\mathit{J}+{(\mathit{log}\mathit{J})}^{\mathbf{2}}\right].$$ (17)

We start the line search by initializing $\mathit{S}(\mathit{x})\leftarrow \mathit{x}-\mathit{\Delta }{\frac{\mathit{\delta }\mathit{D}}{\mathit{\delta }\mathit{S}}|}_{\mathit{S}=\text{Id}}$ for a small step size Δ. Then, at step k of the line search (starting from k = 1), we compute the intermediate transformation T^k ← T ∘ S and regularize it with ${\mathit{T}}^{\mathit{k}}\leftarrow {\mathit{T}}^{\mathit{k}}-{\mathbf{2}}^{\mathit{k}-\mathbf{1}}\mathit{\Delta }{\frac{\mathit{\delta }\mathit{R}}{\mathit{\delta }\mathit{T}}|}_{\mathit{T}={\mathit{T}}^{\mathit{k}}}$.⁷ Next we compute the value of the cost function for the transformation T^k, and compare it with those of the previous steps of the line search. If the current value of cost function has not been the minimum for M consecutive line-search steps, then we stop the line search and update T to be the intermediate transformation T^k* that minimized the cost function. Otherwise, inspired by the diffeomorphic demons (Vercauteren et al., 2009), we update S ← S ∘ S and move on to step k+1 of the line search. We heuristically chose M = 4 and M = 2 for our 2D and 3D experiments, respectively. Note that the intermediate transformations T^k do not need all to be stored in the memory during the line search; only T^k* that minimizes the cost function needs to be kept in the memory and updated at each line-search step.

As already mentioned, the proposed continuous cost function in Eq. (12) is symmetric, meaning that it is unaffected by concurrently changing the order of the input images and inverting the transformation. We take a unidirectional cost-function optimization approach, such as in (Leow et al., 2007), which resamples I₂, but not I₁. Therefore, discretization artifacts may produce a symmetry-breaking bias. Yushkevich et al. (2010) have shown, however, that in hippocampal atrophy estimation most of such bias could be attributed to linear registration, rather than the subsequent deformable registration, and that symmetrizing the linear registration (Reuter et al., 2010) can remove the bias. Nonetheless, further steps can be taken to address the discretization bias in deformable registration, for instance, by avoiding the resampling step after interpolation, thereby eliminating the resampling artifacts and their resultant registration asymmetry. To that end, the continuously interpolated integrals of the cost function can be either calculated analytically or estimated by quasi-random sampling (Aganj et al., 2013). An alternative strategy to reduce asymmetry due to discretization would be to optimize the cost function in a bidirectional fashion, i.e. with respect to both T and T⁻¹ (Sabuncu et al., 2009). Although this is (in the continuous domain) still equivalent to optimizing the original unidirectional cost function, it will balance the resampling error on the two image spaces at the discretization step.

3. Results and discussion

We tested the proposed deformable registration method on brain T1-weighted magnetic resonance images, which had been pre-processed (intensity-normalized, skull-stripped, linearly co-registered, and resampled to 1-mm³ isotropic voxel) in FreeSurfer (Fischl, 2012). We used both 2D and 3D images in our experiments (in sections 3.1 and 3.2, respectively) to demonstrate the applicability of our method independently of the dimensionality of the images. To evaluate our mid-space-independent (MSI) registration data term, we compared it to two other data terms; first, the standard asymmetric SSD:

$${\mathit{D}}_{\text{asymmetric}}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}):={\int }_{\mathit{\Omega }}{\left({\mathit{I}}_{\mathbf{1}}(\mathit{x})-({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})(\mathit{x})\right)}^{\mathbf{2}}\mathit{dx},$$ (18)

with the following compositive gradient:

$${\frac{\mathit{\delta }}{\mathit{\delta }\mathit{S}}{\mathit{D}}_{\text{asymmetric}}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}\circ \mathit{S})|}_{\mathit{S}=\text{Id}}=-\mathbf{2}({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})\mathit{\nabla }({\mathit{I}}_{\mathbf{2}}\circ \mathit{T});$$ (19)

second, with the popular symmetrization data term that minimizes the average of the asymmetric forward and backward data terms (Cachier and Rey, 2000; Christensen and Johnson, 2001; Tagare et al., 2009; Trouvé and Younes, 2000):

$${\mathit{D}}_{\text{symmetrization}}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}):={\int }_{\mathit{\Omega }}{\left({\mathit{I}}_{\mathbf{1}}(\mathit{x})-({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})(\mathit{x})\right)}^{\mathbf{2}}\frac{\mathbf{1}+\mathit{J}(\mathit{x})}{\mathbf{2}}\mathit{dx},$$ (20)

with the following compositive gradient, computed using the Lemma in Appendix B:

$${\frac{\mathit{\delta }}{\mathit{\delta }\mathit{S}}{\mathit{D}}_{\text{symmetrization}}({\mathit{I}}_{\mathbf{1}},{\mathit{I}}_{\mathbf{2}},\mathit{T}\circ \mathit{S})|}_{\mathit{S}=\text{Id}}=-({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})\left(\mathit{J}\mathit{\nabla }{\mathit{I}}_{\mathbf{1}}+\mathit{\nabla }({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})\right).$$ (21)

To assess the performance of the data terms independently of the regularization, we first compared the three (asymmetric, symmetrization, and MSI) data terms while employing the standard asymmetric regularization term ${\int }_{\mathit{\Omega }}{\Vert \mathit{\partial }\mathit{T}(\mathit{x})-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dx}$ (with the variation −2∇²T) for all three of them. Subsequently, to evaluate the proposed regularization, we used the MSI data and regularization terms together in registration.⁸

We implemented the deformable image registration (for all the mentioned cost functions) in Matlab using the displacement-field representation of the transformation.⁹ All input images had intensity values between 0 and 1, and we used the double-precision floating-point format to represent the images and transformations. Registration was done subsequently in three levels of resolution: quarter-, half-, and full-resolution. In the quarter- and half-resolution levels, we used cubic interpolation to down-sample the input images, and then to up-sample the output transformation to be used as the initial value for the next level of resolution.

The optimal values for the registration parameters generally vary among different cost functions. Thus, for a fair comparison, we considered the results of each cost function with the parameter values that lead to its optimal performance. For each pair of input images and for each method, we repeatedly ran the registration with a wide range of values for the regularization parameter λ and the step size Δ, and then chose the experiment with the best result (i.e. the one producing the optimal value for the criterion that is explained in each of the following subsections). In the following, we report the cross-subject mean of the optimal parameter values λ^* and Δ^* in each set of experiments (choosing extreme values for λ or Δ produced suboptimal results or caused the optimization to not converge).

3.1. Retrieval of synthetic 2D deformations

In the first set of experiments, we compared the methods on the mid-sagittal planes of 20 brain images taken from the publicly available Open Access Series of Imaging Studies (OASIS) database (Marcus et al., 2007), using synthetic deformations that provide us with the entire ground-truth transformation. From each volume, we extracted the sagittal slice, I, located four voxels to the right of the mid-sagittal plane, and resampled it to the size 128×128 (Figure 1a). For each subject, we synthesized two random deformation fields ${\mathit{T}}_{\mathbf{1}}^{\mathit{synth}}$ and ${\mathit{T}}_{\mathbf{2}}^{\mathit{synth}}$, spatially low-pass filtered them, and applied them to I. Next, we added Gaussian noise (η₁ and η₂) with standard deviation of 0.1 to the deformed images to obtain two synthetic images, ${\mathit{I}}_{\mathbf{1}}=\mathit{I}\circ {\mathit{T}}_{\mathbf{1}}^{\mathit{synth}}+{\mathit{\eta }}_{\mathbf{1}}$ and ${\mathit{I}}_{\mathbf{2}}=\mathit{I}\circ {\mathit{T}}_{\mathbf{2}}^{\mathit{synth}}+{\mathit{\eta }}_{\mathbf{2}}$ (Figure 1b,c), which we then registered with each other using different methods (Figure 1d).¹⁰ The registration experiments consisted of 1000 gradient descent iterations at each of the three resolution levels. For each of the 20 subjects and each of the 4 methods, we repeated the registration with the regularization parameter λ taking a range of 50 different values (from 0.00001 to 1), and the step size Δ taking a range of 30 different values (from 0.01 to 256).

Fig 1.

The original image (a) was deformed by two synthetic random transformations, and Gaussian noise was added (b,c). The deformed images were then registered with each other (d).

Ideally, we would expect to retrieve a transformation T satisfying ${\mathit{T}}_{\mathbf{1}}^{\mathit{synth}}\approx {\mathit{T}}_{\mathbf{2}}^{\mathit{synth}}\circ \mathit{T}$. Accordingly, we computed the error $\mathit{\epsilon }:={\int }_{\mathit{\Omega }}{\Vert {\mathit{T}}_{\mathbf{1}}^{\mathit{synth}}(\mathit{x})-\left({\mathit{T}}_{\mathbf{2}}^{\mathit{synth}}\circ \mathit{T}\right)(\mathit{x})\Vert }_{\mathbf{2}}^{\mathbf{2}}\mathit{dx}$, and for each subject and method, chose the best result (the one with the smallest ε) among the experiments with different values of λ and Δ. The cross-subject averages of the optimal ε^*, λ^*, and Δ^* are presented in Table 1, and a summary of the results of pairwise comparisons between different methods is presented in Table 2. As can be seen, all the methods significantly reduced the error compared to the pre-registration (i.e. T(x) = x) error. The proposed MSI data term with standard asymmetric regularization resulted in the smallest cross-subject average error. Specifically, the MSI data term significantly outperformed the symmetrization data term, but performed only slightly better than the asymmetric data term.¹¹ Regarding the proposed regularization, methods using standard asymmetric regularization resulted in significantly smaller errors than the one using MSI regularization did (see Section 3.2 for a discussion).

Table 1.

Minimum error ε^*, optimal regularization parameter λ^*, optimal step size Δ^*, and the runtime of optimal experiment have been averaged for each method across subjects and presented along with their standard errors of the mean.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
ε*	5.602 ± 0.082	4.425 ± 0.075	4.470 ± 0.075	4.421 ± 0.070	4.701 ± 0.078
λ*	-	0.0027 ± 0.0003	0.006 ± 0.001	0.004 ± 0.001	0.021 ± 0.002
Δ*	-	29 ± 5	11 ± 3	49 ± 7	0.3 ± 0.1
Runtime (CPU)	-	8.8 ± 0.6 m	8.1 ± 0.2 m	7.9 ± 0.1 m	22.4 ± 0.5 m

Table 2.

Pairwise comparison of the results in Section 3.1. In row i and column j, the percentage of the subjects for which method i outperformed method j (resulted in lower ε^*) is shown, along with the p-values for two-tailed paired Student’s t- (p_t) and sign rank (p_s) tests.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
Before registration	-	0%	0%	0%	0%
		pt = 7 × 10−17	pt = 6 × 10−16	pt = 1 × 10−15	pt = 7 × 10−15
		ps = 9 × 10−5	ps = 9 × 10−5	ps = 9 × 10−5	ps = 9 × 10−5
Data term: Asymmetric Regularization: Asymmetric	100%	-	70%	45%	100%
	pt = 7 × 10−17		pt = 0.02	pt = 0.8	pt = 6 × 10−8
	ps = 9 × 10−5		ps = 0.03	ps = 0.8	ps = 9 × 10−5
Data term: Symmetrization Regularization: Asymmetric	100%	30%	-	25%	100%
	pt = 6 × 10−16	pt = 0.02		pt = 0.02	pt = 7 × 10−7
	ps = 9 × 10−5	ps = 0.03		ps = 0.01	ps = 9 × 10−5
Data term: MSI Regularization: Asymmetric	100%	55%	75%	-	100%
	pt = 1 × 10−15	pt = 0.8	pt = 0.02		pt = 5 × 10−8
	ps = 9 × 10−5	ps = 0.8	ps = 0.01		ps = 9 × 10−5
Data term: MSI Regularization: MSI	100%	0%	0%	0%	-
	pt = 7 × 10−15	pt = 6 × 10−8	pt = 7 × 10−7	pt = 5 × 10−8
	ps = 9 × 10−5	ps = 9 × 10−5	ps = 9 × 10−5	ps = 9 × 10−5

The runtimes averaged across the optimal experiments are shown in Table 1 (see Section 3.2 for hardware specifications). GPU parallelization did not offer any advantage in our 2D experiments.

3.2. Registration of labeled 3D images

We performed a second set of experiments using a dataset of 3D brain T1-weighted magnetic resonance images (Fischl et al., 2002). The dataset included 39 (14 male and 25 female) subjects, of which 28 were healthy and 11 had been diagnosed with dementia. The mean and standard deviation of the age was 55 and 23 years, respectively. This dataset additionally included manual-label volumes for 37 neuroanatomical structures (Figure 2), which had been used in the creation of the volumetric atlas of FreeSurfer (Fischl, 2012). To reduce the blank areas in the background, we cropped the images to the size 158×182×190. Out of the 39×38=1482 possible combinations for ordered pairs, we randomly selected 40 pairs of images and registered each pair using the aforementioned methods. The registration experiments consisted of 100 gradient descent iterations at each of the three resolution levels. For each of the 40 pairs of subjects and each of the 4 methods, we repeated the registration with the regularization parameter λ taking a range of 30 different values (from 0.00001 to 1), and the step size Δ taking a range of 15 different values (from 0.01 to 500).

Fig 2.

Coronal slices from 3D labels of four of the subjects used in Section 3.2.

For validation and comparison, we considered 12 key subcortical regions: left/right amygdala, caudate, hippocampus, pallidum, putamen, and thalamus.¹² As a similarity score, we computed in the union of these regions the portion of the voxels with matching labels between the two images. The scores are defined respectively in the native spaces of image 1 and image 2 as:

$$\begin{array}{l}{\mathit{\psi }}_{\mathbf{1}}:=\frac{\mathbf{1}}{\mid {\mathit{\Omega }}_{\mathbf{1}}\mid }{\sum }_{\mathit{x}\in {\mathit{\Omega }}_{\mathbf{1}}}{\mathit{\delta }}_{{\mathit{l}}_{\mathbf{1}}(\mathit{x}),({\mathit{l}}_{\mathbf{2}}\circ \mathit{T})(\mathit{x})},\\ {\mathit{\psi }}_{\mathbf{2}}:=\frac{\mathbf{1}}{\mid {\mathit{\Omega }}_{\mathbf{2}}\mid }{\sum }_{\mathit{z}\in {\mathit{\Omega }}_{\mathbf{2}}}{\mathit{\delta }}_{\left({\mathit{l}}_{\mathbf{1}}\circ {\mathit{T}}^{-\mathbf{1}}\right)(\mathit{z}),{\mathit{l}}_{\mathit{2}}(\mathit{z})},\end{array}$$ (22)

where Ω_i is the set of all voxels inside the union of the 12 subcortical regions in image i, with |Ω_i| being its size; ${\mathit{\delta }}_{\mathit{i},\mathit{j}}=\{\begin{array}{ccc}\mathbf{0}& ,& \mathit{i}\ne \mathit{j}\\ \mathbf{1}& ,& \mathit{i}=\mathit{j}\end{array}$ is the Kronecker delta; and l₁ and l₂ are the label images that take 13 different integer values: 1 to 12 indicating each of the 12 subcortical regions, and 0 for a voxel outside of those 12 regions. Consequently, δ_l1(x),l2(z) is 1 if l₁ has the same label at voxel x as l₂ does at voxel z, and is 0 otherwise. For ψ₁, we deformed the label volume of image 2 with the computed transformation T, and for ψ₂, we pushed the label values from image 1 forward to the space of image 2 according to T. We used the nearest-neighbor label interpolation in both cases. For each of the 4 methods and each of the 40 pairs of subjects, we chose the maximal ${\mathit{\psi }}_{\mathbf{1}}^{\mathbf{\ast }}$ and the maximal ${\mathit{\psi }}_{\mathbf{2}}^{\mathbf{\ast }}$, achieved among the experiments with different values of λ and Δ. The cross-subject average of ${\mathit{\psi }}_{\mathbf{1}}^{\mathbf{\ast }}$ in the native space of image 1, along with the averages of optimal λ^* and Δ^* are presented in Table 3, and a summary of the pairwise comparisons between different methods is presented in Table 4. Results of the label-matching analysis in the native space of image 2 (i.e., using ${\mathit{\psi }}_{\mathbf{2}}^{\mathbf{\ast }}$) are presented in Table 5 and Table 6.

Table 3.

Maximum label-matching score ${\mathit{\psi }}_{\mathbf{1}}^{\mathbf{\ast }}$ in the native space of image 1, optimal regularization parameter λ^*, optimal step size Δ^*, label-matching asymmetry σ, the portion of voxels with J≤0, and the runtime of the optimal experiment have been averaged for each method across subjects and presented along with their standard errors of the mean.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
${\mathit{\psi }}_{\mathbf{1}}^{\mathbf{\ast }}$	0.638 ± 0.012	0.849 ± 0.004	0.835 ± 0.004	0.847 ± 0.003	0.784 ± 0.006
λ*	-	0.003 ± 0.001	0.006 ± 0.001	0.003 ± 0.001	0.0006 ± 0.0002
Δ*	-	143 ± 13	59 ± 9	151 ± 15	3.1 ± 0.5
σ	-	0.099 ± 0.008	0.023 ± 0.003	0.083 ± 0.007	0.050 ± 0.003
J≤0	-	(2.2 ± 0.2) %	(0.08 ± 0.02) %	(1.7 ± 0.3) %	(4.2 ± 0.3) %
Runtime (CPU)	-	8.4 ± 0.3 hr	7.9 ± 0.7 hr	7.7 ± 0.3 hr	26.9 ± 0.4 hr
Runtime (GPU)	-	30 ± 1 m	29 ± 3 m	30 ± 2 m	115 ± 2 m

Table 4.

Pairwise comparison of the label-matching results in the native space of image 1. In row i and column j, the percentage of the subjects for which method i outperformed method j (resulted in higher ${\mathit{\psi }}_{\mathbf{1}}^{\mathbf{\ast }}$) is shown, along with the p-values for two-tailed paired Student’s t-(p_t) and sign rank (p_s) tests.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
Before registration	-	0% pt = 3 × 10−20 ps = 4 × 10−8	0% pt = 7 × 10−20 ps = 4 × 10−8	0% pt = 2 × 10−20 ps = 4 × 10−8	0% pt = 7 × 10−17 ps = 4 × 10−8
Data term: Asymmetric Regularization: Asymmetric	100% pt = 3 × 10−20 ps = 4 × 10−8	-	92.5% pt = 2 × 10−8 ps = 1 × 10−7	72.5% pt = 0.13 ps = 0.01	100% pt = 6 × 10−15 ps = 4 × 10−8
Data term: Symmetrization Regularization: Asymmetric	100% pt = 7 × 10−20 ps = 4 × 10−8	7.5% pt = 2 × 10−8 ps = 1 × 10−7	-	10% pt = 3 × 10−7 ps = 1 × 10−7	100% pt = 3 × 10−12 ps = 4 × 10−8
Data term: MSI Regularization: Asymmetric	100% pt = 2 × 10−20 ps = 4 × 10−8	27.5% pt = 0.13 ps = 0.01	90% pt = 3 × 10−7 ps = 1 × 10−7	-	100% pt = 2 × 10−14 ps = 4 × 10−8
Data term: MSI Regularization: MSI	100% pt = 7 × 10−17 ps = 4 × 10−8	0% pt = 6 × 10−15 ps = 4 × 10−8	0% pt = 3 × 10−12 ps = 4 × 10−8	0% pt = 2 × 10−14 ps = 4 × 10−8	-

Table 5.

Maximum label-matching score ${\mathit{\psi }}_{\mathbf{2}}^{\mathbf{\ast }}$ in the native space of image 2, optimal regularization parameter λ^*, optimal step size Δ^*, label-matching asymmetry σ, the portion of voxels with J≤0, and the runtime of the optimal experiment have been averaged for each method across subjects and presented along with their standard errors of the mean.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
${\mathit{\psi }}_{\mathbf{2}}^{\mathbf{\ast }}$	0.6218 ± 0.0161	0.8143 ± 0.0065	0.8158 ± 0.0059	0.8159 ± 0.0063	0.7365 ± 0.0089
λ*	-	0.004 ± 0.001	0.005 ± 0.001	0.005 ± 0.001	0.003 ± 0.001
Δ*	-	35 ± 2	25 ± 2	34 ± 2	0.3 ± 0.1
σ	-	0.010 ± 0.003	0.005 ± 0.001	0.006 ± 0.001	0.033 ± 0.003
J≤0	-	(0.033 ± 0.008) %	(0.011 ± 0.003) %	(0.018 ± 0.005) %	(1.8 ± 0.2) %
Runtime (CPU)	-	6.19 ± 0.05 hr	6.32 ± 0.06 hr	6.17 ± 0.03 hr	26.6 ± 0.6 hr
Runtime (GPU)	-	22.0 ± 0.1 m	22.3 ± 0.2 m	21.82 ± 0.04 m	119 ± 2 m

Table 6.

Pairwise comparison of the label-matching results in the native space of image 2. In row i and column j, the percentage of the subjects for which method i outperformed method j (resulted in higher ${\mathit{\psi }}_{\mathbf{2}}^{\mathbf{\ast }}$) is shown, along with the p-values for two-tailed paired Student’s t-(p_t) and sign rank (p_s) tests.

Mean across optimal experiments:	Before registration	Data term: Asymmetric Regularization: Asymmetric	Data term: Symmetrization Regularization: Asymmetric	Data term: MSI Regularization: Asymmetric	Data term: MSI Regularization: MSI
Before registration	-	0% pt = 3 × 10−18 ps = 4 × 10−8	0% pt = 5 × 10−18 ps = 4 × 10−8	0% pt = 2 × 10−18 ps = 4 × 10−8	0% pt = 3 × 10−15 ps = 4 × 10−8
Data term: Asymmetric Regularization: Asymmetric	100% pt = 3 × 10−18 ps = 4 × 10−8	-	45% pt = 0.2 ps = 0.4	30% pt = 0.002 ps = 0.004	100% pt = 5 × 10−18 ps = 4 × 10−8
Data term: Symmetrization Regularization: Asymmetric	100% pt = 5 × 10−18 ps = 4 × 10−8	55% pt = 0.2 ps = 0.4	-	30% pt = 0.9 ps = 0.06	100% pt = 5 × 10−18 ps = 4 × 10−8
Data term: MSI Regularization: Asymmetric	100% pt = 2 × 10−18 ps = 4 × 10−8	67.5% pt = 0.002 ps = 0.004	70% pt = 0.9 ps = 0.06	-	100% pt = 1 × 10−18 ps = 4 × 10−8
Data term: MSI Regularization: MSI	100% pt = 3 × 10−15 ps = 4 × 10−8	0% pt = 5 × 10−18 ps = 4 × 10−8	0% pt = 5 × 10−18 ps = 4 × 10−8	0% pt = 1 × 10−18 ps = 4 × 10−8	-

All the methods significantly improved label matching compared to the pre-registration case. When we compared the labels in the native space of image 1, asymmetric registration produced the highest label-matching score, which was significantly higher than those by both the symmetrization method and (according to sign rank test but not t-test) the proposed MSI method. This is expected, given that the asymmetric method minimizes a cost function that is defined in the space of image 1. Conversely, when we compared the labels in the native space of image 2, the MSI data term with asymmetric regularization resulted in the highest label-matching score among the methods, which was significantly higher than that of the asymmetric data term. In both native spaces, the proposed MSI data term with asymmetric regularization resulted in higher label-matching score than the symmetrization data term did, which was significant in the native space of image 1.

Using the MSI regularization significantly reduced the label-matching score compared to using the standard asymmetric regularization. The sub-optimal performance of this regularization term may be due to numerical instabilities that are caused by the matrix inversion (J𝕀 + ∂T∂T^T)⁻¹, which is necessary to compute the variation of the regularization term in Eq. (15). Despite the fact that the exponential update (for the data term) and the logarithmic barrier keep the transformation mostly diffeomorphic, there still remain some voxels with non-positive Jacobian determinant in the transformation that can make the mentioned matrix inversion unstable. This is not an issue for the asymmetric regularization as the variation of the asymmetric regularization term, −2∇²T, is linear with respect to the transformation and not susceptible to non-diffeomorphic regions.

Next, we assessed the sensitivity of the label-matching scores to the ordering of the input images. Using the same optimal parameters, we repeated the experiments but this time with swapped input images (I₁ ↔I₂), and computed ${\mathit{\psi }}_{\mathbf{1}}^{\mathit{bw}}$ and ${\mathit{\psi }}_{\mathbf{2}}^{\mathit{bw}}$ similarly to above for the new backward experiments. For symmetric registration, we expect the score of the forward registration in the reference space (ψ₁) to be similar to that of the backward registration in the moving image space ( ${\mathit{\psi }}_{\mathbf{2}}^{\mathit{bw}}$), and also the score of the forward registration in the moving image space (ψ₂) to be similar to that of the backward registration in the reference space ( ${\mathit{\psi }}_{\mathbf{1}}^{\mathit{bw}}$). Accordingly, we computed the following measure of label-matching asymmetry, whose mean is reported for different methods in Table 3 and Table 5:

$$\mathit{\sigma }:=\left(\right|{\mathit{\psi }}_{\mathbf{1}}-{\mathit{\psi }}_{\mathbf{2}}^{\mathit{bw}}|+|{\mathit{\psi }}_{\mathbf{2}}-{\mathit{\psi }}_{\mathbf{1}}^{\mathit{bw}}\left|\right)/\mathbf{2}.$$ (23)

As expected, the asymmetric data term resulted in a larger σ than the two symmetric data terms did (with asymmetric regularization). Discretization artifacts – during both registration and score computation – are an important factor contributing to the positive values of σ for the symmetric cost functions (see Section 2.4). The values of σ are considerably smaller for optimal transformations that are chosen based on label-matching in the space of image 2 (Table 5), where labels in the moving image space are supposedly better defined due to smoother inverse transformations. Asymmetric regularization is another factor contributing to an elevated σ. Using the MSI (instead of asymmetric) regularization helped to reduce σ for the MSI data term when transformations had been optimized based on ψ₁ (but not when based on ψ₂).

To assess the smoothness of the optimal transformations, we also computed the percentage of voxels where the Jacobian determinant of the optimal transformation was non-positive (J(x) ≤ 0), shown in Table 3 and Table 5. This value is lowest for the symmetrization data term. Furthermore, for all the methods, this value is significantly lower when optimal transformations are chosen based on label matching in the native space of image 2. This may be because a lower number of voxels with non-positive Jacobian determinant implies a smoother inverse transform, hence a better label definition in the space of image 2.

Although we distributed the bulk of our experiments on various hardware platforms, we used consistent hardware for the abovementioned inverse optimal experiments while measuring the runtimes, which are shown in Table 3 and Table 5. When the asymmetric regularization was used, the runtimes were roughly in the same range for the three data terms, while the experiments with optimal label matching in the space of image 2 were faster than those with optimal matching in the space of image 1. We observed that the differences between the runtimes were mainly because of the variation in the total number of line-search iterations performed in the experiments, to which the runtime is proportional. When the MSI regularization was used, registration took about 4 times longer, mostly due to the matrix-inversion operations in Eq. (11) and Eq. (15). For the CPU experiments, we used processors based on the Intel Xeon E5-2667 v3 Haswell 3.2 GHz (3.6 GHz with turbo) with DDR 4 memory. We ran two experiments simultaneously on each 8-core 56GB-RAM virtual machine. (Note that Matlab may internally multithread some of its matrix operations.) Additionally, we repeated each optimal experiment using an NVIDIA Tesla K80 graphics processing unit (GPU) and 1 CPU core, which made the 3D registration on average 16 times faster.

Lastly, to see the range of label-matching scores that existing state-of-the-art registration toolboxes would produce on our data, we repeated the experiments using Advanced Normalization Tools (ANTs) Symmetric Normalization (SyN) (Avants et al., 2008) and FSL FMRIB’s Nonlinear Image Registration Tool (FNIRT) (Andersson et al., 2007). We used the same three levels of resolution and a maximum of 100 iterations per level, although registration often converged in less than 100 iterations. The cross-subject mean of (ψ_1,ψ₂) was (0.74 ± 0.02, 0.72 ± 0.02) for ANTs SyN and (0.80 ± 0.01, 0.70 ± 0.02) for FNIRT. Since these label-matching scores were computed using the toolboxes’ default or suggested parameter values¹³, they should not be directly compared to the values in Table 3 and Table 5, which have been optimized by an exhaustive search in the parameter space.

4. Conclusions

We have demonstrated for the first time that implicit-atlas image registration is inherently independent of the mid-space, which led to deriving a new data term for deformable image registration. We have also derived a new regularization term by analytically reformulating the regularization independently of the mid-space. The independence of the cost function from the image-to-atlas transformations alleviates the need for enforcing artificial anti-drift constraints that potentially restrict the space of possible deformations. We have evaluated our method through experiments on 2D and 3D brain magnetic resonance image datasets, and shown that the proposed data term often outperformed competing methods. Future research includes: stabilizing the MSI regularization via a proper projection of the Jacobian matrix onto the space of diffeomorphism, detailed implementation and thorough validation of MSI group-wise deformable registration, and incorporation of the MSI cost function in surface-based registration.

Highlights.

• A new mid-space-independent method for deformable image registration is proposed.

• The need for enforcing artificial anti-drift constraints are alleviated.

• The proposed MSI method is validated on brain magnetic resonance image datasets.

• The toolbox has been made publicly available.

Appendix A. Extension to group-wise registration

In this appendix, we show how our MSI framework can be extended for the registration of N images I₁,…,I_N:Ω→ℝ, where Ω⊆ℝ^d. For group-wise registration, we want to compute the set of regular transformations {T_ij:Ω→Ω|i,j=1,…,N}, where T_ij deforms I_i so as to make I_i ∘T_ij and I_j most similar to each other. In a common approach to group-wise registration, the pairwise transformations {T_ij} are parameterized as ${\mathit{T}}_{\mathit{ij}}={\mathit{T}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{j}}^{-\mathbf{1}}$, where T₁,…,T_N are transformations that take I₁,…,I_N to a mid-space (see Section 1 for references to the mid-space approaches).

Similar to the pairwise case of Eq. (1), we define the SSD data term as:

$$\stackrel{\sim }{\mathit{D}}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}},\mathit{A};{\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}}):={\sum }_{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\int }_{\mathit{\Omega }}{\left({\mathit{I}}_{\mathit{i}}(\mathit{x})-\left(\mathit{A}\circ {\mathit{T}}_{\mathit{i}}^{-\mathbf{1}}\right)(\mathit{x})\right)}^{\mathbf{2}}\mathit{dx}$$ (24)

After applying the change of variables x=T_i(y) in each integral, the Â(y) minimizing the cost function is derived as (Ma et al., 2008):

$$\widehat{\mathit{A}}(\mathit{y})=\frac{{\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y})}{{\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})},$$ (25)

where J_i(y):= det∂T_i(y). The atlas can now be eliminated from Eq. (24) by substituting A(y) with Â(y), leading to an implicit-atlas data term:

$$\begin{array}{l}\stackrel{\sim }{\mathit{D}}\left({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}},\widehat{\mathit{A}};{\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}}\right):={\int }_{\mathit{\Omega }}\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\left(({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y})-\widehat{\mathit{A}}(\mathit{y})\right)}^{\mathbf{2}}{\mathit{J}}_{\mathit{i}}(\mathit{y})\mathit{dy}\\ ={\int }_{\mathit{\Omega }}\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\left(({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y})-\frac{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})({\mathit{I}}_{\mathit{k}}\circ {\mathit{T}}_{\mathit{k}})(\mathit{y})}{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})}\right)}^{\mathbf{2}}{\mathit{J}}_{\mathit{i}}(\mathit{y})\mathit{dy}\\ =\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\int }_{\mathit{\Omega }}\left({\mathit{I}}_{\mathit{i}}^{\mathbf{2}}\circ {\mathit{T}}_{\mathit{i}}\right)(\mathit{y}){\mathit{J}}_{\mathit{i}}(\mathit{y})\mathit{dy}+{\int }_{\mathit{\Omega }}{\left(\frac{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})({\mathit{I}}_{\mathit{k}}\circ {\mathit{T}}_{\mathit{k}})(\mathit{y})}{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})}\right)}^{\mathbf{2}}\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{J}}_{\mathit{i}}(\mathit{y})\mathit{dy}\\ -\mathbf{2}{\int }_{\mathit{\Omega }}\frac{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})({\mathit{I}}_{\mathit{k}}\circ {\mathit{T}}_{\mathit{k}})(\mathit{y})}{{\sum }_{\mathit{k}}{\mathit{J}}_{\mathit{k}}(\mathit{y})}\sum _{\mathit{i}=1}^{\mathit{N}}{\mathit{J}}_{\mathit{i}}(\mathit{y})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y})\mathit{dy}\\ ={\int }_{\mathit{\Omega }}\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{I}}_{\mathit{i}}^{\mathbf{2}}(\mathit{x})\mathit{dx}-{\int }_{\mathit{\Omega }}\frac{{({\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y}))}^{\mathbf{2}}}{{\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})}\mathit{dy}.\end{array}$$ (26)

The first term of the expression derived above, which was obtained by the changes of variables x = T_i(y), is constant with respect to the transformations, and so can be ignored in the optimization. Consequently, the problem is reduced to maximizing the following data term:

$$\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};{\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}}):={\int }_{\mathit{\Omega }}\frac{{({\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y}))}^{\mathbf{2}}}{{\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})}\mathit{dy}.$$ (27)

An important property of this data term is that composing T₁,…,T_N with a new transformation S does not change D:

$$\begin{array}{l}\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};{\mathit{T}}_{\mathbf{1}}\circ \mathit{S},\dots ,{\mathit{T}}_{\mathit{N}}\circ \mathit{S}):={\int }_{\mathit{\Omega }}\frac{{({\sum }_{\mathit{i}}({\mathit{J}}_{\mathit{i}}\circ \mathit{S})(\mathit{z}){\mathit{J}}_{\mathit{S}}(\mathit{z})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}}\circ \mathit{S})(\mathit{z}))}^{\mathbf{2}}}{{\sum }_{\mathit{i}}({\mathit{J}}_{\mathit{i}}\circ \mathit{S})(\mathit{z}){\mathit{J}}_{\mathit{S}}(\mathit{z})}\mathit{dz}\\ ={\int }_{\mathit{\Omega }}\frac{{({\sum }_{\mathit{i}}({\mathit{J}}_{\mathit{i}}\circ \mathit{S})(\mathit{z})({\mathit{I}}_{\mathit{i}}\circ {\mathbf{T}}_{\mathit{i}}\circ \mathit{S})(\mathit{z}))}^{\mathbf{2}}}{{\sum }_{\mathit{i}}({\mathit{J}}_{\mathit{i}}\circ \mathit{S})(\mathit{z})}{\mathit{J}}_{\mathit{S}}(\mathit{z})\mathit{dz}\\ ={\int }_{\mathit{\Omega }}\frac{{({\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})({\mathit{I}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{i}})(\mathit{y}))}^{\mathbf{2}}}{{\sum }_{\mathit{i}}{\mathit{J}}_{\mathit{i}}(\mathit{y})}\mathit{dy}\\ =\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};{\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}}),\end{array}$$ (28)

where J_S(z) := det ∂S(z), and y := S(z) with dy = J_S(z)dz. The shrinkage-type issues that were mentioned in Section 2.1 are alleviated because of this property of the atlas-based registration, which is thanks to the fact that the atlas and the images are compared in the physically-meaningful native image spaces, as opposed to the abstract mid-space. This property also implies the inherent independence of the data term from the mid-space, which we exploit here. We first choose the native space of one of the images – say, the first image – as a hub, and define ${\mathit{Q}}_{\mathit{i}}:={\mathit{T}}_{\mathit{i}\mathbf{1}}={\mathit{T}}_{\mathit{i}}\circ {\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}$. Note that Q₁ = Id is fixed as the identity transformation. We then rewrite the objective function with respect to T₁ and Q₂,…,Q_N. To obtain {T_ij}, it will be sufficient to solve for Q₂, …, Q_N (but not T₁), since ${\mathit{T}}_{\mathit{ij}}={\mathit{T}}_{\mathit{i}}\circ {\mathit{T}}_{\mathit{j}}^{-\mathbf{1}}={\mathit{Q}}_{\mathit{i}}\circ {\mathit{T}}_{\mathbf{1}}\circ {\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}\circ {\mathit{Q}}_{\mathit{j}}^{-\mathbf{1}}={\mathit{Q}}_{\mathit{i}}\circ {\mathit{Q}}_{\mathit{j}}^{-\mathbf{1}}$. With the new parameterization, the data term to be maximized becomes:

$$\begin{array}{l}\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};{\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}})=\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};{\mathit{T}}_{\mathbf{1}},{\mathit{Q}}_{\mathbf{2}}\circ {\mathit{T}}_{\mathbf{1}}\dots ,{\mathit{Q}}_{\mathit{N}}\circ {\mathit{T}}_{\mathbf{1}})\\ =\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};\text{Id},{\mathit{Q}}_{\mathbf{2}},\dots ,{\mathit{Q}}_{\mathit{N}}),\end{array}$$ (29)

where we used the composition property in Eq. (28). With the data term being independent of T₁,…,T_N, the mid-space disappears, thereby eliminating the problem of the mid-space drift and the need for antidrift constraints as well.

As for the Tikhonov regularization term, we define similarly to Eq. (5):

$$\stackrel{\sim }{\mathit{R}}({\mathit{T}}_{\mathbf{1}},\dots ,{\mathit{T}}_{\mathit{N}}):=\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\int }_{\mathit{\Omega }}{\Vert \mathit{\partial }\left({\mathit{T}}_{\mathit{i}}^{-\mathbf{1}}(\mathit{x})\right)-𝕀\Vert }_{\mathit{F}}^{\mathbf{2}}\mathit{dx}.$$ (30)

Choosing I₁ as a hub again, by following similar calculations as in Section 2.3.2, minimizing R̃(T₁,…,T_N) can be seen to be equivalent to maximizing the following:

$${\int }_{\mathit{\Omega }}\left({\Vert \mathit{V}(\mathit{x}){\mathit{U}}^{-\mathbf{1}}{(\mathit{x})}^{\mathit{T}}\Vert }_{\mathit{F}}^{\mathbf{2}}-{\Vert \mathit{\partial }\left({\mathit{T}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)\mathit{U}(\mathit{x})-\mathit{V}(\mathit{x}){\mathit{U}}^{-\mathbf{1}}{(\mathit{x})}^{\mathit{T}}\Vert }_{\mathit{F}}^{\mathbf{2}}\right)\mathit{dx},$$ (31)

where:

$$\begin{array}{l}\mathit{U}(\mathit{x})\mathit{U}{(\mathit{x})}^{\mathit{T}}:=\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}(\mathit{x}){\mathit{C}}_{\mathit{i}}(\mathit{x}),\\ \mathit{V}(\mathit{x}):=\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{C}}_{\mathit{i}}(\mathit{x}).\end{array}$$ (32)

${\mathit{C}}_{\mathit{i}}(\mathit{x}):={\mathit{J}}_{{\mathit{Q}}_{\mathit{i}}}(\mathit{x})\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}{(\mathit{x})}^{\mathit{T}}$ is the cofactor matrix of ∂Q_i(x), with J_Qi(x) :=det ∂Q_i(x).¹⁴ Note that diffeomorphism (J_i(x)>0) guarantees the existence of U(x) that satisfies Eq. (32), since the right hand side, ${\sum }_{\mathit{i}}\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}(\mathit{x}){\mathit{C}}_{\mathit{i}}(\mathit{x})={\sum }_{\mathit{i}}{\mathit{J}}_{{\mathit{Q}}_{\mathit{i}}}(\mathit{x})\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}(\mathit{x})\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}{(\mathit{x})}^{\mathit{T}}$, is a linear combination of positive semi-definite matrices with positive weighting, and therefore is itself positive semi-definite.

Since the data term, D(I₁,…,I_N;Id,Q₂,…,Q_N), is independent of T₁, the objective function reaches its maximum value with respect to T₁ when the second Frobenius norm in Eq. (31) equals zero, which happens for T̂₁ that satisfies $\mathit{\partial }\left({\widehat{\mathit{T}}}_{\mathbf{1}}^{-\mathbf{1}}(\mathit{x})\right)=\mathit{V}(\mathit{x}){\left(\mathit{U}(\mathit{x})\mathit{U}{(\mathit{x})}^{\mathit{T}}\right)}^{-\mathbf{1}}$. As mentioned before, we do not need the value of T̂₁, since the N − 1 transformations Q₂,…,Q_N provide us with the complete solution to group-wise registration. Thus, we proceed by assigning T₁ =T̂₁, thereby reducing the regularization term to only its first Frobenius norm, expanded as:

$$\mathit{R}({\mathit{Q}}_{\mathbf{2}},\dots ,{\mathit{Q}}_{\mathit{N}}):={\int }_{\mathit{\Omega }}\mathit{tr}\left[\left(\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{C}}_{\mathit{i}}(\mathit{x})\right){\left(\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}\mathit{\partial }{\mathit{Q}}_{\mathit{i}}^{-\mathbf{1}}(\mathit{x}){\mathit{C}}_{\mathit{i}}(\mathit{x})\right)}^{-\mathbf{1}}{\left(\sum _{\mathit{i}=\mathbf{1}}^{\mathit{N}}{\mathit{C}}_{\mathit{i}}(\mathit{x})\right)}^{\mathit{T}}\right]\mathit{dx}.$$ (33)

Therefore, registration will be performed by maximizing the following objective function:

$${\widehat{\mathit{Q}}}_{\mathbf{2}},\dots ,{\widehat{\mathit{Q}}}_{\mathit{N}}=\underset{{\mathit{Q}}_{\mathbf{2}},\dots ,{\mathit{Q}}_{\mathit{N}}}{\mathit{argmax}}\mathit{D}({\mathit{I}}_{\mathbf{1}},\dots ,{\mathit{I}}_{\mathit{N}};\text{Id},{\mathit{Q}}_{\mathbf{2}},\dots ,{\mathit{Q}}_{\mathit{N}})+\mathit{\lambda }\mathit{R}({\mathit{Q}}_{\mathbf{2}},\dots ,{\mathit{Q}}_{\mathit{N}}),$$ (34)

where λ is the positive regularization parameter. In the continuous case, this objective function – originating from Eq. (24) and Eq. (30) – is unbiased towards the choice of the hub image (here I₁). See Section 2.4 for a discussion on discretization artifacts.

Note that using the MSI group-wise registration is not helpful for template construction, since the template exists in the mid-space that is eliminated.

Appendix B. Variation of an integral that includes the Jacobian determinant

We use the following Lemma to compute the derivative of an integral term with respect to a transformation.

Lemma

For T:Ω→Ω (with Ω⊆ℝ^d), F:Ω² × R⁺ →ℝ, and J(x):=det∂T(x), let:

$$\mathit{\psi }:={\int }_{\mathit{\Omega }}\mathit{F}(\mathit{T}(\mathit{x}),\mathit{J}(\mathit{x}),\mathit{x})\mathit{dx}.$$ (35)

The variation of ψ with respect to T is:

$$\frac{\mathit{\delta }\mathit{\psi }}{\mathit{\delta }\mathit{T}}={\mathit{\nabla }}_{\mathit{T}}\mathit{F}-(\mathit{J}{\mathit{\nabla }}_{\mathit{T}}+\mathit{C}{\mathit{\nabla }}_{\mathit{x}})\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}-\frac{{\mathit{\partial }}^2\mathit{F}}{\mathit{\partial }{\mathit{J}}^2}\mathit{C}\mathit{\nabla }\mathit{J},$$ (36)

where C is the cofactor matrix of the Jacobian matrix ∂T, and ∇_T and ∇_x are vectors of gradient with respect to the elements of T and x, respectively. The notation (x) has been dropped for brevity.

Proof

We start by using the following variation formula:

$$\frac{\mathit{\delta }\mathit{\psi }}{\mathit{\delta }\mathit{T}}={\mathit{\nabla }}_{\mathit{T}}\mathit{F}-\mathit{div}{\mathit{\nabla }}_{\mathit{\partial }\mathbf{T}}\mathit{F},$$ (37)

where ∇_∂T is the matrix of the gradient with respect to the elements of ∂T, and div X := (∇^TX^T)^T. Since in Eq. (35), ∂T only appears in the Jacobian determinant, J, the chain rule implies that:

$${\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{F}=\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}{\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{J}.$$ (38)

Next we apply the product rule:

$$\mathit{div}{\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{F}=({\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{J})\mathit{\nabla }\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)+\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\mathit{div}{\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{J}.$$ (39)

According to Eq. (37), div ∇_∂TJ is the variation of −∫_ΩJ(x)dx with respect to T. The change of variables z : = T(x), with dz = J(x)dx, however, reveals that this integral is constant, −∫_ωJ(x)dx = −∫_ωdz = −|Ω|, making its variation vanish: div ∇_∂TJ = 0.

Given that ∇_∂TJ = C, the first term in the right-hand side of Eq. (39) can be expanded by the chain rule as:

$$({\mathit{\nabla }}_{\partial \mathit{T}}\mathit{J})\mathit{\nabla }\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)=\mathit{C}\mathit{\nabla }\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)=\mathit{C}{\mathit{\nabla }}_{\mathit{x}}\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)+\mathit{C}\mathit{\partial }{\mathit{T}}^{\mathit{T}}{\mathit{\nabla }}_{\mathit{T}}\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)+\frac{{\mathit{\partial }}^2\mathit{F}}{\mathit{\partial }{\mathit{J}}^2}\mathit{C}\mathit{\nabla }\mathit{J}.$$ (40)

The matrix inversion rule, $\mathit{\partial }{\mathit{T}}^{-\mathbf{1}}=\frac{\mathbf{1}}{\mathit{J}}{\mathit{C}}^{\mathit{T}}$, leads to C∂T^T = J𝕀. Therefore:

$$({\mathit{\nabla }}_{\mathit{\partial }\mathit{T}}\mathit{J})\mathit{\nabla }\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)=\mathit{C}{\mathit{\nabla }}_{\mathit{x}}\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)+\mathit{J}{\mathit{\nabla }}_{\mathit{T}}\left(\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)+\frac{{\mathit{\partial }}^{\mathbf{2}}\mathit{F}}{\mathit{\partial }{\mathit{J}}^{\mathbf{2}}}\mathit{C}\mathit{\nabla }\mathit{J}.$$ (41)

Combining Eqs. (37,39,41) proves the Lemma.

A particular use of this lemma is in compositive updates, where the following needs to be computed:

$$\mathit{\partial }{\mathit{T}}^{\mathit{T}}\frac{\mathit{\delta }\mathit{\psi }}{\mathit{\delta }\mathit{T}}=\mathit{\partial }{\mathit{T}}^{\mathit{T}}{\mathit{\nabla }}_{\mathit{T}}\left(\mathit{F}-\mathit{J}\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}\right)-\mathit{J}{\mathit{\nabla }}_{\mathit{x}}\frac{\mathit{\partial }\mathit{F}}{\mathit{\partial }\mathit{J}}-\frac{{\mathit{\partial }}^{\mathbf{2}}\mathit{F}}{\mathit{\partial }{\mathit{J}}^{\mathbf{2}}}\mathit{J}\mathit{\nabla }\mathit{J}.$$ (42)

In particular, for the weighted SSD, F(T(x),J(x),x) = (I₁(x) − (I₂ ∘T)(x))²f(J(x)), we have:

$$\mathit{\partial }{\mathit{T}}^{\mathit{T}}\frac{\mathit{\delta }\mathit{\psi }}{\mathit{\delta }\mathit{T}}=-\mathbf{2}({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})[\left(\mathit{f}(\mathit{J})-\mathit{g}(\mathit{J})\right)\mathit{\nabla }{\mathit{I}}_{\mathbf{1}}+\mathit{g}(\mathit{J})\mathit{\nabla }({\mathit{I}}_{\mathbf{2}}\circ \mathit{T})]+{({\mathit{I}}_{\mathbf{1}}-{\mathit{I}}_{\mathbf{2}}\circ \mathit{T})}^{\mathbf{2}}\mathit{\nabla }\left(\mathit{g}(\mathit{J})\right),$$ (43)

where g(J) := f(J) − Jf′(J). Letting $\mathit{f}(\mathit{J})=\frac{\mathit{J}}{\mathbf{1}+\mathit{J}}$ and $\mathit{f}(\mathit{J})=\frac{\mathbf{1}+\mathit{J}}{\mathbf{2}}$ lead to Eq. (14) and Eq. (21), respectively.

Appendix C. Variation of the regularization integral

We compute the variation of R(T) (Eq. (11)), starting from Eq. (37) with F = tr[Γ^TΛ⁻¹Γ], where Γ := ∂T−𝕀 and $\mathit{\Lambda }:=𝕀+\frac{\mathbf{1}}{\mathit{J}}\mathit{\partial }\mathit{T}\mathit{\partial }{\mathit{T}}^{\mathit{T}}$. Since F only depends on the derivative of T (but not on T itself), we have ∇_TF = 0, and:

$$\frac{\delta R(T)}{\delta T}=-\mathit{div}{\mathit{\nabla }}_{\partial T}F.$$ (44)

We now compute (∇_∂TF)_ij (the element i, j of the matrix ∇_∂TF) by deriving F with respect to ∂T_ij. For brevity, we shall fix i and j and denote the derivative of a matrix X with respect to ∂T_ij simply as X′. Using the differentiation rules for the inverse, (Λ⁻¹)′ = −Λ⁻¹Λ′Λ⁻¹, and the product, one can verify that:

$$F^{\prime }=tr{[{\mathit{\Gamma }}^T{\mathit{\Lambda }}^{-1}\mathit{\Gamma }]}^{\prime }=tr[(2{{\mathit{\Gamma }}^{\prime }}^T-{\mathit{\Gamma }}^T{\mathit{\Lambda }}^{-1}{\mathit{\Lambda }}^{\prime }){\mathit{\Lambda }}^{-1}\mathit{\Gamma }].$$ (45)

By definition, Γ′ = (∂T − 𝕀)′ = Θ^ij, where Θ^ij is the d×d matrix that has 1 as its element i, j and 0 elsewhere; i.e., (Θ^ij)_kl = δ_kiδ_lj, with δ the Kronecker delta. As for Λ′, we have:

$${\mathrm{\Lambda }}^{\prime }={\left(𝕀+\frac{1}{\mathrm{J}}\partial \mathrm{T}\partial {\mathrm{T}}^{\mathrm{T}}\right)}^{\prime }=-\frac{{\mathrm{C}}_{\text{ij}}}{{\mathrm{J}}^2}\partial \mathrm{T}\partial {\mathrm{T}}^{\mathrm{T}}+\frac{1}{\mathrm{J}}{\mathrm{\Theta }}^{\text{ij}}\partial {\mathrm{T}}^{\mathrm{T}}+\frac{1}{\mathrm{J}}\partial \mathrm{T}{\mathrm{\Theta }}^{\text{ji}},$$ (46)

where we used J′ = C_ij, with C the cofactor matrix of ∂T. By substituting for Γ, Γ′, Λ, and Λ′ in Eq. (45), while using the trace properties $\text{tr}[{\mathrm{X}}^{\mathrm{T}}\mathrm{X}]={\Vert \mathrm{X}\Vert }_{\mathrm{F}}^2$ and tr[Θ^ijX] = X_ji, Eq. (44) leads to the gradient presented in Eq. (15).