Avoiding Symmetry-Breaking Spatial Non-Uniformity in Deformable Image Registration via a Quasi-Volume-Preserving Constraint

Abstract

The choice of a reference image typically influences the results of deformable image registration, thereby making it asymmetric. This is a consequence of a spatially non-uniform weighting in the cost function integral that leads to general registration inaccuracy. The inhomogeneous integral measure – which is the local volume change in the transformation, thus varying through the course of the registration – causes image regions to contribute differently to the objective function. More importantly, the optimization algorithm is allowed to minimize the cost function by manipulating the volume change, instead of aligning the images. The approaches that restore symmetry to deformable registration successfully achieve inverse-consistency, but do not eliminate the regional bias that is the source of the error. In this work, we address the root of the problem: the non-uniformity of the cost function integral. We introduce a new quasi-volume-preserving constraint that allows for volume change only in areas with well-matching image intensities, and show that such a constraint puts a bound on the error arising from spatial non-uniformity. We demonstrate the advantages of adding the proposed constraint to standard (asymmetric and symmetrized) demons and diffeomorphic demons algorithms through experiments on synthetic images, and real X-ray and 2D/3D brain MRI data. Specifically, the results show that our approach leads to image alignment with more accurate matching of manually defined neuroanatomical structures, better tradeoff between image intensity matching and registration-induced distortion, improved native symmetry, and lower susceptibility to local optima. In summary, the inclusion of this space- and time-varying constraint leads to better image registration along every dimension that we have measured it.

1. Introduction

Image registration is a crucial step in numerous clinical and neuroscientific imaging studies involving the comparison of images, such as population investigations and longitudinal analyses. In the basic case with only two images involved, pairwise registration provides dense point-wise correspondences between voxels of the two input images. The set of such correspondences is often thought of as a transformation that takes each point in one image to the corresponding point in the other image. In that case, registration is said to align the two images; i.e., morph and overlay a moving image on a fixed image (or alternatively move both images) so that they appear similar or identical to each other. Interpreting registration as alignment requires the definition of a reference space, in which the images are aligned and compared. The choice of the reference influences the results, particularly when the transformation is assumed non-rigid (beyond merely translation and rotation) – as is necessary in most cross-subject registration applications. It is common to arbitrarily select the native space of one of the images (the space where the image is undistorted) as the reference space, hence the dependence of the results on the choice of the so-called reference image¹. It must be noted that interpreting registration as alignment – and consequently selection of a reference image/space – is intrinsically unnecessary for many applications (e.g., comparison of local cortical thickness of two brain images, where correspondences established between the two images do not necessarily represent alignment). With no image designated as the reference, swapping the two images should not affect the resulting point-wise correspondences, making pairwise registration inherently symmetric with respect to the input images. Obtaining the same results after reversing the direction of registration – known as inverse-consistency² – is therefore necessary for a pairwise registration method to be considered reliable and unbiased.³

Pairwise deformable registration is performed by maximizing some measure of similarity between the corresponding regions of the two images (for a survey, see (Sotiras et al., 2013)). Since a perfect match cannot generally be achieved due to noise and anatomical variability, typically a local image mismatch measure aggregated over the entire space is minimized. Such cost functions (CFs), with the most common example being the sum of squared difference (SSD) of image intensities, require the images to be aligned in a reference space in which the mismatch measure is integrated, thus raising the question of how to select a suitable reference space. Registration results depend on the choice of the reference as a consequence of the non-rigid (specifically, volume-changing) nature of the transformation, since, as we will see in Section 2.3, spatially uniform integration in the reference space is generally equivalent to nonuniform integration in (one or both) native spaces. In other words, for an arbitrary deformation, the integral of the mismatch measure has space-varying weightings in at least one of the native spaces of the images, with a weighting that varies depending on the deformation and the choice of the reference space. As we will see, the weighting in the CF integral is the Jacobian determinant of the transformation, which represents the local volume change in the deformation field. Assigning weighting to image regions introduces a regional bias. Furthermore, given that the weighting depends on the transformation, the optimization algorithm will drive the deformation also towards ‘lowering the weighting of the mismatched areas’, instead of only improving the alignment (e.g. by shrinking a region of one image so much as to make it almost vanish). Figure 1 presents an example of such a phenomenon, which results in a counterproductive increase in SSD in the native space of the interpolated image during registration. The dependence of the degree of expansion and shrinkage – representing the Jacobian determinant – on the choice of the reference image has also been observed by Cachier and Rey (2000), and has been reported to bias the quantification of the evolution of lesions in multiple sclerosis studies by hampering the equal retrieval of expanding and shrinking areas (Rey et al., 2002). These sources of error exist regardless of whether the reference space is chosen to be the native space of an input image or some sort of mid-space, as one or both of the native spaces of the images – which are the only physically meaningful spaces – will be integrated with regional bias.

Figure 1.

In this illustrative example, two one-dimensional images are registered by a given SSD-minimizing algorithm, with the blue image (the M-shaped intensity profile) chosen as the reference and the green one chosen as the interpolated (moving) image. The evolution of the registration is depicted from left to right, where the two rows visualize the process in the native spaces of the blue (top row) and the green (bottom row) images. Therefore, the deformations of the green (top row) and blue (bottom row) images are the inverse of each other. Given that the blue image matches better the background rather than the bright spot of the green image, the latter is shrunk in the reference space (top row). It can be seen that the SSD, while decreasing in the native space of the reference image (top row, where the CF integral is uniform), increases in the native space of the interpolated image (bottom row, where the CF integral is weighted by the volume change). The CF, being a non-uniform integral in the native space of the interpolated image, is here minimized by decreasing the weighting, even though resulting in an increase in the mismatch measure. (This toy example is provided to illustrate the disagreement on the decrease/increase in the SSD when the transformation is observed in the two native spaces, without commenting on whether the SSD-minimizing transformation here is the “expected” result. The two red curves in Figure 4 are another such example, yet using a symmetrization approach. For simplicity, we assume the regularization to be small relative to the data-attachment term.)

In standard implementation of deformable registration the reference space is commonly chosen as the native space of one of the images (say, the first image), and the results are consequently influenced by the ordering of the input images, thereby breaking the symmetry of registration. The spurious dependence of the point-wise correspondences on the choice of the reference image has been shown to be related to a bias introduced into the estimation of Alzheimer's disease effects (Fox et al., 2011; Hua et al., 2011; Thompson and Holland, 2011; Yushkevich et al., 2010). In longitudinal studies in particular, favoring one time point over another may result in errors dominating the subtle changes one seeks to measure (Reuter et al., 2012). In addition, in radiation therapy, the implication of registration asymmetry has been discussed for daily dose computation (Yang et al., 2008) and auto re-contouring (Ye and Chen, 2009). To address this issue, existing approaches primarily aim to restore inverse-consistency to registration by computing the integral in both image spaces and taking the average (Alvarez et al., 2007; Bondar et al., 2010; Cachier and Rey, 2000; Christensen and Johnson, 2001; Chui, 2001; Feng et al., 2009; Geng, 2007; Gholipour et al., 2010; Leow et al., 2007; Modat et al., 2012; Mohagheghian et al., 2010; Sabuncu et al., 2009; Tagare et al., 2009; Tao et al., 2009; Trouvé and Younes, 2000; Vercauteren et al., 2008b; Zeng and Chen, 2008; Zhang et al., 2006) or computing the integral in an abstract mid-space chosen to be “in between” the native spaces of the images (Beg and Khan, 2007; Chen and Ye, 2010; Joshi et al., 2004; Lorenzen et al., 2004; Lorenzi et al., 2013; Noblet et al., 2008; Škrinjar et al., 2008; Yang et al., 2008; Ye and Chen, 2009). Other approaches based on similar ideas have been proposed in the literature, including (Ashburner et al., 1999, 2000; Avants et al., 2008; Basri et al., 1998; Christensen and Johnson, 2003; Dedeoglu and Kanade, 2005; He and Christensen, 2003; Rogelj and Kovačič, 2006; Yanovsky et al., 2008b; Yeung et al., 2008). Although these methods are effective in making the registration invariant to the ordering of the input images, they do not alleviate the time-varying regional bias, which is the source of the problem. Indeed, inverse-inconsistency is merely a symptom of non-uniform integration of the mismatch measure on the images, and symmetrization of registration does not necessarily eliminate the underlying cause (non-uniformity of the integral) and the consequent inaccuracies introduced into the registration. Our underlying hypothesis is that the freedom for the algorithm to minimize the CF by altering the Jacobian determinant instead of improving image matching leads to suboptimal registration solutions in terms of 1) reduced image intensity matching, 2) increased (and unnecessary) distortion in the warp field, and 3) the creation of local minima that reduce the accuracy of the resulting correspondences.

In this work, instead of symmetrizing the CF, we address the root of the problem: non-uniform integrals of CFs defined on the native spaces of images. We propose to restrict the deformation such that the integrals in the native spaces of the images are (almost) unweighted, except in regions where weighting contributes (almost) no error to the CF. Our adaptive constraint – which, as we will see in Section 3.1, is quasi-volume-preserving (QVP) – keeps the deformation field away from zones that would lead to non-uniformity-induced error, by limiting the local volume changes except for regions where image intensities match well. As a result, the proposed method yields overall improvement in the alignment (Section 4) when incorporated in an SSD-like deformable registration algorithm.⁴ Furthermore, a natural consequence of the QVP constraint is that the values of the native CFs, i.e. those with uniform integral on the native space of an input image, remain arbitrarily close to each other throughout the registration. This property, which we shall name native symmetry, is a stronger form of symmetry and is improved by our method. Native symmetry implies that both native CFs agree on the progress of registration, as opposed to only one of them (as in asymmetric registration, e.g. Figure 1) or only the average of them (as in symmetrization). An additional advantage of restricting the deformation in dissimilar regions is helping to avoid entrapment of the iterative algorithm in local minima due to too much flexibility, thereby guiding it towards a good overall QVP fit before relaxing the constraints and achieving an optimum warp. This is particularly important in registration of medical images with possibly large anatomical variation. We will show improvement in registration in terms of better label alignment, better tradeoff between intensity matching and geometric distortion, native symmetry, and lower susceptibility to local optima, using two-dimensional (2D) non-diffeomorphic and three-dimensional (3D) diffeomorphic registration on several datasets.

This article extends our previous conference version (Aganj et al., 2013a). In particular, we provide a thorough theoretical justification (Section 2), more detailed description of the method (Section 3), more comprehensive experimental validation (Section ‎4), and further implementation details (appendices).

2. Spatial non-uniformity problem description

2.1. Definitions

The data attachment term of the standard asymmetric cost function (CF) that is minimized in deformable registration is defined as:

$$I_f(f,g,T):={\int }_{\mathrm{\Omega }}D(f(x),g\circ T(x))\mathrm{d}x,$$ (1)

where Ω ⊆ ℝ^d is the d-dimensional image space, f, g, Ω ↦ ℝ are two input images, T: Ω ↦ Ω is the invertible transformation with respect to which the CF is minimized, and D:ℝ² ↦ ℝ_≥0 is the (often symmetric) nonnegative intensity distance (mismatch measure); e.g., defining D (u, v) := (u − v)² results in the SSD CF.⁵ Some global regularization is also typically either added explicitly to this data-attachment CF, or performed at each iteration (see Appendix B). By searching for the transformation T that minimizes the CF, the optimization algorithm is expected to align corresponding regions of the two images.

Since image f is not deformed by T in the integral of Eq. (1), we say that the integral is represented in the native space of f. Image g, on the other hand, appears as g ∘ T in Eq. (1) and is generally deformed; however, by a simple change of variable y :=T(x), the same integral can be represented in the native space of g (see Eq. (3) below). We call I_f the f-native CF, since the integral of D is unweighted (taken uniformly) when represented in the native space of f.

2.2. Pitfalls of non-uniformity

We explain here the pitfalls of having a non-uniform (weighted) CF, which will help later to describe the key problem addressed in this paper. Let us assume that instead of Eq. (1), the following weighted version of it were minimized,

$$\underset{T}{min}{\int }_{\mathrm{\Omega }}D(f(x),g\circ T(x))W_T(x)\mathrm{d}x,$$ (2)

where W_T: Ω ↦ ℝ_≥0 is an arbitrary nonnegative weight function that generally depends on T. Allowing the image regions to contribute differently to the CF would introduce a multiplicative regional bias that would change throughout the registration process, depending on the input images and the registration parameters. More troublesome, however, would be the dependence of the bias on the transformation, thereby giving the optimization algorithm the ability to control the weighting in a counterproductive manner. More precisely, the algorithm would minimize such a weighted CF by:

1. searching for T that makes the transformed image better match the reference, f(x) ≈ g ∘ T(x), in a larger subset of Ω, and/or

2. manipulating T so as to lower the weighting W_T(x), particularly in regions where the image intensities do not match well (i.e., with larger D).

The first point in this list is the raison d'être of CF optimization, and what the registration algorithm is supposed to do. The second, however, creates a mathematical loophole in the optimization, allowing the algorithm to not only evade its main task (i.e., decreasing D), but possibly lead the deformation towards even worse alignment (by changing T in a local fashion such that D increases, but both W_T and the product D · W_T decrease). It is important to avoid regionally biasing the CF integral (unless intentionally done for a particular benefit) to prevent such negative consequences and make the algorithm focus solely on improving the alignment. Next, we show that standard deformable registration algorithms have CFs that are non-uniform (weighted) in at least one of the two native spaces, thus suffering the disadvantages described here.

2.3. Intrinsic non-uniformity in standard deformable image registration

As discussed in Section 2.2, the mismatch measure D should be integrated uniformly in order to avoid registration inaccuracy. Yet, there are two native spaces – that of f and that of g – which are equally physically meaningful. The f -native CF (Eq. (1)) satisfies uniformity in the native space of f; however, rewriting it in the native space of g via the change of variable y :=T(x) leads to the following non-uniform integral:

$$I_f(f,g,T)={\int }_{\mathrm{\Omega }}D(f\circ T^{-1}(y),g(y))/J\circ T^{-1}(y)\mathrm{d}y,$$ (3)

where the function J: Ω ↦ ℝ_>0 is the Jacobian determinant of the transformation, J (x): = det ∂T(x), which we assume to be positive (i.e., T preserves the topology). Given that J represents the relative volume change caused by the transformation and is in general inhomogeneous, we can see that the f -native CF is weighted by 1/J and does not satisfy uniformity in our second native space, i.e. that of g. The uniform CF in the native space of g, or the g-native CF, has the following form:

$$I_g(f,g,T):={\int }_{\mathrm{\Omega }}D(f\circ T^{-1}(y),g(y))\mathrm{d}y.$$ (4)

The change of variable y = T(x) again reveals that the g-native CF does not have a uniform representation in the native space of f:

$$I_g(f,g,T)={\int }_{\mathrm{\Omega }}D(f(x),g\circ T(x))J(x)\mathrm{d}x.$$ (5)

It can be seen that, regardless of which native space is picked as the reference space, the undesirable weighting of the integral shows up in at least one of the native spaces (Figure 2a-c), except for the especially restrictive cases where J(x)=1 is imposed, such as in rigid registration, or more generally volume-preserving registration (Elen et al., 2008; Haber and Modersitzki, 2007; Hirota et al., 2000; Li et al., 2009; Loeckx et al., 2005; Mansi et al., 2011; Park et al., 2011; Rohlfing et al., 2003; Saddi et al., 2007; Tanner et al., 2002; Yin et al., 2009). Volume-preserving approaches, however, make the assumption that the deformation causes no local shrinkage or expansion, which may generally not be the case.

Figure 2.

a) Two images f (top) and g (bottom) are given as input to the registration algorithm. The deformed images in an intermediate registration iteration are illustrated in: b) native space of f, c) native space of g, and d) a mid-space. The grid distortion in (b,c,d) demonstrates that no matter what space is chosen to compute the CF integral, at least one of the images is integrated non-uniformly.

The lack of simultaneous uniformity of the CFs in both native spaces, as mathematically revealed above, is an inherent imperfection of standard deformable registration. Specifically, since the weighting J(x) is the local volume change, the algorithm may attempt to enforce artificial expansion or shrinkage – depending on which image is chosen as the reference – to decrease the weighted CF without accurate alignment (see Section 2.2). It is important to stress that this is not an esoteric argument that rarely occurs in practice; instead, it is inevitable in any nonlinear registration in any region in which there is an initial mismatch (see Figure 1 for an example). In these regions a biased CF integral will lead the algorithm to attempt to reduce J(x) (along with D) as much as possible within the limits of the imposed regularization. In fact, standard registration has been shown to privilege either expanding or shrinking regions depending on the choice of the reference (Cachier and Rey, 2000; Rey et al., 2002). The added global regularization restricts this artificial volume change, but only to some extent; therefore we need to further prevent this weight factor from interfering in the data-attachment term's driving of the registration towards the desired optimal alignment.

An implication of the non-uniform weighting in exactly one of the native spaces is that in general I_f ≠ I_g. Also, since:

$$I_g(f,g,T)=I_f(g,f,T^{-1}),$$ (6)

one can see that the native CFs are inverse-inconsistent, i.e. I_f(f, g, T) ≠ I_f(g, f, T⁻¹), and I_g(f, g, T) ≠ I_g(g, f, T⁻¹). The primary approach in the literature to address this inverse-inconsistency is the minimization of the following average CF (Cachier and Rey, 2000; Christensen and Johnson, 2001; Tagare et al., 2009; Trouvé and Younes, 2000)⁶:

$$I_{\mathit{\text{ave}}}:=\frac{I_f(f,g,T)+I_g(f,g,T)}{2}.$$ (7)

Substituting for I_g from Eq. (6), one can see that I_ave is a symmetrized version of either of the native CFs and is therefore inverse-consistent, i.e. I_ave(f, g, T) = I_ave(g, f, T⁻¹). However, by rewriting it in the native spaces of the two images, once using Eqs. (1,5) and again using Eqs. (3,4),

$$I_{\mathit{\text{ave}}}(f,g,T)={\int }_{\mathrm{\Omega }}(f(x),g\circ T(x))\frac{1+J(x)}{2}\mathrm{d}x={\int }_{\mathrm{\Omega }}D(f\circ T^{-1}(y),g(y))\frac{1+1/J\circ T^{-1}(y)}{2}\mathrm{d}y,$$ (8)

it turns out that I_ave is a weighted integral in both native spaces (Figure 2d), and so the registration still incurs the regional bias errors (Section 2.2), in spite of its inverse-consistency.

In an alternative approach to restore symmetry, the mismatch error is integrated uniformly in an abstract mid-space, chosen to be equally close to both native spaces (Avants et al., 2008; Beg and Khan, 2007; Lorenzen et al., 2004; Noblet et al., 2008). Provided that the mid-space is chosen “in between” the two native spaces, changing the ordering of the input images will not affect the CF, hence the inverse-consistency. Nevertheless, it can be easily shown similarly as above that rewriting such a mid-space CF in the native spaces of the images, once again, results in the Jacobian determinant of the transformation (that takes the mid-space to the native space) to appear as a weighting in the native-space representations of the CF. Therefore, the integral of such a mid-space CF is non-uniform in both of the physically meaningful native spaces, and only uniform in the virtual mid-space, which is physically nonexistent and possibly changing during the registration process. As a consequence, the problems associated with the regional bias (Section 2.2) remain unresolved in these approaches. For instance, since the mid-space and the weighting often depend on T, the optimization algorithm has the ability to alter the mid-space so as to decrease the CF without necessarily making the two images more similar. An example of this behavior is when the algorithm changes T in such a way that regions with mismatching image intensities are shrunk in the mid-space. This results in a decrease in the (mid-space) CF but not necessarily in the native CFs, since the images will look more similar in the mid-space but not necessarily similar in the two native spaces. Constraints that are used to keep the mid-space “in between” the native spaces of the two images usually prevent such regions from totally shrinking, but depending on the strength of these constraints, partial shrinkage may still happen.

It should be noted that minimizing I_ave or a mid-space CF results in transformations that are not necessarily minimizers of the native CFs, and for which, I_f and I_g are not necessarily equal. Here, we define the native symmetry property of an image registration algorithm, as the quality of producing transformations minimizing the native CFs, while keeping them equal (I_f= I_g). Rigid, and more generally any volume-preserving registration is natively symmetric (disregarding the asymmetry due to resampling artifacts (Aganj et al., 2013b; Reuter et al., 2010; Sabuncu et al., 2009)), since with J(x) = 1 everywhere, the two native CFs (Eqs. (1,5)) are always equal. Native symmetry is a stronger form of symmetry, indicating that both native CFs agree with the progress of registration. Neither asymmetric (minimizing either I_f or I_g) nor symmetrized (minimizing I_ave or a mid-space CF) deformable registration is guaranteed to be natively symmetric.

3. Proposed methods

3.1. Quasi-volume-preserving constraint

In this work we introduce a constraint on the deformation field that keeps the non-uniformity error in the CF arbitrarily small. Examining Eqs. (3,5,8), we can see that the condition J(x) = 1, ∀x ∈ Ω would be sufficient for I_f, I_g, and I_ave to be free of non-uniformity error. Nevertheless, such a restrictive volume-preserving constraint is not a necessary condition, as J can indeed be different from where the mismatch measure D is zero, and no error will yet arise from non-uniformity. Accordingly, by equating the integrand to its unweighted version, i.e. D(f(x),g ∘ T(x))J(x) = D(f(x),g ∘ T(x)), the following less restrictive constraint (here written in the native space of f) can also be seen to guarantee the elimination of the non-uniformity error:

$$D(f(x),g\circ T(x))(J(x)-1)=0,\forall x\in \mathrm{\Omega }.$$ (9)

Nonetheless, in practice, exact intensity matching, D(f(x),g ∘ T(x)) = 0, and absolute volume preservation, J(x) = 1, are seldom achieved. Thus, to allow for large deformations, particularly in the presence of noise and poor intensity match, here we propose the following relaxation of the above constraint, which we call the quasi-volume-preserving (QVP) constraint:

$$D(f(x),g\circ T(x))|J(x)-1|<\epsilon ,\forall x\in \mathrm{\Omega },$$ (10)

where the constant ε is a positive threshold on the local non-uniformity error. In fact, the prefix quasi- indicates the fact that volume preservation is imposed only in regions with disagreeing image intensities, as well as the relaxation of Eq. (9) to Ineq. (10). Roughly speaking, for a small ε, the QVP constraint ensures that there is almost no volume change (J(x) ≅ 1), except for regions where the image intensities almost match (D(f(x),g ∘ T(x)) ≅ 0).⁷ The performance of enforcing the QVP constraint with different values of ε has been evaluated in 3D brain image registration in Section 4.2. As we shall see, smaller values of ε, which further restrict the deformation, actually lead to improvements in the overall registration quality, indicating that the QVP constraint guides the registration away from problematic regions in the energy landscape, yielding better CF optima. A higher value for ε, however, might be needed to accommodate larger deformations and regions with poorer intensity match and higher noise (e.g., in longitudinal brain imaging when a tumor both grows and changes intensity).

By constraining the deviation of the volume change J(x) from as in the QVP inequality (10), the total non-uniformity error in the native CFs will be bounded by ε|Ω| (with |Ω| the size of the space), which can be made arbitrarily small by choosing a small enough ε.⁸ Remaining in this (almost) error-free QVP zone ensures that the optimization algorithm is not misled by a biased CF gradient that can potentially steer it away from the global minimum, for example, by encouraging complex warps in regions where no matching of image intensities is possible. In addition, by combining Eqs. (1,5,10), the error in native symmetry can similarly be seen to be bounded:

$$|I_f-I_g|=|{\int }_{\mathrm{\Omega }}D(f(x),g\circ T(x))(J(x)-1)\mathrm{d}x|\le {\int }_{\mathrm{\Omega }}D(f(x),g\circ T(x))|J(x)-1|\mathrm{d}x<\epsilon |\mathrm{\Omega }|,$$ (11)

with I_ave being close to both native CFs (using Eqs. (7,11)):

$$|I_{\mathit{\text{ave}}}-I_{f/g}|<\frac{\epsilon |\mathrm{\Omega }|}{2}.$$ (12)

The local nature of the QVP constraint also ensures regional native symmetry; i.e., satisfying Ineq. (10) keeps native CFs defined in any local Ω′⊂Ω within ε|Ω′| of each other.⁹

By keeping the integrals of the CFs (almost) uniform in the native spaces of both images except for regions where non-uniformity barely perturbs the CF, the QVP constraint ensures that CF minimization is uniquely aimed at better alignment, and that no regional bias irrelevant to registration can be exploited to minimize the CF (Section 2.2).

3.2. Adaptive regularization interpretation

Enforcing the QVP constraint restricts J(x) to remain close to 1 in regions where the difference between the two images is large, but relaxes this restriction where the image intensities are similar. An additional practical advantage of the proposed method is its role as adaptive spatially-variant regularization (Cahill et al., 2009; Freiman et al., 2012; Glocker et al., 2009; Loeckx et al., 2004; Ou et al., 2011; Pace et al., 2013; Risholm et al., 2009; Staring et al., 2007; Stefanescu et al., 2004; Suárez et al., 2002; Tang et al., 2010). By restricting the deformation in regions where there is not a good match and letting the surrounding area to mostly drive the displacement, the QVP constraint allows the deformed image to “pass through” regions of poor match with restricted volume change, to potentially find regions where a good registration is possible, after which the constraint is naturally relaxed. Intuitively, the algorithm avoids fitting noise in regions where a good overall match cannot be found. Examples of such behavior are provided in sections 4.1.1 and 4.1.2.

The factor D in Ineq. (10) is the major contrast between our proposed approach and the volume-preserving methods in the literature (Section 2.3); our algorithm preserves volume only in regions where it finds no good match between the images. This means that the transformation may still contain compression and expansion in areas where the two images are locally similar, allowing good matching in regions where it is feasible. Although the QVP constraint is automatically relaxed in such similar-intensity regions, the inclusion of the global regularization still keeps the deformation field regular.¹⁰

3.3. Incorporation of the QVP constraint

The QVP constraint, which can be seen as a restriction on the feasible region of the optimization search space, can be applied to any deformable registration tool that optimizes an aggregate objective function, i.e. sum of a local similarity measure (such as SSD). To enforce the QVP inequality (10), at each iteration of the main registration algorithm we reduce the non-uniformity error:

$$e(x):=D(f(x),g\circ T(x))|J(x)-1|,$$ (13)

until e(x) < ε for all x ε Ω. To that end, e(x) may be directly reduced by iteratively minimizing the integral ∫_Ω R(e(x) − ε)dx to zero, with R(t) :=(t + |t|)/2 the ramp function, thereby satisfying e(x) < ε for all x. The dependence e(x) of on both T(x) and its derivative J(x), however, makes this direct minimization complicated.¹¹ Thus, for simplicity and numerical stability, we choose a different approach to take T(x) to the QVP space.

To estimate the derivative e(x) of with respect T(x) to and moving T(x) in its descent direction, we first note that e(x) is the product of two factors: the mismatch measure D(f(x),g ∘ T(x)) and the Jacobian deviation |J(x)−1|. According to the product rule, d(uv) = vdu + udv, the derivative e(x) of has two terms, in each of which only one of the two factors is derived. The first term has the derivative of the mismatch measure D(f(x),g ∘ T(x)) and is a gradient term similar to that of the main optimization algorithm, simply leading to additional CF minimization. Since the mismatch measure is already minimized by the main optimization algorithm, we do not include this image-aligning term in the QVP constraining step,¹² and rather focus on the second term of the derivative.

To reduce the Jacobian deviation factor |J(x)−1| of e(x), whose derivative appears in the second term of the derivative of e(x), we take advantage of the fact that the descent direction for the following measure of regularity (where K(x) is a weighting function),

$${\int }_{\mathrm{\Omega }}K(x){\Vert \partial T(x)-\mathbb{I}\Vert }_F^2\mathrm{d}x,$$ (14)

is the well-known inhomogeneous diffusion, applied to the displacement field,

$$T_{t+1}(x)=T_t(x)+\gamma \mathrm{\nabla }·[K(x)(\mathrm{\nabla }{T}_t(x)-\stackrel{\rightharpoonup }{1})],$$ (15)

where t, γ, K(x), and $\stackrel{\mathbf{\rightharpoonup }}{\mathbf{1}}$ are the diffusion iteration number, the step size, the space-variant diffusion coefficient, and the vector of all 1s, respectively. Therefore, the above diffusion process reduces the deviation of the Jacobian ∂T from the identity matrix (the higher K(x), the more strongly). We have shown in Appendix C that such small updates, while bringing the Jacobian ∂T closer to identity, also make the Jacobian determinant J asymptotically closer to 1. Since we want the diffusion to be much stronger in the areas with higher error, we make the diffusion coefficient grow quadratically with e(x) as follows:

$$K(x):=\alpha e^2(x).$$ (16)

It is important for K(x) to be a smooth function of e(x), as its derivative is computed in the diffusion process. We perform the inhomogeneous diffusion of Eq. (15) after both the main gradient-descent/Newton iteration and the standard demons regularization have been performed. To speed up the diffusion process, we apply a number of (10 in our experiments) iterations of diffusion, then update e(x) and K(x) with the new T(x), and if the QVP constraint e(x) < ε,∀x ∈Ω is not yet satisfied we repeat with further diffusion.¹³ The stopping criterion (satisfaction of QVP) makes this operation idempotent.

As we adopt the demons deformable registration (Thirion, 1998) in this work, the inhomogeneous blurring of the displacement field (primarily in areas with higher e(x)) at each registration iteration can be seen to be related to the Gaussian blurring (equivalent to homogeneous diffusion, see Appendix B) in the demons method, which blurs the displacement field at each iteration. However, a major contrast here is that the (space-variant) blurring depends on the mismatch measure D(f(x),g ∘ T(x)) that includes image data, since both (i) the spatial weighting of the diffusion K(x), and (ii) the important stopping criterion for the diffusion, i.e. the QVP inequality (10), depend on D(f(x),g ∘ T(x)).

We chose α = 70 in our experiments. Furthermore, to keep the diffusion stable, we choose the diffusion step size γ to be at most 2^−(d+1) (Johnson et al., 2014), and threshold K(x) so it remains below a certain value (2 in our experiments). Also to make sure that local noise does not result in an over restrictive constraint, we denoise K(x) by a Gaussian low-pass filter with the standard deviation of 1 voxel, before using it in the diffusion step.

4. Results and discussion

To show that the proposed QVP constraint can be incorporated into and improve different implementations of deformable registration techniques, we validate it on both demons (in Matlab) and diffeomorphic demons (in Insight Toolkit (ITK)) registration. In all of the experiments, the image intensity values were initially scaled to be between 0 and 1.

4.1. Demons registration of 2D images

We first validate our method using 2D (non-diffeomorphic) demons registration by comparing the performance of the following three approaches: minimization of I_f, minimization of I_ave, and minimization of I_ave with the proposed QVP constraint (with the heuristically-determined optimum ε). Since with no explicit diffeomorphism constraints the topology is not necessarily preserved, for each method and experiment, we choose the optimum general demons regularization parameter λ (see Appendix B for definition) heuristically to achieve the best convergence with no observable topology break. For each experiment, we plot I_f and I_g with respect to the iteration number to assess the registration native symmetry by the extent to which these two native CFs agree with each other. We evaluate this throughout all the iterations, as opposed to only at the convergence, since this error is an aggregate measure of non-uniformity, which in turn can cause inaccurate gradient descent directions at every iteration and lead the optimization algorithm to undesirable solutions.¹⁴ In these non-diffeomorphic experiments, we also evaluate the overall results by the number of iterations that each algorithm takes to converge, and the best achievable ultimate CF value that preserves the topology, which is an indicator of the CF-distortion tradeoff. The experiments in this subsection are performed using our unidirectional Matlab implementation of CF minimization (Appendix A) and global regularization (Appendix B), meaning that only the forward CF needs to be explicitly evaluated.

4.1.1. Synthetic data

We first compare the three methods on synthetic data, by registering two letters ‘B’ with noticeably different shapes. As the plot in Figure 3 demonstrates, minimizing I_f (dark and light green) results in a relatively large difference I_g − I_f > 0, which is expected, since I_g is not in any form accounted for in the optimization. Minimizing I_ave (dark and light red) reduces the gap between I_f and I_g, as equal weight is given to both of them in the optimization. Nevertheless, this approach – not being natively symmetric – does not guarantee the equality of the two native CFs. The proposed QVP approach with ε = .2 (dark and light blue), however, enforces a bound on |I_f − I_g| and therefore, as illustrated in the plot, produces transformations on the cost of which both native CFs agree (within the error margin ε). In addition, this method converges at an earlier gradient descent iteration (∼Itr. 750, see the orange rectangles in Figure 3 (top)) compared to the two others (∼Itr. 1800), while resulting in lower final native CF values than the standard asymmetric and the symmetrized approaches do.¹⁵

Figure 3.

Registration of synthetic data by a) minimizing I_f, b) minimizing I_ave, and c) minimizing I_ave with the proposed QVP constraint. Top: Cyan and magenta indicate respectively the reference and the interpolated images, with blue being their intersection. Middle/Bottom: The native CFs for each method, and a zoomed version.

The better convergence of the QVP method may be related to the fact that its optimum general regularization parameter λ = .2 is naturally lower compared to the other two techniques (λ = .3 and λ = .7 for the standard and the symmetrization methods), given that it also performs a separate inhomogeneous regularization (Eq. (15)). This method allows the transformation to freely absorb and compress one ‘B’ into the other, since the corresponding volume change happens in regions where the intensities match, for which the inhomogeneous regularization, and consequently the total regularization is weak. The inhomogeneous regularization, in contrast, prevents a topology break by regularizing more aggressively in areas where the difference between the two images (and therefore the SSD force) is large. Conversely, the other two methods need a higher λ to avoid topology break, and yet apply this strong general regularization to the entire image (including regions with matching intensities and no gradient force), hence a slower convergence and higher final CF (worse CF-distortion tradeoff).

4.1.2. X-ray images

Next, we compared the three methods on a pair of public jaw X-ray images acquired before (Figure 4a) and after (Figure 4b) an orthognathic surgery. Prior to deformable registration, high-resolution versions of the two images were superimposed, the upper jaws were rigidly aligned manually, and the images were low-pass-filtered and resampled to the lower resolution that is shown in Figure 4. As in the previous case, minimization of the standard asymmetric forward CF, I_f (Figure 4, the two green curves), soon creates a considerable gap between the native CFs. Minimizing I_ave (Figure 4, the two red curves) results in a steady optimization of the average of the native CFs. However, since the native CFs are not individually considered, not only does the gap between them still mostly exist, but a jump that abruptly changes them – yet not their average – occurs around iteration 2000, which is possibly a sign of the potential susceptibility of the algorithm to local optima. In fact, an optimization algorithm might decide that it has converged and stop the procedure prematurely before iteration 2000, thus leaving the teeth region misaligned (as was actually the case here; the misalignment was observed in the deformed image for iterations before 2000). This is another example (similar to Figure 1) showing a transformation that decreases the SSD in one native space but increases it in the other native space, despite the fact that a symmetrized CF has been used. Conversely, the proposed QVP method with ε = .2 (Figure 4, the two blue curves) produces the smallest native CF gap for iterations before 2000, while obtaining the lowest final CF values (best CF-distortion tradeoff) compared to the standard asymmetric and the symmetrized approaches. The warped image obtained by the QVP approach is depicted in Figure 4c, along with the computed deformation field. Results from the two other methods were visually similar, except for the fact that the displacement in the upper jaw, which is supposed to be small because of the initial rigid alignment of the upper jaws, was lowest in the deformation obtained by the QVP approach (2.5 pixels, averaged in an upper jaw mask), and therefore least affected by the general regularization, compared to the standard (3.9 pixels) and the symmetrization (3.2 pixels) approaches. This is likely again due to the lower value of λ required by QVP, as explained in Section 4.1.1.

Figure 4.

Top: X-ray images taken a) before and b) after surgery, with c) registration results using the proposed QVP approach. Middle/Bottom: Native CFs for the three methods, and a zoomed version.

4.1.3. Brain images

We also tested the three algorithms on the mid-sagittal planes of 20 brain images taken from the publicly available OASIS database (Marcus et al., 2007), which we pre-processed in FreeSurfer (Dale et al., 1999; Fischl, 2012; Fischl et al., 1999). The intensity-normalized and resampled volumes (1-mm³ isotropic voxel size) were made upright by robust rigid registration (Reuter et al., 2010) of each volume to its left-right mirrored version. The sagittal slice located four voxels to the right of the mid-sagittal plane was extracted from each volume, and to adjust for any nodding rotation, was rigidly registered to that slice of the first volume, and resampled to the size 128×128. Out of the 20 sagittal slices corresponding to the 20 subjects, the one closest (in L₂ norm) to the rest was chosen as the reference, and non-rigidly registered to the other 19 subjects individually. For each subject and method, we ran the registration with 21 different values for general regularization parameter λ. We observed throughout our experiments that a lower value of λ that still preserved the topology consistently resulted in a lower final CF. Therefore, in order for each algorithm to present its best CF-distortion tradeoff, we inspected the results (19×3×21=1197 images), and for each subject and method we chose the one obtained with the lowest λ containing no visible topology break. The optimally chosen λ was the same for the symmetrization and the QVP (ε = .17) methods in 13 subjects; however, for the other 6 subjects, QVP passed the visual inspection test at a lower λ (see Figure 5 for examples). We computed the native asymmetry error as the mean absolute value of the difference of the two native CFs through all iterations. When averaged across subjects, this error was 161% and 16% higher for the asymmetric and the symmetrization techniques, respectively, compared to the proposed QVP approach. We also compared the final values of the native CFs, (I_f, I_g), among the methods, which were (30%, 56%) and (5%, 4%) higher for the asymmetric and the symmetrization approaches, respectively, compared to the QVP method. We hypothesized that the QVP approach results in a better CF-distortion tradeoff by producing ultimate native CF values (for the lowest λ preserving the topology) that are lower than those of the two other techniques. A left-tailed Student's t-test with a .05 significance level rejected the null hypothesis when comparing the QVP with both the asymmetric (p = 10⁻⁶) and the symmetrization (p = .02) approaches.

Figure 5.

a) QVP registration results on four subjects with optimal values of λ. b) Results of I_ave minimization using the same λ as in (a). Note the distortion due to under-regularization in the encircled areas. c) Results of I_ave minimization using optimal (higher) values of λ, thus with higher CF values.

4.2. Diffeomorphic demons registration of 3D brain images

Next, we examined the effects of QVP constraining on the accuracy of registration using 3D labeled data, by incorporating the QVP constraint into the diffeomorphic demons registration module (Vercauteren et al., 2008a) of ITK. We randomly selected 50 pairs of images, with each image chosen from a set of 40 3D brain volumes that had 37 neuroanatomical structures manually labeled (Fischl et al., 2002). These pairs were registered with and without the QVP constraint for various values of λ (here the global regularization parameter of diffeomorphic demons) for 50 Newton iterations (enough for convergence). When QVP was used, four values of ε = .05, .1, .2, .3 were tried (with ε = .05 tried only for three values of λ because of its high computational cost).¹⁶

As illustrated in Figure 6, the inclusion of the QVP constraint consistently improved the results in every way that we quantified them. 1) QVP resulted in better alignment of manual labelings, i.e. lower number of differently-classified corresponding voxels (Figure 6, top). We then considered the CF-distortion tradeoff in the results. 2) QVP achieved a lower final CF value, despite the fact that it further regularizes the deformation field (Figure 6, 2^nd row). This provides evidence that the QVP constraint leads the algorithm into a better optimum. 3) QVP resulted in transformations with lower distortion in terms of the elastic energy, computed following (Hagemann et al., 1999; Yanovsky et al., 2008a), and the number of voxels with Jacobian determinant values smaller than 10⁻³ or larger than 10³ (Figure 6, two bottom rows). This improvement is expected, as QVP increases the regularization.

Figure 6.

Adding the QVP constraint to the diffeomorphic demons algorithm results in improved alignment of manual labelings (top), better intensity matching (2^nd row), and less distortion (two bottom rows). Quantities are averaged across 50 experiments, and the error bars indicate the standard error of the mean. This is the optimum range for λ; we tried higher values of it, which degraded the performance of both methods, with QVP still outperforming.

Figure 7 depicts the CF-distortion tradeoff for the experiments.¹⁷ Indeed, a tighter QVP constraint (smaller ε) always improved both the label matching and the CF-distortion tradeoff. (This is true for ε = .05 as well, even though it did not reduce the final CF as much as ε = .1.) Disabling the global regularization (λ = 0) did not produce the lowest CF, indicating that too much flexibility increases the chances of being trapped in a local optimum. We stress that while it is impossible to exactly match the amount of regularization in the QVP and non-QVP case, in all of the aforementioned registration experiments with a wide range of regularization levels the incorporation of the QVP constraint yielded better accuracy and image matching, and lower levels of distortion.

Figure 7.

CF-distortion tradeoff in diffeomorphic registration experiments. Each curve corresponds to a unique value of ε, and each point on the curve represents a different value of λ. For each point on the curve, the CF and distortion values have been averaged across subjects.

In practice, there may be a limit to the improvement that can be achieved by lowering ε. In particular, the number of necessary diffusion iterations (for each Newton iteration) goes to infinity as ε approaches zero, and for the extreme choice of ε = 0 the QVP constraint is never satisfied. In our experiments, an order of 1, 10, and 100 rounds of diffusion were needed for ε = .3, .2, ε = .1,and ε = .05, respectively, to satisfy QVP for all the voxels in the image.

Since the non-uniformity error is the culprit in breaking the symmetry of registration, it is expected that limiting this error by QVP constraining would make the registration more inverse-consistent. For the representative regularization value of λ = .1, we repeated the experiments above by swapping the order of input images and registering them again once without QVP, and a second time with QVP (ε = .3). We then computed the inverse-consistency error as ${\int }_{\mathrm{\Omega }}{\Vert T_1\circ T_2(x)-x\Vert }_2^2\mathrm{d}x/|\Omega |$, where T₁ and T₂ are the two transformations achieved by forward and backward registrations. As demonstrated by the histogram in Figure 8, in all of the experiments, enforcing the QVP constraint reduced the inverse-consistency error (mean ± SD: 5.2% ± 3.3%). A paired left-sided Wilcoxon signed rank test revealed the significance value of p = 3 × 10⁻⁸.

Figure 8.

The histogram of the decrease in inverse-consistency error by including the QVP constraint (ε = .3) confirms that reducing the non-uniformity error results in improvement in inverse-consistency.

5. Conclusions

We have proposed a quasi-volume-preserving (QVP) constraint on the deformation field in deformable registration that keeps the optimization algorithm away from the regions violating the uniformity of the cost function integral and consequently the symmetry. The main focus of this work is addressing the inaccuracies arising from the nonuniformity of the cost functions, which is the major distinction between our method and the symmetrization and mid-space techniques. In that regard, we have shown theoretically and through experiments that the QVP constraint improves, not only native symmetry, but the overall registration results. Enforcing the QVP constraint led to cross-subject registration with increased accuracy in the alignment of manually defined neuroanatomical structures, better tradeoff between the intensity error and distortion in the warp field, improved native symmetry, and reduced susceptibility to local optima. Subjects of the future research include: intrinsic enforcement of QVP via specific parameterization of the deformation field (without the need for blurring the displacement field), optimal ε determination based on image properties, proving the convergence and further stabilizing the diffusion step, and assessing the incorporation of QVP in the mid-space approaches.

Highlights.

• We propose a quasi-volume-preserving (QVP) constraint in non-rigid registration.

• QVP bounds the cost function nonuniformity error, thereby improving the alignment.

• We show results using 2D demons and 3D diffeomorphic demons registration.

• More accurate matching of manually defined labels is one of the improvements.

Appendix A. Unidirectional minimization of the average cost function

We minimize the average CF (Eq. (7)) in the experiments of Section 4.1 via gradient descent in the same unidirectional manner as in (Leow et al., 2007), with differences that are outlined here. Basically, we calculate the variation of the average CF with respect to the transformation, δI_ave/δT(x), and update T(x) towards its opposite (descent) direction.

I_ave is the sum of two terms, with the first term, I_f/2, being written in terms of T(x) as in Eq. (1). The variation of I_f with respect to T(x) is straightforward to compute:

$$\frac{\delta I_f}{\delta T(x)}=D_2(f(x),g\circ T(x))(\mathrm{\nabla }g)\circ T(x),$$ (17)

where D_k is the partial derivative of D with respect to the k^th argument, and ∇ is the image gradient operator. The second term of I_ave is I_g/2, which is written in terms of T⁻¹(y) as in Eq. (4) and, yet, is not an explicit function of T(x) ¹⁸ Here, instead of direct inversion of T, we compute the variation of I_g with respect to T(x) indirectly. First we note that the variation of I_g with respect to T⁻¹(y) can be computed similarly to Eq. (17), as:

$$\frac{\delta I_g}{\delta T^{-1}(y)}=D_1(f\circ T^{-1}(y),g(y))(\mathrm{\nabla }f)\circ T^{-1}(y).$$ (18)

Let us assume that a small variation ΔT (x) changes the transformation as T*(x) = T(x) + ΔT(x). This change results in a small variation in T⁻¹(y), as T⁻¹*(y) = T⁻¹(y) + ΔT⁻¹(y). Keeping the first-order terms, the relationship T^−1* ∘ T* (x) = x leads to:

$$\mathrm{\Delta }{T}^{-1}(T(x))=-\partial T{(x)}^{-1}\mathrm{\Delta }T(x),$$ (19)

where ∂T(x) is the Jacobian matrix of T(x) (Leow et al., 2007). Furthermore, by definition, I_g will vary as follows:

$$\mathrm{\Delta }{I}_g={\int }_{\mathrm{\Omega }}{\left[\frac{\delta I_g}{\delta T^{-1}(y)}\right]}^{\mathrm{T}}\mathrm{\Delta }{T}^{-1}(y)\mathrm{d}y.$$ (20)

Making the change of variable y = T(x) and substituting for ΔT⁻¹(T(x)) from Eq. (19) gives:

$$\mathrm{\Delta }{I}_g={\int }_{\mathrm{\Omega }}{(-J(x)\partial T{(x)}^{-1^{\mathrm{T}}}{\left[\frac{\delta I_g}{\delta T^{-1}(y)}\right]}_{y=T(x)})}^{\mathrm{T}}\mathrm{\Delta }T(x)\mathrm{d}x,$$ (21)

with J(x) := det ∂T(x). The variation of I_g with respect to T(x) is therefore derived as:

$$\frac{\delta I_g}{\delta T(x)}=-C(x){\left[\frac{\delta I_g}{\delta T^{-1}(y)}\right]}_{y=T(x)},$$ (22)

where C(x) = J(x)∂T(x)^−1T is the cofactor matrix of ∂T(x), defined as C_ij = (−1)^i+jM_ij with M the minor of ∂T (i.e., M_ij is the determinant of the matrix that results from deleting row i and column j of ∂T). Substituting from Eq. (18),

$$\frac{\delta I_g}{\delta T(x)}=-D_1(f(x),g\circ T(x))C(x)\mathrm{\nabla }f(x),$$ (23)

and combining Eqs. (7,17,23), the variation I_ave of with respect to T(x) is derived as follows:

$$\frac{\delta I_{\mathit{\text{ave}}}}{\delta T(x)}=\frac{D_2(f(x),g\circ T(x))(\mathrm{\nabla }g)\circ T(x)-D_1(f(x),g\circ T(x))C(x)\mathrm{\nabla }f(x)}{2}.$$ (24)

For instance, in case of the SSD CF, i.e. D(u, v) := (u − v)², Eq. (24) reduces to:

$$\frac{\delta I_{\mathit{\text{ave}}}}{\delta T(x)}=-(f(x)-g\circ T(x))(C(x)\mathrm{\nabla }f(x)+(\mathrm{\nabla }g)\circ T(x)).$$ (25)

Note that the descent direction for I_ave was here computed by deriving both its terms –I_f and I_g – consistently with respect to T(x). On the contrary, in (Leow et al., 2007), the descent directions are computed separately for I_f and I_g with respect to T(x) and T⁻¹(y), respectively, with the latter direction being subsequently translated in terms of change in T(x).¹⁹

Next, we explain how to incorporate the demons regularization symmetrically.

Appendix B. Unidirectional symmetric demons regularization

In addition to QVP, we used a global regularization in the experiments, which is an essential part of the demons registration (Thirion, 1998). In our 2D implementation in Section 4.1, we included a general Tikhonov term $½{\Vert \partial T(x)-\mathbb{I}\Vert }_F^2$ (weighted by λ, with the identity matrix) inside the integral of the f-native CF (Eq. (1)), whereas in Section 4.2 we used ITK's built-in regularization. The variation of $R_f:={\int }_{\mathrm{\Omega }}½{\Vert \partial T(x)-\mathbb{I}\Vert }_F^2\mathrm{d}x$ with respect to T(x) can be derived as:

$$\frac{\delta R_f}{\delta T(x)}=-{\mathrm{\nabla }}_x^{2}T(x),$$ (26)

where ${\mathrm{\nabla }}_x^2$ is the Laplacian with respect to x performed on each element of T. Updating the deformation field in the opposite direction of this variation is equivalent to solving the diffusion equation, which in turn results in a Gaussian blurring in the displacement field, i.e. what is done directly in the original demons algorithm.²⁰ The average CF in Eq. (7), however, includes the g-native CF (Eq. (4)), which also needs such a term as $R_g:={\int }_{\mathrm{\Omega }}½{\Vert \partial T^{-1}(y)-\mathbb{I}\Vert }_F^2\mathrm{d}y$ for the symmetry to be preserved. To minimize this term unidirectionally, we need to compute its variation with respect to T(x). As in Appendix A, we first compute its variation with respect to T⁻¹(y),

$$\frac{\delta R_g}{\delta T^{-1}(y)}=-{\mathrm{\nabla }}_y^2{T}^{-1}(y),$$ (27)

and then derive its variation with respect to T(x) using the change of variable y = T(x) and Eq. (22), as:

$$\frac{\delta R_g}{\delta T(x)}=-C(x){\left[\frac{\delta R_g}{\delta T^{-1}(y)}\right]}_{y=T(x)}=C(x){\left[{\mathrm{\nabla }}_y^{2}x\right]}_{y=T(x)}.$$ (28)

So the problem boils down to the computation of ${\mathrm{\nabla }}_y^{2}x={\mathrm{\nabla }}_y·{\mathrm{\nabla }}_{y}x$, represented in x. The gradient of x with respect to y is simply the inverse Jacobian matrix:

$${\mathrm{\nabla }}_{y}x=\partial T{(x)}^{-1},$$ (29)

which we shall denote by Q :=∂T⁻¹.²¹ We now compute the divergence of the above:

$${\mathrm{\nabla }}_y·{\mathrm{\nabla }}_{y}x=\sum _k\frac{\partial Q_{:,k}}{\partial y_k},$$ (30)

where the subscript (:, k)means the k^th column of the matrix. The chain rule implies:

$$\frac{\partial }{\partial y_k}=\sum _l\frac{\partial x_l}{\partial y_k}\frac{\partial }{\partial x_l}=\sum _l{Q}_{l,k}\frac{\partial }{\partial x_l},$$ (31)

and combining Eqs. (30,31) leads to:

$${\mathrm{\nabla }}_y^{2}x=\sum _{k,l}\frac{\partial Q_{:,k}}{\partial x_l}{Q}_{l,k}=\sum _l{\left[\frac{\partial Q}{\partial x_l}{Q}^{\mathrm{T}}\right]}_{:,l}.$$ (32)

We now compute $\frac{\partial Q}{\partial x_l}$ by taking the derivative of the relationship Q∂T = as follows:

$$\frac{\partial }{\partial x_l}(Q\partial T)=0,$$ (33)

$$\frac{\partial Q}{\partial x_l}\partial T+Q\frac{\partial }{\partial x_l}\partial T=0,$$ (34)

$$\frac{\partial Q}{\partial x_l}=-Q\frac{\partial }{\partial x_l}\partial T\partial T^{-1}=-Q\frac{\partial }{\partial x_1}\partial TQ.$$ (35)

Equation (32) thus becomes:

$${\mathrm{\nabla }}_y^{2}x=-\sum _l{(Q\frac{\partial }{\partial x_l}\partial TQ{Q}^{\mathrm{T}})}_{:,l}=-Qh,$$ (36)

with the vector h defined as:

$$h:=\sum _l{(\frac{\partial }{\partial x_l}\partial TQ{Q}^{\mathrm{T}})}_{:,l}.$$ (37)

The n^th element of h can be calculated as follows:

$$h_n=\sum _l{(\frac{\partial }{\partial x_l}\partial TQ{Q}^{\mathrm{T}})}_{n,l}=\sum _l\sum _k{(\frac{\partial }{\partial x_l}\partial T)}_{n,k}{(Q{Q}^{\mathrm{T}})}_{k,l}=\sum _{k,l}\frac{{\partial }^2{T}_n}{\partial x_k\partial x_l}{(Q{Q}^{\mathrm{T}})}_{k,l}=\sum _{k,l}{H}_{k,l}^n{(Q{Q}^{\mathrm{T}})}_{k,l}=\text{tr}(H^{n}Q{Q}^{\mathrm{T}})=\text{tr}(Q^{\mathrm{T}}{H}^{n}Q),$$ (38)

where T_n is the n^th element of T, and Hⁿ is the Hessian matrix of T_n. By combining Eqs. (28,36), the variation of R_g with respect to T(x) is obtained as:

$$\frac{\delta R_g}{\delta T(x)}=-C(x)Q(x)h(x).$$ (39)

Therefore, we first compute the elements of the vector h(x), followed by the variation of R_g with respect to T(x), which allows for T⁻¹(y) to be regularized without being explicitly calculated.

An issue that we faced in the computation of Eq. (39) is the calculation of the inverse Jacobian matrix Q where diffeomorphism is not preserved. As a remedy, we only performed the regularization of T⁻¹ in regions where the Jacobian matrix had a positive determinant and singular values greater than a threshold (.25 in our experiments).

Appendix C. Deviation of the Jacobian determinant

Using the following Theorem, we can see that for a diffeomorphic d-dimensional transformation, J(x) − 1 is bounded from above and below by two functions of ‖∂T(x) − ‖_F, as follows:

$$max{(1-{\Vert \partial T(x)-\mathbb{I}\Vert }_F,0)}^d-1\le J(x)-1\le {(1+\frac{1}{\sqrt{d}}{\Vert \partial T(x)-\mathbb{I}\Vert }_F)}^d-1.$$ (40)

Both of these bounds approach 0 monotonically as ‖∂T(x) − ‖_F→0, thereby taking J(x) − 1 to 0 following the squeeze theorem.²²

Theorem. For a d × d matrix M with det M ≥ 0, we have the following bounds for det M:

$$max{(1-{\Vert M-\mathbb{I}\Vert }_F,0)}^d\le detM\le {(1+\frac{1}{\sqrt{d}}{\Vert M-\mathbb{I}\Vert }_F)}^d,$$ (41)

where is the d × d identity matrix.

Proof. We have $M\le |detM|={\prod }_{k=1}^d{\sigma }_k$, where σ_k is the k^th singular value of M. According to the triangular inequality for the Frobenius norm,

$${\Vert M-\mathbb{I}\Vert }_F\ge {\Vert M\Vert }_F-{\Vert \mathbb{I}\Vert }_F=\sqrt{\sum _{k=1}^d{\sigma }_k^2}-\sqrt{d}.$$ (42)

Combining this with the root mean square-geometric mean inequality, $\sqrt{\frac{1}{d}{\sum }_{k=1}^d{\sigma }_k^2}\ge {({\prod }_{k=1}^d{\sigma }_k)}^{\frac{1}{d}}$, yields:

$$detM\le {(1+\frac{1}{\sqrt{d}}{\Vert M-\mathbb{I}\Vert }_F)}^d,$$ (43)

which proves the right-hand side of Ineq. (41). Note that this upper bound is a supremum, since for a given ‖M− ‖_F, equality can be achieved, e.g., for $M=(1+\frac{1}{\sqrt{d}}{\Vert M-\mathbb{I}\Vert }_F)\mathbb{I}$. We did not need det M ≥ 0 to prove this upper bound.

Regarding the lower bound, one can see that for ‖M− ‖_F≥1 the lower limit det M = 0 can be reached, for instance for a diagonal M whose first diagonal entry is 0, and all the rest of its d − 1 diagonal entries are equal to $1+\sqrt{\frac{1}{d-1}({\Vert M-\mathbb{I}\Vert }_F^2-1)}$. Concerning the case with ‖M − ‖_F < 1, we use the inequality σ_min(A − B) ≤ σ_min(A) + σ_max(B) (Horn and Johnson, 1994), with σ_min(A) and σ_max(A) the minimum and maximum singular value of A, respectively. Letting A = M and B = M − gives:

$$1={\sigma }_{min}(\mathbb{I})\le {\sigma }_{min}(M)+{\sigma }_{max}(M-\mathbb{I}).$$ (44)

Since σ_max(M − ) ≤ ‖M − ‖_F, we have:

$${\sigma }_{min}(M)\ge 1-{\Vert M-\mathbb{I}\Vert }_F.$$ (45)

Considering that $detM=|detM|={\prod }_{k=1}^d{\sigma }_k(M)\ge {\sigma }_{min}{(M)}^d$ and the assumption 1 − ‖M − ‖_F > 0, we raise Eq. (45) to the power of d, resulting in:

$$detM\ge {(1-{\Vert M-\mathbb{I}\Vert }_F)}^d.$$ (46)

Finally, we rewrite the two cases of ‖M − ‖_F ≥ 1 and ‖M − ‖_F < 1 in the following compact form, which proves the left-hand side of Ineq. (41):

$$max{(1-{\Vert M-\mathbb{I}\Vert }_F,0)}^d\le detM.$$ (47)