MIND Demons: Symmetric Diffeomorphic Deformable Registration of MR and CT for Image-Guided Spine Surgery

Abstract

Intraoperative localization of target anatomy and critical structures defined in preoperative MR/CT images can be achieved through the use of multimodality deformable registration. We propose a symmetric diffeomorphic deformable registration algorithm incorporating a modality-independent neighborhood descriptor (MIND) and a robust Huber metric for MR-to-CT registration. The method, called MIND Demons, finds a deformation field between two images by optimizing an energy functional that incorporates both the forward and inverse deformations, smoothness on the integrated velocity fields, a modality-insensitive similarity function suitable to multimodality images, and smoothness on the diffeomorphisms themselves. Direct optimization without relying on the exponential map and stationary velocity field approximation used in conventional diffeomorphic Demons is carried out using a Gauss-Newton method for fast convergence. Registration performance and sensitivity to registration parameters were analyzed in simulation, phantom experiments, and clinical studies emulating application in image-guided spine surgery, and results were compared to mutual information (MI) free-form deformation (FFD), local MI (LMI) FFD, normalized MI (NMI) Demons, and MIND with a diffusion-based registration method (MIND-elastic). The method yielded sub-voxel invertibility (0.008 mm) and nonzero-positive Jacobian determinants. It also showed improved registration accuracy in comparison to the reference methods, with mean target registration error (TRE) of 1.7 mm compared to 11.3, 3.1, 5.6, and 2.4 mm for MI FFD, LMI FFD, NMI Demons, and MIND-elastic methods, respectively. Validation in clinical studies demonstrated realistic deformations with sub-voxel TRE in cases of cervical, thoracic, and lumbar spine.

I. Introduction

Spinal disorders are a major cause of disability [1] and cover a broad spectrum of pathologies, such as spinal injury (25% of trauma patients [2]), spine metastases (in 10% of cancer patients [3]), and scoliosis (presenting in up to 32% of adults and 68% of the elderly [4]). Such spinal diseases are treatable by surgery; however, the complexity of spinal structure and function can challenge safe and accurate intervention. Especially for minimally invasive surgical approaches, image guidance improves the localization of target anatomy (e.g., vertebral levels and tumors) and critical structures (e.g., nervous and vascular systems) and has been shown to improve surgical accuracy and outcomes in pedicle screw placement [5], [6], correction of spinal deformities [5], [6], trauma surgery [7], percutaneous vertebroplasty [8], and resection of tumors [9].

Preoperative images (MR or CT) are often the basis for surgical planning to define the location of the target and adjacent structures and can be related to intraoperative imaging (e.g., MR, CT, cone-beam CT, or ultrasound) using deformable image registration. A wide range of deformable image registration methods have been proposed, as reviewed in [10]. Diffeomorphic registration methods [11]–[14] are of particular interest due to the ability to achieve smooth deformations and preserve the topology of anatomical structures. For example, SyN has been shown to be a reproducible and reliable diffeomorphic registration method [14], [15].

Preoperative MRI often forms the basis for surgical planning, since it provides clear delineation of soft tissues (e.g., the spinal cord, intervertebral discs, cerebrospinal fluid, nerve bundles, and vasculature). On the other hand, CT or cone-beam CT often forms the basis for intraoperative image guidance, since it provides comparatively fast 3D imaging capability and clear depiction of bone anatomy in relation to surgical tools. Therefore, a modality-independent image registration method would be valuable in resolving deformation between preoperative MR and intraoperative CT images. Registration based on features (e.g., points, and meshes) requires reliable feature definition and may impede standard clinical workflow. In the work reported below, we focus on volumetric registration methods that permit automatic registration of 3D image data directly. Mutual information (MI) and its normalized variants are prevalent similarity metrics for multimodality imaging [16], [17]; however, since both are global (i.e., based on the entire image histogram), they are sensitive to intensity non-uniformities (e.g., scatter artifacts in CT and shading artifacts due to coil sensitivity patterns in MR). To reduce such sensitivity, the metrics can be evaluated locally [18]. Still, application of MI-based metrics for MR-CT registration can be challenged by the assumption of tissue (intensity)-class correspondences [19].

Aside from relying on similarity metrics that are intrinsically more suitable to disparate modalities, images can be transformed into a consistent representation in which voxel values capture some characteristics of the original images. Scalar-value representations [20]–[22] are not always capable of capturing rich structural/contextual information. The ability to encode such local structural information is increased by using a descriptor (vector) representation. Gabor attributes, for example, can be used to capture local shape information [23]. Also, descriptors constructed based on non-local means (NLM) have been proposed to capture local structure [24] and local context [25]. For registration of MR and CT, we focus on a representation of local structure. For example, modality-independent neighborhood descriptors (MIND) within a diffusion-based registration method (referred to MIND-elastic) have been shown to provide elastic mappings between MR and CT images of the lung [24]. Viscoelastic mappings (i.e., a combination of elastic and fluid deformations)—as approximated by Demons, for example—could improve the ability to resolve large deformation within noisy data [26].

To resolve the deformation of the spine associated with patient positioning in the operating room, we propose a new symmetric diffeomorphic deformable registration method. Called MIND Demons, the method extends the SyN [13], diffeomorphic Demons [27], and MIND-elastic [24] approaches. It optimizes a novel single energy functional that incorporates explicit smoothness priors on both velocity fields and diffeomorphisms and implements a number of additional advances distinct from previous work. It incorporates a soft constraint on the conservation of Lagrangian momentum to enable simple optimization. To achieve symmetry, it estimates a pair of time-dependent diffeomorphisms subject to a geodesic shortest length constraint and satisfying an inverse condition. The method uses Lagrangian push-forward to bypass a Jacobian change of variables used in optimization. It imposes smoothness of the diffeomorphisms through minimization of their harmonic energies. For robust MR-CT registration, it incorporates MIND descriptors [24] and a robust Huber distance metric [28]. The method alternates between the Gauss-Newton (GN) optimization to maximize image alignment and Tikhonov regularization to minimize harmonic energies. Finally, the MIND Demons method does not use a constant velocity field approximation as in conventional diffeomorphic Demons. The theory and method for the approach are described in Section II. Sections III and IV demonstrate registration performance in comparison to other well-established reference methods in physical experiments using an ovine (sheep) spine. Validation of the method in clinical studies is also presented in Sections III and IV. Advantages and limitations are discussed in Section V. The Appendixes included in the supporting document¹ present analyses of parameter sensitivity and registration performance in simulations as well as validation of the nominal parameter settings for MIND Demons.

II. Mind Demons Registration

A. Notation

The notation used in this work is adopted from [11]–[13]. Let Ω ∈ ℝⁿ be a closed domain (n = 2 or 3 for 2D or 3D, respectively) and let images be functions I: Ω → ℝ^d where d is the dimension of the voxel values. Registration seeks a diffeomorphic mapping ψ: Ω × t ∈ [0,1] → Ω between a moving image I₀ and a fixed image I₁ such that I₀ ∘ ψ(z, 0) = I₁(z) (Fig. 1(a)) [29]. The diffeomorphisms ψ(z, 0) can be decomposed into a pair of diffeomorphisms ϕ_i(x, i) for i ∈ {0,1} as shown in Fig. 1(a). The diffeomorphisms ϕ_i(x, t) are a flow of time-dependent velocity fields,

$$\begin{array}{l}{\mathit{\varphi }}_i(\mathit{x},t)={\mathit{\varphi }}_i(\mathit{x},0.5)+\\ {\displaystyle {\int }_0^{{(-1)}^i(0.5-t)}{\mathit{v}}_i({\mathit{\varphi }}_i(\mathit{x},0.5-{(-1)}^i\tau )},0.5-{(-1)}^i\tau )d\tau ,i=0,1.\end{array}$$ (1)

Fig. 1.

Lagrangian description of the time-dependent velocity fields and diffeomorphisms [11]. The inverses of the diffeomorphisms are shown in blue. (a) Push-forward of the Lagrangian frame defined at time 0.5 to t_i = 0.5 − (−1)ⁱτ using ϕ_i(x, t_i) and $\mathit{\psi }(\mathit{z},0)={\mathit{\varphi }}_0({\mathit{\varphi }}_1^{-1}(\mathit{z},0.5),0)$. (b) Velocities associated with ϕ₀(x, 0.5 − τ) and ${\mathit{\varphi }}_0^{-1}(\mathit{y},\tau )$ at $\mathit{p}={\mathit{\varphi }}_0(\mathit{x},0.5-\tau )={\mathit{\varphi }}_0^{-1}(\mathit{y},\tau )$. (c) Composition of diffeomorphisms and its inverse.

Since the two diffeomorphisms are defined for different ranges of time t—i.e., ϕ₀(x, t) is defined for t ∈ [0, 0.5] and ϕ₁(x, t) is defined for t ∈ [0.5, 1], it is convenient to define the new variable t_i = (0.5 − (−1)ⁱτ) ∈ [0,1] using pseudo-time τ ∈ [0, 0.5]. Here, ${\mathit{v}}_i({\mathit{\varphi }}_i(\mathit{x},t_i),t_i)\in L^2(\mathrm{\Omega }\times \left[0,1\right],V)=\{\mathit{v}:\mathrm{\Omega }\times t\in \left[0,1\right]\to V|\Vert \mathit{v}\Vert ={\displaystyle {\int }_0^1{\Vert \mathit{v}(t)\Vert }_{v}dt={\displaystyle {\int }_0^1{\Vert L\mathit{v}(t)\Vert }_2}dt={\displaystyle {\int }_0^1{\displaystyle {\int }_{\mathit{x}\in \mathrm{\Omega }}}{\Vert L\mathit{v}(\mathit{x},t)\Vert }_{2}d\mathit{x}\mathit{dt}<\infty }}\}$ is a time-dependent velocity field in a space V of compact support vector fields. The differential operator L = (Id + a²∇²) for a ∈ ℝ defines a norm on V as ‖v_i (t_i)‖_V = ‖Lv_i(t_i)‖₂ (i.e., an explicit smoothness prior on v_i(t_i)) [30]. Throughout this work, equations written in terms of i apply for both i = 0 and i = 1.

The diffeomorphisms ϕ_i (x, t_i) and velocity fields v_i(ϕ_i(x, t_i), t_i) are defined with respect to the Lagrangian frame of a virtual image I_0.5 domain defined at time 0.5 such that ϕ_i(x, 0.5) = x and v_i(x, 0.5) = 0, ∀x ∈ Ω. The diffeomorphisms ϕ_i(x, t_i) push the Lagrangian frame forward to time t_i. For example, as shown in Fig. 1(a), ϕ₀(x, 0) pushes a point x in I_0.5 to a point y in I₀. Since ϕ_i(x, t_i) always depends on x, and v_i(ϕ_i(x, t_i), t_i) always depends on ϕ_i(x, t_i), we can drop the spatial-position arguments as ϕ_i(x, t_i) = ϕ_i(t_i) and v_i(ϕ_i(x, t_i), t_i) = v_i(t_i) for the sake of brevity when there is no ambiguity. Similarly we can drop the time arguments for the diffeomorphisms and the velocity fields at the endpoints as ϕ_i = ϕ_i(i) and v_i = v_i(i).

The diffeomorphisms ${\mathit{\varphi }}_i^{-1}({\mathit{\varphi }}_i(\mathit{x},t_i),t_i+{(-1)}^i\tau )$ are inverses of ϕ_i(x, t), e.g., ${\mathit{\varphi }}_0^{-1}({\mathit{\varphi }}_0(\mathit{x},0.5-\tau ),0.5)=\mathit{x}$ as shown in Fig. 1(c). As illustrated in Appendix I.A as well as [11, eq. (1)] and [31, eq. (1)], the velocity fields of the inverses of ϕ_i(x, t_i) are defined as ${\mathit{v}}_i^{\prime }(\mathit{p},i+{(-1)}^i\tau )=-D_{\mathit{p}}{\mathit{\varphi }}_i^{-1}(\mathit{p},i+{(-1)}^i\tau ){\mathit{v}}_i(\mathit{p},i+{(-1)}^i\tau )$ where p = ϕ_i(x, t_i) and $D_{\mathit{p}}{\mathit{\varphi }}_i^{-1}(\mathit{p},i+{(-1)}^i\tau )$ is a time-dependent spatial Jacobian. Fig. 1(b) depicts the velocities ${\mathit{v}}_0^{\prime }(\mathit{p},\tau )$ and v₀(p, 0.5 − τ) as the tangent vectors at a point $\mathit{p}={\mathit{\varphi }}_0^{-1}(\mathit{y},\tau )={\mathit{\varphi }}_0(\mathit{x},0.5-\tau )$. For brevity, we drop the spatial-position arguments as ${\mathit{\varphi }}_0^{-1}(\tau )={\mathit{\varphi }}_0^{-1}(\mathit{y},\tau )$ and ${\mathit{\varphi }}_1^{-1}(1-\tau )={\mathit{\varphi }}_1^{-1}(\mathit{z},1-\tau )$. Similarly, we define the inverse diffeomorphisms at the endpoints as ${\mathit{\varphi }}_i^{-1}={\mathit{\varphi }}_i^{-1}(0.5)$.

The energy of the flows ϕ_i(t_i) can be computed as the time-integrated square norm of v_i(t_i) [11],

$$\begin{array}{l}{E}_{{\mathit{v}}_i}={\displaystyle \underset{0}{\overset{0.5}{\int }}{\Vert L{\mathit{v}}_i(0.5-{(-1)}^i\tau )\Vert }_2^{2}d\tau }\\ ={\displaystyle \underset{0}{\overset{0.5}{\int }}{\displaystyle \underset{\mathit{x}\in \mathrm{\Omega }}{\int }\Vert L{\mathit{v}}_i({\mathit{\varphi }}_i(\mathit{x},0.5-{(-1)}^i\tau ),0.5-{(-1)}^i\tau )\Vert }d\mathit{x}d\tau .}\end{array}$$

The geodesic shortest length—measured using the left-invariance metric [29] (i.e., inverse invariance²)—is defined in terms of the minimizing energy as

$$\rho ({\mathit{\varphi }}_i(\mathit{x}),{\mathit{\varphi }}_i(\mathit{x},0.5))=\underset{v_i}{\text{inf}}\left\{\sqrt{E_{v_i}}\right\}$$ (2)

with a boundary ϕ_i(x, 0.5) = x and ϕ₀(x) = y and ϕ₁(x) = z.

The Lagrangian momentum is a linear form M_i = Lv_i: V → ℝ and, from the fundamental principle of mechanics, is constant in time along geodesics of ϕ_i in the absence of external forces [30]. Thus, M_i(t_i) = Lv_i(t_i) = M_i(0.5) where M_i(0.5) is an initial measure-based momentum field.³ Since the energy of the flows can be computed in terms of M_i(t_i) as $E({\mathit{\varphi }}_i)={\displaystyle {\int }_0^{0.5}{\Vert {\mathit{M}}_i(0.5-{(-1)}^i\tau )\Vert }_2^{2}d\tau }$, and by the conservation of momentum and use of Lagrangian push-forward, we have

$$\underset{v_i}{\text{inf}}{\displaystyle \underset{0}{\overset{0.5}{\int }}{\Vert {\mathit{M}}_i(0.5-{(-1)}^i\tau )\Vert }_2^{2}d\tau \equiv \underset{v_i}{\text{inf}}{\Vert {\mathit{M}}_i(0.5)\Vert }_2^2}$$ (3)

implying that, within the Lagrangian framework, using a time-step of Δt = 0.5 is equivalent to imposition of the conservation of Lagrangian momentum.

B. Symmetric Diffeomorphic Demons Optimization

The optimization of time-dependent diffeomorphisms with prior information on smoothness of the diffeomorphisms ties the Demons [27] and SyN methods [13] together. As in [27], imposition of prior information is separated from maximization of image alignment by introducing hidden variables, representing intermediate diffeomorphisms

$$\begin{array}{l}{\mathit{\eta }}_i(\mathit{x},t)={\mathit{\eta }}_i(\mathit{x},0.5)+\\ {\displaystyle \underset{0}{\overset{{(-1)}^i(0.5-t)}{\int }}{\mathit{v}}_i({\mathit{\eta }}_i(\mathit{x},0.5-{(-1)}^i\tau ),0.5-{(-1)}^i\tau )d\tau }\end{array}$$ (4)

into the registration process. This separation simplifies the optimization process, allows estimation of various deformation models (e.g., fluid, elastic, and viscoelastic) [26], and could allow simple integration of additional prior information (e.g., a rigidity constraint on vertebral bodies [32]); however, it could yield a local optimum that is different from the true optimum of the energy functional and could increase overall computation time (owing to the alternating steps in each iteration).

The optimization uses Lagrangian push-forward to bypass the computation of a Jacobian change of variables [12], [13] and is performed in the Lagrangian frame with Δt = 0.5 to impose a soft constraint on the conservation of momentum [30]. The method is general in the sense that it can be used with various similarity metrics as well as image representations. For this reason, the similarity metric is presented in an abstract form, denoted S(I₀, I₁, x), and is evaluated with respect to the Lagrangian frame. The Huber distance as well as the MIND descriptor representation of an image, both used to define S(I₀, I₁, x), are described in the next section.

Our method estimates ϕ_i for i ∈ {0,1} subject to a geodesic length constraint ρ(ϕ₀(x, 0.5), ϕ₀(x)) = ρ(ϕ₁(x, 0.5), ϕ₁(x)). As shown in Appendix I.B, the geodesic length constraint (GLC) yields the relation ${\mathit{\varphi }}_i(0.5-{(-1)}^i\tau )={\mathit{\varphi }}_j^{-1}(j+{(-1)}^j\tau )$ for i ≠ j ∈ {0,1} from the inverse invariance property of the geodesic shortest length [29] and the uniqueness of the ordinary differential equation (ODE) ∂_tϕ_i(x, t_i) = v_i(ϕ_i(x, t_i), t_i) with the initial condition ϕ_i(x, 0.5) = x [33]. We use this relation to impose the geodesic length constraint in the energy minimization as

$$\begin{array}{l}E({\mathit{\varphi }}_0,{\mathit{\varphi }}_1,{\mathit{\eta }}_0,{\mathit{\eta }}_1)=\underset{{\mathit{\eta }}_i,{\mathit{\varphi }}_i}{\text{inf}}\frac{1}{2}\{{\alpha }_S^2{\displaystyle \underset{\mathit{x}\in \mathrm{\Omega }}{\int }S{(I_0\circ {\mathit{\eta }}_0,I_1\circ {\mathit{\eta }}_1,\mathit{x})}^{2}d\mathit{x}}\\ +{\alpha }_P^2({\Vert \nabla {\mathit{\varphi }}_1^{-1}\Vert }_2^2+{\Vert \nabla {\mathit{\varphi }}_0^{-1}\Vert }_2^2)\}\\ +\frac{{\alpha }_U^2}{2}(\rho {({\mathit{\varphi }}_1^{-1},{\mathit{\eta }}_0)}^2+\rho {({\mathit{\varphi }}_0^{-1},{\mathit{\eta }}_1)}^2)\\ \text{subject to}{\mathit{\varphi }}_i(0.5)={\mathit{\eta }}_i(0.5)=\mathit{Id},\\ \rho ({\mathit{\varphi }}_0(\mathit{x},0.5),{\mathit{\varphi }}_0(\mathit{x}))=\rho ({\mathit{\varphi }}_1(\mathit{x},0.5),{\mathit{\varphi }}_1(\mathit{x})),\text{and}{\mathit{\varphi }}_i\circ {\mathit{\varphi }}_i^{-1}=Id.\end{array}$$ (5)

where α_S, α_P, and α_U control the regularization strengths, $\rho ({\mathit{\varphi }}_j^{-1},{\mathit{\eta }}_i)$ measures the geodesic shortest length between ${\mathit{\varphi }}_j^{-1}$ and η_i, and the L² norm squared ${\Vert \cdot \Vert }_2^2=\langle \cdot ,\cdot \rangle$ the L² inner product. Similar to the diffeomorphic Demons algorithm [27], an alternating optimization is performed over η_i and ϕ_i, which simplifies the optimization of (5) into two simple steps: 1) maximization of image alignment using GN and 2) Tikhonov regularization of the diffeomorphisms.

1) Inexact Image Matching using the Gauss-Newton Method

In the first step, given an estimate of ${\mathit{\varphi }}_j^{-1}$ for i ≠ j ∈ {0,1}, the diffeomorphisms η_i with the shortest geodesic length that maximize alignment of I₀ and I₁ are optimized by minimizing

$$\begin{array}{l}{E}_1({\mathit{v}}_0,{\mathit{v}}_1)=\frac{1}{2}\{{\displaystyle \underset{\mathit{x}\in \mathrm{\Omega }}{\int }S{(I_0\circ {\mathit{\eta }}_0,I_1\circ {\mathit{\eta }}_1,\mathit{x})}^{2}d\mathit{x}+}\\ \frac{{\alpha }_U^2}{{\alpha }_S^2}{\displaystyle \underset{0}{\overset{0.5}{\int }}({\Vert L{\mathit{v}}_0(0.5-\tau )\Vert }_2^2+{\Vert L{\mathit{v}}_1(0.5+\tau )\Vert }_2^2)d\tau \}}\\ \text{subject to}{\mathit{\varphi }}_i(0.5)={\mathit{\eta }}_i(0.5)\mathit{Id}\text{and}\\ \rho ({\mathit{\varphi }}_0(\mathit{x},0.5),{\mathit{\varphi }}_0(\mathit{x}))=\rho ({\mathit{\varphi }}_1(\mathit{x},0.5),{\mathit{\varphi }}_1(\mathit{x}))\end{array}$$ (6)

where $\rho ({\mathit{\varphi }}_j^{-1},{\mathit{\eta }}_i)=\underset{{\mathit{v}}_i}{\text{inf}}{{\displaystyle {\int }_0^{0.5}\Vert L{\mathit{v}}_i(0.5-{(-1)}^i\tau )\Vert }}_2^{2}d\tau$ from (4), and η_i(0.5) = ϕ_i(0.5) = Id. The functional (6) consists of the first and last terms in (5). The first term measures similarity between I₀ ∘ ϕ₀ and I₁ ∘ ϕ₁ in the Lagrangian frame, and the last term measures the kinetic energy of the diffeomorphisms. This objective function is similar to that optimized in [12, eq. (1)] and [13, eq. (3)], except E₁ is evaluated with respect to the Lagrangian frame (defined at t = 0.5) by pushing the frame forward to t = 0 and t = 1, which allows us to bypass the Jacobian change of variables used in [12, eq. (9)] and [13, eqs. (6, 7)]. Owing to the conservation of Lagrangian measure-based momentum, the optimal solution of (6) should satisfy, at each time point [30]:

$${\mathit{M}}_i(t_i)={\mathit{M}}_i(0.5)=S{\nabla }_{i}S$$ (7)

where ∇_iS is the Fréchet derivative of S with respect to ϕ_i. As shown in Appendix I.C, (7) is equivalent to the Fréchet derivative of E₁ with respect to v_i. The velocity field is updated using the momentum field (7) and the gradient descent (GD) approach [12], [13] as

$${\mathit{v}}_i^k={\mathit{v}}_i^{k-1}-\epsilon K{({\mathit{M}}_i)}_{L^2}=-\epsilon K{(S{\nabla }_{i}S)}_{L^2}$$ (8)

where k is an iteration number, ${\mathit{v}}_i^{k-1}=\mathbf{0}$ from push-forward of the Lagrangian frame, and K = (L^†L)⁻¹ is the Green kernel projecting a momentum field in L² to a smooth velocity field in V as in [12, eq. (6)], which, for α = 1, is approximated by a Gaussian kernel $G_{{\sigma }_U}$ with width σ_U [13] (whose value was chosen empirically as described in Appendix II.A.2).

Recent work [34], [35] shows that GN can yield comparable registration accuracy to GD in fewer iterations. The GN update uses an approximation to the second-order derivative of E₁ as

$$\begin{array}{l}{\mathit{v}}_i^k=-K{({({\nabla }_{ii}{E}_1^{\prime })}^{-1}{\mathit{M}}_i)}_{L^2}\\ =-K{\left({({\nabla }_{i}S{\nabla }_i^{T}S+\frac{{\alpha }_U^2}{{\alpha }_S^2}\mathit{I})}^{-1}S{\nabla }_{i}S\right)}_{L^2}\\ =-K{({\mathit{u}}_i)}_{L^2}\end{array}$$ (9)

where ${\nabla }_{ii}{E}_1^{\prime }$ approximates the second-derivative of E₁ (Appendix I.C) and I is an identity matrix. Eq. (9) can be considered a general case of (8) with ${\nabla }_{ii}{E}_1^{\prime }=(1/\epsilon )\mathit{I}$. In addition to application of a uniform scale ε in GD, GN allows the momentum fields to steer toward potentially better directions (closer to local optima)—i.e., a soft constraint on the conservation of momentum. The inverse term in (9) is computed using the Sherman-Morrison formula as in [27], [36] which results in voxel-wise estimation of u_i as

$${\mathit{u}}_i(\mathit{x})=\frac{-S(\mathit{x}){\nabla }_{i}S(\mathit{x})}{{\alpha }_U^2/{\alpha }_S^2+{\Vert {\nabla }_{i}S(\mathit{x})\Vert }_2^2}$$ (10)

where we apply α_S(x) = 1/S(x) to penalize noise in a spatially varying manner. As shown in [40], using the fact (α_U|S(x)| − ‖∇_iS(x)‖₂)² > 0, we have a constraint on the length of an update as ‖u_i(x)‖₂ ≤ 1/2α_U. Eq. (10) with α_S(x) = 1/S(x) is similar to the force equation using mean square intensity difference in [14], [27], and [37] and using local correlation coefficient in [36].

The intermediate diffeomorphisms are updated using (9) and (10) as

$${\mathit{\eta }}_i^k={\mathit{\varphi }}_i^{k-1}+{\mathit{v}}_i^k\circ {\mathit{\varphi }}_i^{k-1}$$ (11)

$$={({\mathit{\varphi }}_j^{k-1})}^{-1}+{\mathit{v}}_i^k\circ {({\mathit{\varphi }}_j^{k-1})}^{-1}$$ (12)

where (11) derives from (4) with Δt=0.5, and (12) comes from the GLC relation ${\mathit{\varphi }}_i={\mathit{\varphi }}_j^{-1}$ (i.e., ${\mathit{\varphi }}_i(0.5-{(-1)}^i\tau )={\mathit{\varphi }}_j^{-1}(j+{(-1)}^j\tau )$ for τ = Δt = 0.5) for imposition of the geodesic length constraint. After both intermediate diffeomorphisms are computed, the registration continues to the second step.

2) Tikhonov Regularization of Diffeomorphisms

The diffeomorphisms ${\mathit{\varphi }}_i^k$ are regularized under smoothness and invertibility constraints by minimizing the energy functional consisting of the last two terms in (5) as

$$\begin{array}{l}{E}_2({\mathit{\varphi }}_0,{\mathit{\varphi }}_1)=\frac{1}{2}\{(\Vert L({\mathit{\varphi }}_1^{-1}-{\mathit{\eta }}_0){{\Vert }_2^2+\Vert L({\mathit{\varphi }}_0^{-1}-{\mathit{\eta }}_1)\Vert }_2^2)\\ +\frac{{\alpha }_P^2}{{\alpha }_U^2}({\Vert \nabla {\mathit{\varphi }}_1^{-1}\Vert }_2^2+{\Vert \nabla {\mathit{\varphi }}_0^{-1}\Vert }_2^2)\}\\ \text{subject to}{\mathit{\varphi }}_i\circ {\mathit{\varphi }}_i^{-1}=Id\end{array}$$ (13)

where $\rho {({\mathit{\varphi }}_j^{-1},{\mathit{\eta }}_i)}^2=\underset{{\mathit{v}}_i}{\text{inf}}{\displaystyle {\int }_0^{0.5}{\Vert L{\mathit{v}}_i(0.5-{(-1)}^i\tau )\Vert }_2^{2}d\tau =\underset{{\mathit{\varphi }}_j^{-1}}{\text{inf}}}{\Vert L({\mathit{\varphi }}_j^{-1}-{\mathit{\eta }}_i)\Vert }_2^2$ from τ = Δt = 0.5 and (12). By omitting the invertibility constraint, (13) can be optimized using Tikhonov regularization [38]. A necessary condition for ${\mathit{\varphi }}_j^{-1}$ to be a (local) minimizer of E₂ is

$${\nabla }_{{\mathit{\varphi }}_j^{-1}}{E}_2=(L^{†}L)({\mathit{\varphi }}_j^{-1}-{\mathit{\eta }}_i)+\frac{{\alpha }_P^2}{{\alpha }_U^2}{\nabla }^2{\mathit{\varphi }}_j^{-1}=0$$ (14)

By letting a = 0 as in previous work [11], [30], [39], we have L = Id and the solution of (14) can be approximated by ${\mathit{\varphi }}_j^{-1}=G_{{\sigma }_D}\ast {\mathit{\eta }}_i$ (i.e., a solution to an isotropic heat equation) where $G_{{\sigma }_D}$ is a Gaussian kernel with width ${\sigma }_D=\sqrt{2}{\alpha }_P/{\alpha }_U$ (a nominal value of σ_D was chosen empirically as described in Appendix II.A.2). Solving (14) with a > 0 (in the Fourier domain) could increase smoothness of ${\mathit{\varphi }}_j^{-1}$ and potentially improve registration accuracy, but the increase in computational complexity may outweigh the benefit. The invertibility constraint is sequentially imposed by minimizing

$$E_{Id}({\mathit{\zeta }}_j)=\frac{1}{2}{\Vert {\mathit{\varphi }}_j^{-1}\circ {\mathit{\zeta }}_j-Id\Vert }_2^2$$ (15)

where we seek ζ_j ≈ ϕ_j using GD. We do not use GN here, since it requires inversion of the second-order term (which cannot be simplified using the Sherman-Morrison formula), and the increase in computation time for the required matrix inversion could compromise the fast convergence rate of the method. The Fréchet derivative of E_Id obtained via a variation is

$$\begin{array}{l}{\nabla }_{{\mathit{\zeta }}_j}{E}_{Id}=({\mathit{\varphi }}_j^{-1}\circ {\mathit{\zeta }}_j-Id)D_{{\mathit{\zeta }}_j}{\mathit{\varphi }}_j^{-1}\\ ={\mathit{\varphi }}_j^{-1}\circ {\mathit{\zeta }}_j-Id\end{array}$$ (16)

where $D_{{\mathit{\zeta }}_j}{\mathit{\varphi }}_j^{-1}=D_{{\mathit{\zeta }}_j}{\mathit{\zeta }}_j^{-1}={(D_{{\mathit{\zeta }}_j}{\mathit{\zeta }}_j)}^{-1}=\mathit{I}$ and $({\mathit{\varphi }}_j^{-1}\circ {\mathit{\zeta }}_j)(\mathit{x})={\mathit{\varphi }}_j^{-1}({\mathit{\zeta }}_j(\mathit{x}))+{\mathit{\zeta }}_j(\mathit{x})$. The optimization gives an estimate of ${\mathit{\varphi }}_j^k$ such that ${\mathit{\varphi }}_j^k\circ {({\mathit{\varphi }}_j^k)}^{-1}\approx Id$. The invertibility constraint ${\mathit{\varphi }}_i\circ {\mathit{\varphi }}_i^{-1}=Id$ and the GLC relation ${\mathit{\varphi }}_i={\mathit{\varphi }}_j^{-1}$ lead to the inverse condition ϕ_i ∘ ϕ_j = Id (i.e., ϕ₀ and ϕ₁ are inverse of each other). After both diffeomorphisms have been estimated, the alternating optimization yields the diffeomorphic map $\mathit{\psi }={\mathit{\varphi }}_0\circ {\mathit{\varphi }}_1^{-1}$ between I₀ and I₁. The method is symmetric since it is independent of the order of input images as proven in Appendix I.D.

C. Modality Independent Neighborhood Descriptor

A MIND descriptor [24] builds from the concept of NLM [40] and self-similarity [41] and can be used to capture corresponding local structures in MR and CT images. It is a vector representation of each voxel, and its computation involves other voxels in its neighborhood. The configuration of neighboring voxels used in the calculation is called a stencil and is given the symbol ${\mathcal{N}}_S$ [24]. Stencils can be arranged in a variety of patterns—e.g., the 2D and 3D examples shown in Fig. 2.

Fig. 2.

MIND stencil configurations used in this work. (a,b) Stencil for 2D and 3D images comprising 24 pixels and 34 voxels, respectively. (c) Corresponding coronal slices of the 3D stencil in (b). Gray voxels mark voxels in a stencil, and the black voxel marks the voxel for which MIND is computed.

Consider a MIND descriptor m_I(x) = [m_I,_j(x)] for $j=1-|{\mathcal{N}}_S|$ for a voxel x in an image I. Each element m_I,j in the descriptor corresponds to a voxel ${\mathit{r}}_j\in {\mathcal{N}}_S$ in the stencil, and its value is computed as

$$m_{I,j}=c\text{exp}\left(-\frac{d(I,\mathit{x},{\mathit{r}}_j)}{V(I,\mathit{x})}\right)$$ (17)

where c is a normalization factor making $\underset{j\in \left[1,|{\mathcal{N}}_S|\right]}{max}\left\{m_{I,j}(\mathit{x})\right\}=1$ (i.e., for illumination and contrast invariance) and d(I, x, r_j) measures the distance between a patch of x and a patch of r_j The patch distance is computed as

$$d(I,\mathit{x},{\mathit{r}}_j)={\displaystyle \sum _{\mathit{z}\in {\mathcal{N}}_p}{G}_{{\sigma }_p}(\mathit{z}){(I(\mathit{x}+\mathit{z})-I({\mathit{r}}_j+\mathit{z}))}^2}$$ (18)

where ${\mathcal{N}}_p$ denotes a neighborhood configuration of a patch (e.g., a cube), z is an offset from the center voxel in ${\mathcal{N}}_p$, $G_{{\sigma }_p}$ is a discrete Gaussian kernel—with width σ_p mm and truncation (tail cut-off) errors $t_p=1-{\sum }_{\mathit{z}\in {\mathcal{N}}_p}{G}_{{\sigma }_p}(\mathit{z})$—used to increase the importance of the central voxel. V(I, x) in (17) approximates the local variance at x as

$$V(I,\mathit{x})=\frac{1}{|{\mathcal{N}}_N|}{\displaystyle \sum _{{\mathit{n}}_j\in {\mathcal{N}}_N}d(I,\mathit{x},{\mathit{n}}_j)}$$ (19)

where ${\mathcal{N}}_N$ denotes a stencil consisting of the nearest neighboring voxels of x (i.e., 4 and 6 nearest neighbors for 2D and 3D images, respectively). An efficient technique for computing descriptors is described in [24].

D. Huber Distance Metric

Registration is generally ill-posed [42], implying that small changes in images (e.g., due to noise and artifacts) can lead to large changes in the estimated deformations. A metric such as the L¹ norm which is insensitive to noise and outliers [43] could yield more reliable estimation (i.e., less confounded by noise) than quadratic norms (e.g., the L² norm). However, numerical optimization involving the L¹ criterion is difficult since it is not differentiable. Other metrics combining the robustness of L¹ and the differentiability of L² can be used to provide a reliable estimate while allowing simple numerical optimization. For example, a modified L^p norm has been used in an optical-flow based method to estimate discontinuity-preserving motions in noisy CT images of the chest [44]. Other work [45] uses a Huber penalty to estimate discontinuity-preserving optical flows for restoration of historical video images. To improve robustness against outliers from non-corresponding MIND descriptors of I₀ ∘ ϕ₀, ${\mathit{m}}_{I_0\circ {\mathit{\varphi }}_0}$, and that of I₁ ∘ ϕ₁, ${\mathit{m}}_{I_1\circ {\mathit{\varphi }}_1}$, we use the Huber distance [28]. Note that ${\mathit{m}}_{I_i\circ {\mathit{\varphi }}_i}$, is computed on an image after transformation I_i ∘ ϕ_i and ${\mathit{m}}_{I_i\circ {\mathit{\varphi }}_i}\ne {\mathit{m}}_{I_i}\circ {\mathit{\varphi }}_i$. We denote the difference between an element j of the MIND descriptors as ${\phi }_j(x)=m_{I_1\circ {\mathit{\varphi }}_1,j}(\mathit{x})-m_{I_0\circ {\mathit{\varphi }}_0,j}(\mathit{x})$. The Huber distance between the descriptors is

$$\begin{array}{l}S({\mathit{m}}_{I_0\circ {\mathit{\varphi }}_0},{\mathit{m}}_{I_1\circ {\mathit{\varphi }}_1},\mathit{x})=\\ {\displaystyle \sum _{j=1}^{|{\mathcal{N}}_S|}\{\begin{array}{ll}{\phi }_j^2(\mathit{x})/2\epsilon ,& \text{if}0\le |{\phi }_j(x)|\le \epsilon \\ |{\phi }_j(\mathit{x})|-(\epsilon /2),& \text{otherwise}\end{array}}\end{array}$$ (20)

where ∈ is the threshold between the quadratic and linear parts, and | · | denotes an absolute-value operator (i.e., the L¹ norm). The summation in (20) does not lead to cancelation of residual φ_j(x) owing to nonnegativity of the quadratic function ${\phi }_j^2(\mathit{x})$ and the absolute-value function |φ_j(x)|. The Fréchet derivative of (20) with respect to ϕ_i captures the greatest rate and direction of change as

$$\begin{array}{l}{\nabla }_{i}S({\mathit{m}}_{I_0\circ {\mathit{\varphi }}_0},{\mathit{m}}_{I_1\circ {\mathit{\varphi }}_1},\mathit{x})=\\ {\displaystyle \sum _{j=1}^{|{\mathcal{N}}_S|}\{\begin{array}{ll}{(-1)}^i\nabla m_{I_i\circ {\mathit{\varphi }}_i,j}(\mathit{x}),& \text{if}{\phi }_j(x)<-\epsilon \\ {(-1)}^{i+1}\nabla m_{I_i\circ {\mathit{\varphi }}_i,j}(\mathit{x}),& \text{if}{\phi }_j(x)<-\epsilon \\ {(-1)}^{i+1}[{\phi }_j(\mathit{x})\nabla m_{I_i\circ {\mathit{\varphi }}_i,j}(\mathit{x})],& \text{otherwise}\end{array}}\end{array}$$ (21)

where $\nabla m_{I_i\circ {\mathit{\varphi }}_i,j}$ is the Fréchet derivative of an element j of ${\mathit{m}}_{I_i\circ {\mathit{\varphi }}_i}$ with respect to ϕ_i for i ∈ {0,1}.

The Huber metric yields a more reliable deformation estimation, since the influence of outliers on derivatives of the metric is less than that on derivatives of quadratic norms. Moreover, edges in images can be preserved since it penalizes large differences less than the quadratic penalty in the L² norm.

E. MIND Demons Algorithm

We incorporate a multiresolution strategy to improve robustness against local minima. A multiresolution pyramid (defining coarse-to-fine evolution in each pyramid level) is constructed only for I_0.5 since the method uses the Lagrangian push-forward of the I_0.5 domain to the domain of I₀ and I₁ using ϕ₀ and ϕ₁, respectively. In each level of the pyramid, the optimization described in Section II.B is performed until it reaches either convergence or a maximum number of iterations. The optimizer converges if the maximum normalized magnitude of ∇_iS is less than γ_S = 10^−ℓ where ℓ is a level number. It also reaches convergence if the gradient ∇F of the Huber metric (20) with respect to an iteration number is less than γ_F = 10⁻⁶. The computation of ∇F involves a linear fit to the values of (20) estimated within a window of W = 20 iterations. In each optimization iteration, the MIND descriptor is recomputed since it is not deformation invariant. The underlying continuous representations of I₀ and I₁ are estimated using cubic B-spline interpolation. Parameters in MIND Demons and their nominal ranges and values (see Appendix II.A.2) are listed in Table I. The algorithm was implemented using the Insight Segmentation and Registration Toolkit (ITK) [51] and involved the gradient descent optimization of the inverses of diffeomorphisms (Algorithm I) within the framework summarized in Algorithm II.

ALGORITHM I: Inversion of Diffeomorphisms
Input	Input diffeomorphism (ϕ−1) Maximum number of iterations (N) Gradient magnitude tolerance (γI) Step size (ε)
Output	Inverse diffeomorphism (ζ)
1	Initialize ζ = Id and δ = 0 For iteration k < N
2	Compute ∇ζEId(ζk−1) using (16)
3	Compute the maximum gradient magnitude
	$\delta =\underset{\mathit{x}\in \mathrm{\Omega }}{max}\left\{{\Vert {\nabla }_{\mathit{\zeta }}{E}_{Id}({\mathit{\zeta }}^{k-1})(\mathit{x})\Vert }_2\right\}$
4	If δ < γI, Stop.
5	Update the inverses, ζk = ζk−1 − ε∇ζEId (ζk−1)
In this work, we use ε = 0.5, γI = 0.01, and N = 25 iterations.

ALGORITHM II: MIND Demons
Input	Input images (I0 and I1) Number of pyramid levels (Nℓ) Maximum numbers of iterations (Nk) Gradient magnitude tolerance (γS) Metric value convergence tolerance (γF) Convergence window (W) Registration parameters (σp, tp, ∈, σU, and σD)
Output	Diffeomorphism (ψ)
1	Initialize ϕ0(0.5) = ϕ1 (0.5) = Id and δ = 0 For level ℓ < Nℓ
2	If ℓ > 0, upsample ϕ0 and ϕ1  For iteration k <Nk
3	Compute ${\mathit{m}}_{I_0\circ {\mathit{\varphi }}_0}$ and ${\mathit{m}}_{I_1\circ {\mathit{\varphi }}_1}$
4	Compute S, ∇0S, and ∇1S using (20) and (21)
5	Compute the maximum normalized gradient magnitude    $\delta =\underset{\mathit{x}\in \mathrm{\Omega }}{max}\left\{\frac{1}{C}({\Vert {\nabla }_{0}S(\mathit{x})\Vert }_2+{\Vert {\nabla }_{1}S(\mathit{x})\Vert }_2)\right\}$   where $C={\sum }_{\mathit{x}\in \mathrm{\Omega }}({\Vert {\nabla }_{0}S(\mathit{x})\Vert }_2+{\Vert {\nabla }_{1}S(\mathit{x})\Vert }_2)$
6	If k > W, compute a gradient ∇F of a line fitted to S in (20) evaluated from k − W to k.
7	If δ < γS or ∇F < γF, Stop.
8	Compute ${\mathit{v}}_0^k$ and ${\mathit{v}}_1^k$ using (9) and (10) where K is approximated by $G_{{\sigma }_U}$
9	Estimate ${\mathit{\eta }}_0^k$ and ${\mathit{\eta }}_1^k$ using (12)
10	Regularize ${({\mathit{\varphi }}_0^k)}^{-1}$ and ${({\mathit{\varphi }}_1^k)}^{-1}$ based on (14) using $G_{{\sigma }_D}$
11	Estimate ${\mathit{\varphi }}_0^k$ and ${\mathit{\varphi }}_1^k$ using Algorithm I
12	Compute $\mathit{\psi }={\mathit{\varphi }}_0\circ {\mathit{\varphi }}_1^{-1}$
In this work, we use γS = 10−ℓ, γF = 10−6, and W = 20 iterations.

TABLE I.

MIND Demons Parameters and Nominal Settings

	Parameters	Range	Value
MIND	Weighting kernel width (σp) (mm)	0.5–1.0	0.5
Descriptor	Weighting kernel truncation error (tp)	0.015–0.4	0.1
Huber metric	Huber threshold (∈)	0.001–0.01	0.005
	Update field kernel width (σU) (voxels)	4–7	5
Demons	Displacement field kernel width (σD) (voxels)	0.6–1.4	1

The stencil configuration of MIND descriptors was fixed as shown in Fig. 2 and the Demons parameter (α_U) controlling the magnitude of an update was fixed at 1 voxel.

III. Experimental Methods

The registration performance of MIND Demons was analyzed in comparison to other well-established reference methods, including elastix free-form deformation (FFD) with MI and local MI (LMI) [18], [46], NMI Demons [17], [47] with a symmetric energy formulation, and MIND-elastic [24]. For conciseness, additional studies are summarized in Appendix II, including: 1) an analysis of parameter sensitivity to identify operating ranges and nominal parameter values for each registration method; 2) a comparison of registration performance for MIND Demons and the reference methods in a 2D simulation; and 3) a validation of the nominal parameter settings of MIND Demons in a 3D experiment using an ovine spine phantom. Using the identified parameter settings for each method, the overall registration performance of MIND Demons was compared to the other algorithms in a 3D physical experiment. Finally, registration performance under realistic imaging conditions was validated in clinical studies. All registration methods were initialized using NMI rigid registration. The maximum number of iterations for the 2D studies was 300 iterations (for a large scoliotic deformation) and that for the 3D studies was 100 iterations.

A. Analysis of Registration Performance

Registration performance was quantified in terms of geometric accuracy and diffeomorphic properties of the estimated deformations. Computation time was measured on a Dell Precision T7600 with two 2-GHz Intel Xeon processors and 32 GB RAM.

1) Target Registration Error (TRE)

The geometric accuracy was measured using TRE as a distance between corresponding target points in I₁ and I₀ after registration as

$$\text{TRE}({\mathit{x}}_0,{\mathit{x}}_1;\mathit{\psi })={\Vert {\mathit{x}}_0-\mathit{\psi }({\mathit{x}}_1)\Vert }_2$$ (22)

where x_i denotes a target point in I_i and ψ is the estimated deformation.

2) Invertibility ( $\mathcal{J}$)

A desirable characteristic of diffeomorphisms as described above is their invertibility, which can be characterized in terms of the residual

$$\mathcal{J}(\mathit{\psi },{\mathit{\psi }}^{-1};\mathit{y},\mathit{z})=\frac{1}{2}\{{\Vert (\mathit{\psi }\circ {\mathit{\psi }}^{-1})(\mathit{y})-(\mathit{y})\Vert }_2+{\Vert ({\mathit{\psi }}^{-1}\circ \mathit{\psi })(\mathit{z})-\mathit{z}\Vert }_2\}$$ (23)

where y is a point in I₀, z represents a point in I₁, ψ is a diffeomorphism, and ψ⁻¹ is its inverse.

3) Minimum of Jacobian Determinant ( $\mathcal{D}$)

Singularity as well as change in topology (i.e., folding and tearing) occur if the Jacobian determinant of a deformation is less than or equal to 0 [48]. To quantify preservation of structures as well as invertibility, we measured the minimum value of the Jacobian determinant as

$$\mathcal{D}(\mathit{\psi })=\underset{\mathit{x}\in \mathrm{\Omega }}{min}\left\{\text{det}(D_{\mathit{x}}\mathit{\psi }(\mathit{x}))\right\}$$ (24)

where det(·) denotes a matrix determinant.

B. Physical Experiments

1) Image Acquisition

As shown in Fig. 3, an ovine spine was enclosed in a MR-CT compatible and bendable plastic cylinder filled with polyvinyl alcohol (to simulate soft-tissue). The phantom was imaged first with scoliotic curvature for a T₂-weighted MR moving image (I₀), followed by T₂-weighted MR and intraoperative CT fixed images (I₁) with the spine straightened. The MR scans were acquired with 3D acquisition on a 1.5T Magnetom Avanto (Siemens Healthcare, Malvern PA). The MR I₀ was reconstructed at 0.9×0.9×0.9 mm³ with a size of 192×384×128 voxels, and the MR I₁ was reconstructed at 0.5×0.5×0.9 mm³ with a size of 192×384×144 voxels (Fig. 3(b)). The CT image was acquired with a Somatom Definition Flash scanner (Siemens Healthcare, Erlangen, Germany) (100 kVp, 291 mAs) and reconstructed at 0.6×0.6×0.8 mm³ with a size of 256×256×312 voxels (Fig. 3(c)). For visualization and target point definition (TRE calculation), the vertebrae in images were segmented, and 85 target points were defined on unambiguous anatomical features (tips of the spinous processes, transverse processes, and ribs). The MR I₀ and CT I₁ images were used as the moving and fixed images, respectively, in the following experiments. The MR I₁ was visually compared to the transformed MR I₀.

Fig. 3.

Ovine spine phantom. (a) Phantom assembly. (b) T₂-weighted MR images with the scoliotic spine (I₀) and the straight spine (I₁). (c) CT image with the straight spine (I₁).

2) MR-to-CT Registration Performance

The registration performance of MIND Demons was evaluated in comparison to that of MI FFD, LMI FFD, NMI Demons, and MIND-elastic implemented using the parameter settings established as described in Appendix II.A. Specifically, the stencil for MIND-elastic was a six neighborhood with Δ_S = 3 voxels, σ_p = 0.5, and α = 0.1 similar to the values used in [24] (see Appendix II.A.3.d). The three-level image pyramids for MIND-elastic and the Demons-based methods were constructed with Gaussian kernel widths of [4, 2, 1] voxels and downsampling factors of [8, 4, 2] voxels, while those for the FFD-based methods were constructed using only Gaussian smoothing without downsampling.

C. Clinical Studies

An institutional review board (IRB) approved retrospective study was performed to validate the registration performance of MIND Demons using clinical images. The study used three pairs of T₂-weighted MR and CT images acquired for three patients undergoing intervention of cervical, thoracic, and lumbar disorders at our institution.

1) Image Acquisition

The T₂-weighted MR images and their corresponding CT images (for the cervical, thoracic, and lumbar spines) exhibit realistic variations in imaging protocols and image quality (Fig. 4). The MR scans were acquired with 2D (sagittal-slice) acquisition on a 3T Signa HDxt (GE Healthcare, Milwaukee WI), a 1.5T Aera (Siemens Healthcare, Erlangen, Germany), or a 1.5T Vantage Titan (Toshiba Corporation, Tokyo, Japan) with slice thickness of ~3 mm and T_E varied from 100 – 120 ms. The CT images were acquired using a LightSpeed Ultra scanner (GE Healthcare, Milwaukee WI) or a Somatom Definition Flash scanner (Siemens Healthcare, Erlangen, Germany) with scan techniques ranging 120 – 140 kVp and 80 – 165 mAs, and reconstructed at approximately 0.3×0.3×0.5 mm³. The MR images of the cervical, thoracic, and lumbar spine were taken as the moving images I₀ (Figs. 4(a,c,e)) and the corresponding CT images as the fixed images I₁ (Figs. 4(b,d,f)). For visualization and target point definition, the vertebrae in MR were manually segmented, and those in CT were segmented using simple bone thresholding after median filtering. Twelve, eight, and eleven target points were identified for the cervical, thoracic, and lumbar spine, respectively.

Fig. 4.

Clinical MR and CT image data. (a) T₂-weighted MR (I₀) and (b) CT (I₁) images of the cervical spine. (c,d) The same, in the thoracic spine. (e,f) The same, in the lumbar spine.

2) Validation of MR-to-CT Registration Performance

The registration performance of MIND Demons was evaluated and compared to that of MIND-elastic using clinical data and the nominal parameter settings for each method. Comparison are shown to MIND-elastic, since the MIND Demons and MIND-elastic methods demonstrated the best registration performance among the methods investigated in the phantom studies (see Sections III.B.2 and IV.A). The studies used a four-level morphological pyramid with Gaussian kernel widths of [8, 4, 2, 1] voxels and downsampling factors of [16, 8, 4, 2] voxels.

IV. Results

A. Physical Experiments of MR-to-CT Registration

Figs. (5, 6) and Table II summarize the overall registration performance of MIND Demons compared to that of the reference methods for the physical experiment in registering MR and CT images of the ovine spine. Fig. 5(b) demonstrates sub-voxel $\mathcal{J}$ (mean $\mathcal{J}=0.008\mathrm{mm}$) with a small interquartile range (IQR) for MIND Demons compared to NMI Demons and MIND elastic. None of the methods introduced folding or tearing of anatomical structures as $\mathcal{D}>0$; however, MI FFD yielded min $\mathcal{D}$ close to 0 (Table II and Figs. 5(c) and 6(b)). The large ranges in $\mathcal{D}$ associated with MI FFD, MIND elastic, and MIND Demons reveal a large change in local volume (i.e., expansion and compression) from large local motions. Computation time for each method is shown in Table II. The FFD-based methods benefit from more optimized implementation (elastix [46]) than the MIND-elastic (custom Matlab [24]) and MIND Demons (custom ITK) methods. MIND-elastic was faster than MIND Demons due to the smaller stencil, non-alternating optimization, and linear interpolation to represent continuous I₀ and I₁, as specified in Appendix II.A.3.d. Table II and Figs. 5(a) and 6 summarize the registration accuracy of each method. The top row in Fig. 6 depicts semi-opaque overlays of the pink MR I₀ and the cyan CT I₁ after registration. The cyan spheres represent the target points in I₁, and the yellow line segments mark the distance between the corresponding target points after registration. The bottom row shows the yellow Canny-edges of the MR I₀ after registration superimposed on the gray MR I₁. MIND Demons achieved statistically significant improvement (p ≪ 0.001) in registration accuracy with mean TRE of 1.7 mm.

Fig. 5.

MR-to-CT registration. (a–c) TRE, $\mathcal{J}$, and $\mathcal{D}$ for each registration method.

Fig. 6.

MR-to-CT registration. (Top) Semi-opaque surface rendering of the pink MR moving image I₀ (after registration) and the cyan fixed CT image I₁. Cyan spheres represent the target points in I₁ and yellow lines mark distances between corresponding target points in I₀ and I₁ after registration. (Bottom) Superimposition of yellow Canny edges of the MR moving image I₀ after registration on the MR fixed image I₁. (a) NMI rigid registration. (b) MI FFD. (c) LMI FFD. (d) NMI Demons. (e) MIND elastic. (f) MIND Demons.

TABLE II.

Summary of Registration Results in Phantom Experiments

Method		MI FFD	LMI FFD	NMI Demons	MIND elastic	MIND Demons
TRE (mm)	Mean	11.3	3.1	5.6	2.4	1.7
	IQR	6.9	1.5	3.8	1.3	1.1
		12.9	3.7	7.3	3.1	2.1
	p-value	< 0.001	< 0.001	< 0.001	< 0.001	–
$\mathcal{J}$ (mm)	Mean	–	–	0.008	0.037	0.008
	IQR	–	–	0.004	0.003	0.002
		–	–	0.011	0.011	0.008
	p-value	–	–	< 0.001	< 0.001	–
$\mathcal{D}$	Min	0.01	0.48	0.70	0.34	0.15
	IQR	0.53	0.80	0.85	0.70	0.61
		0.91	1.01	0.97	0.99	0.93
	p-value	< 0.001	< 0.001	< 0.001	< 0.001	–
Runtime (mins)		1	1	19	67	104

$\mathcal{J}$ was not computed for the FFD methods, since they only provide forward deformation (i.e., mapping I₀ to I₁). The p-values measure statistical significant difference in the measured mean value of each metric from that of MIND Demons.

B. Clinical Studies of MR-to-CT Registration

Figs. 7 and 8 summarize the registration performance of MIND Demons compared to MIND-elastic in clinical studies using MR and CT images of the cervical, thoracic, and lumbar spine. Fig. 7(b) illustrates degradation in invertibility of the deformations estimated by MIND-elastic, particularly in the thoracic spine. The fairly poor performance observed for the MIND-elastic method could arise from susceptibility to image noise for the small stencil. On the other hand, MIND Demons yielded sub-voxel $\mathcal{J}$ in all three cases. Both methods preserved tissue topology, with $\mathcal{D}>0$ (Fig. 7(c)). Fig. 8 shows the MR image of the thoracic spine after NMI rigid, MIND-elastic, and MIND Demons registration relative to the corresponding CT image using surface rendering, superimposition of Canny edges of the CT image, and a checkerboard pattern. The zoomed-in regions show misalignment of cortical-bone edges estimated by MIND-elastic, illustrating the sensitivity of MIND-elastic to undesirable local optima. Fig. 7(a) summarizes the registration accuracy for each spinal section. TRE after NMI rigid, MIND-elastic, and MIND Demons registration for the cervical spine was 3.3 ± 1.2 mm, 2.3 ± 1.9 mm, and 1.6 ± 0.6 mm, respectively. In the same order, TRE for the thoracic spine was 4.4 ± 1.8 mm, 6.5 ± 7.6 mm, and 1.7 ± 0.6 mm, and that for the lumbar spine was 4.3 ± 1.7 mm, 2.4 ± 1.0 mm, and 1.9 ± 0.5 mm. The larger TRE for MIND-elastic in the thoracic case was associated with misalignment of vertebral edges. Alternative parameter settings in the image pyramid or MIND-elastic algorithm could potentially yield improved registration accuracy; however, MIND Demons demonstrated insensitivity to parameter settings, robustness against noise and undesirable local optima, and the ability to resolve deformation induced by variation in patient positioning.

Fig. 7.

Clinical studies of MR-to-CT registration. (a–c) TRE, $\mathcal{J}$, and $\mathcal{D}$ for each registration method measured in images of the cervical, thoracic, and lumbar spine.

Fig. 8.

Clinical studies of MR-to-CT registration of the thoracic spine. From left to right: semi-opaque surface rendering of the cyan fixed CT image I₁ and the pink MR moving image I₀ after registration, zoomed-in regions of the surface overlays marked by white rectangles, superposition of yellow Canny edges of I₁ on the gray I₀ after registration, zoomed-in regions of the Canny-edge overlaid images, checkerboard images of I₁ and I₀ after registration, and zoomed-in regions of the checkerboard images. From top to bottom: registration results after NMI rigid, MIND-elastic, and MIND Demons registration. Green arrows mark misalignment of cortical-bone edges—undesirable local optima reached by MIND-elastic.

V. Discussion and Conclusion

A deformable registration method merging the Demons and SyN approaches for symmetric time-dependent diffeomorphisms has been developed. The algorithm incorporates MIND descriptors and the Huber metric for robust multimodality registration and the Gauss-Newton approach for fast convergence. Sensitivity analysis showed that the Huber metric with a small quadratic region (i.e., ∈ = 0.001 – 0.01) was able to reject outliers from local structural differences captured by corresponding MIND descriptors and provide reliable estimation of the deformation. Locality of the MIND descriptor—determined through the configuration of the stencil and the patches (i.e., values of σ_p and t_p)—led to robustness against intensity distortion, and its patch-based computation reduced sensitivity to image noise. Viscoelastic deformations, with adjustable strength of fluid and elastic models, were able to resolve large deformation in realistically noisy data and obtain fairly accurate registration (sub-voxel TRE < 2.0 mm in clinical studies—potentially suitable for application in spinal intervention [49], [50]). The estimated deformation was diffeomorphic to the extent that topology was preserved with sub-voxel $\mathcal{J}<0.008\mathrm{mm}$ and $\mathcal{D}>0$.

MI-based methods have been somewhat widely used for MR-to-CT volumetric registration; however, MI-based metrics are sensitive to intensity non-uniformity, and they lose robustness when used as local measures [19]. Incorporation of spatial information [19], [51] and/or image features [52], [53] could improve their sensitivity to intensity distortion. However, due to the challenge associated with the assumption of tissue class correspondence, we adopted MIND descriptors as the basis for structural similarity. Such descriptors are not deformation invariant, but this could increase their discriminative power [54].

A diffeomorphism is a bijective map; it therefore assumes consistent anatomical structures to appear in both images. This inhibits applications of the method to resolve deformation involving content mismatch (e.g., due to insertion and removal of surgical tools/implants and/or tissue resection). A method using an asymmetric energy formulation of MIND Demons and additional functionality as in [55], [56] could be applicable in this case.

The time step used in the integration of time-dependent velocity fields was fixed at 0.5 to impose a soft constraint on the conservation of Lagrangian measure-based momentum and improve computational efficiency; however, this degrades temporal smoothness of the estimated diffeomorphisms. A smaller time step could increase smoothness in time and potentially improve registration accuracy, but temporal smoothness is not a necessary requirement of our application, and the increase in computation time may outweigh the benefit. Application of the method in clinical image data demonstrated accurate deformable registration in images of the cervical, thoracic, and lumbar spine. Studies in a larger dataset and with greater degrees of gross deformation are the subject of other ongoing work to further validate the method in additional realistic clinical scenarios.

Considering application to intraoperative images, application of the method to intraoperative CBCT images is the subject of future investigation. To more completely investigate the practical advantages of this method, future studies will include validation in other applications, such as generation of anatomical atlases in which a diffeomorphism is important to allow the knowledge base of the atlas to be transferred to patient-specific target anatomy, data from individuals to be mapped to the atlas coordinate space, and studies on statistics of shape or volume of anatomy of interest [57]. Owing to the voxel-wise nature of the algorithm, distributed and/or parallel computing will be used to improve computation time. Future work may additionally include application of deformation-invariant descriptors, analysis of sensitivity to image noise (e.g., low-dose CT) and artifacts, as well as incorporation of other forms of prior knowledge such as rigidity of the vertebrae to further constrain the solution space.