Intersection Based Motion Correction of Multi-Slice MRI for 3D in utero Fetal Brain Image Formation

Abstract

In recent years post-processing of fast multi-slice MR imaging to correct fetal motion has provided the first true 3D MR images of the developing human brain in utero. Early approaches have used reconstruction based algorithms, employing a two step iterative process, where slices from the acquired data are re-aligned to an approximate 3D reconstruction of the fetal brain, which is then refined further using the improved slice alignment. This two step slice-to-volume process, although powerful, is computationally expensive in needing a 3D reconstruction, and is limited in its ability to recover sub-voxel alignment. Here, we describe an alternative approach which we term slice intersection motion correction (SIMC), that seeks to directly co-align multiple slice stacks by considering the matching structure along all intersecting slice pairs in all orthogonally planned slices that are acquired in clinical imaging studies. A collective update scheme for all slices is then derived, to simultaneously drive slices into a consistent match along their lines of intersection. We then describe a 3D reconstruction algorithm that, using the final motion corrected slice locations, suppresses through-plane partial volume effects to provide a single high isotropic resolution 3D image. The method is tested on simulated data with known motions and is applied to retrospectively reconstruct 3D images from a range of clinically acquired imaging studies. The quantitative evaluation of the registration accuracy for the simulated data sets demonstrated a significant improvement over previous approaches. An initial application of the technique to studying clinical pathology is included, where the proposed method recovered up to 15 mm of translation and 30 degrees of rotation for individual slices, and produced full 3D reconstructions containing clinically useful additional information not visible in the original 2D slices.

I. Introduction

One of the greatest challenges of fetal MRI has been unconstrained fetal motion. Before fast MR sequences were developed, pharmaceutical approaches were taken to sedate the subject, for example, by administering benzodiazepines to the mother [1], [2] or pancuronium bromide directly to the fetus [3], [4]. The introduction of fast MR techniques has dramatically changed the use of fetal MRI. Fast MR sequences applicable to fetuses include echo-planar imaging (EPI) [5], single-shot methods such as half-Fourier acquisition single-shot turbo spin-echo (HASTE) and single shot fast spin echo (SSFSE) [6].

Since these sequences are typically rapid snapshot 2D acquisitions with enhanced contrast, they are acquired as a stack of 2D slices to cover a 3D space [7]. Although fast 2D MR images of the fetus provide insights into fetal brain anatomy, their application is limited to clinical diagnosis or qualitative studies [8], because unconstrained fetal motion can occur between the acquisition of different slices, which distorts the 3D interpretation of the anatomy. While there are several studies on human brain development using volumetric approaches, the scope of the studies covered mainly neonates [9]–[11], or post-mortem fetal subjects [12]. Recent studies on human brain development of fetuses using 2D MR slices can be found in [13]–[15]. The first automated tissue segmentation of developing tissues from motion corrected un-sedated fetal brain anatomy was recently described in utero [16] following the development of 3D reconstruction techniques [17]–[19].

It has been shown that stacks of multiple 2D slices can be used to build a 3D volume of the fetal brain [17], [18]. In the work, three orthogonally planned stacks of 3 mm thick slices were registered by slice-to-volume registration, where the volume was iteratively constructed by the putative estimation of the motion parameters in each iteration. Normalized mutual information (NMI) was employed for the similarity measure of registration, and Gaussian weighted averaging was used for reconstruction. In [20], a similar approach was taken, but using a multiple stacks in a single orientation for both registration and reconstruction. The method was generalized to a wider spectrum of applications including neonates and adults, where the motion correction problem is simplified by the lack of confounding maternal tissue structures. However, unlike earlier techniques where standard clinically acquired orthogonal acquisitions are used to form a 3D volume, the single slice orientation approach is limited in its ability to recover accurate through-plane geometry for cases of through-plane movements. A single slice orientation approach is also not compatible with standard radiological practice, where orthogonal slices need to be acquired for interpretation.

All the approaches above, where the evaluation of the similarity measure is indirectly related to the motion parameters through the reconstruction step, have specific limitations: First, the registration involves the reconstruction of the volume in each iteration, which is computationally intensive. Second, the reconstruction step in the evaluation of the similarity measure slows down the convergence of the optimization. Third, the recovered motion parameters are dependent on the reconstruction method. The approach in [19] has achieved the reconstruction-free registration via matching the intersections between each of two orthogonally planned slice pairs, considering all possible pairings of slices within the multiple slice stacks. This approach defines an energy function as the mean square difference (MSD) [21] between slices. The optimization is simplified by deriving the gradient of the energy function with respect to the motion parameters. In this formulation, the derivatives are functions of the spatial intensity gradient of the 2D images, and have good analytical properties. The complexity of the proposed approach is O(N²M), where N stands for the number of slices and M the 1D size (width or height) of the slices, which is a substantial reduction from O(N²M²) of the existing methods.

In this paper, we present a full description of the approach which we term slice intersection motion correction (SIMC), to both register stacks of clinical 2D MR scans acquired for the human fetal brain and reconstruct a 3D volume that is consistent with the 2D slices. Section II introduces a basic geometry of 2D planes in the 3D space, as well as the energy function of the optimization problem. Section III describes the actual implementation of the optimization and the procedure for the 3D reconstruction from the motion compensated slices. The proposed methods are evaluated by using 3D post-mortem fetal imaging from which motion-corrupted 2D slice stacks are simulated, and by applying the methods to a range of clinically fetal data sets. The results are presented in Section IV.

II. Methods

In this work, we start with the assumption that Q stacks of multiple 2D MR images are acquired through the fetal brain anatomy. We additionally assume that sets of stacks of these images are planned in three approximately orthogonal directions (axial, sagittal and coronal). During the acquisition of the slice stacks, the fetus moves within the mother. We assume the time for the acquisition of a single slice is short compared to the motion of the fetus, and therefore, for the majority of the slices, the images represent geometrically correct anatomy. However, due to the motion of the fetus, and the time required to acquire multiple slices, significant motion can occur both between the acquisition of the stacks and between the acquisition of the slices within a stack. For this reason, the true location of each slice with respect to the fetal anatomy is unknown. We parametrize the location and orientation of the i-th slice in the subject coordinate system by θ_i. Here we consider the intersection of slices in stacks that are planned in approximately orthogonal directions (e.g. for axially planned slices, this will include all the slices in coronally and sagittally planned stacks). For simplicity, we denote all the acquired slices in all the stacks by I₁, ···, I_N, and, in all the acquired stacks, the set of slices that the i-th slice can intersect with by S_i.

The motion estimation problem to be solved is to find the motion parameters θ₁, ···, θ_N that bring the slices back into collective alignment with each other. The proposed solution to this problem is based on the direct slice-to-slice matching approach [19]. When two orthogonally planned slices have an intersection and they are correctly aligned, the intersecting slices should have identical intensity profiles along their intersection. When they are misaligned, as shown in Fig. 1, their profiles do not match, the motion correction can be carried out by manipulating the transformation parameters to improve the match of the two profiles.

Fig. 1.

When two orthogonally planned slice stacks cover a common volume, each slice in one stack always intersects with the slices in the other stack. A sagittal slice (A) and an axial slice (B) from simulated MR image stacks of a premature infant are shown. Panel A and B are viewed in the 3D space in panel C. Two slices can be registered by adjusting the transformation parameters so as to improve the match of their intersection intensity profiles.

Note that intersections are matched across slice stacks. Slices from one orientation sometimes have intersections among themselves and they can also contribute to the registration. However, these intersections are not considered, because they often disappear depending on the relative slice motion, yielding a discontinuity in the energy function. The registration makes use of the intersections between orthogonally planned slices only, as opposed to those slices planned to be parallel. For example, the motion of axial slices is recovered using coronal and sagittal slices, and so is the case for coronal or sagittal stacks.

In the context of matching 2D slices of the same MRI sequence, we assume the MRI contrasts are identical and, unlike matching a slice to the reconstructed volume, images have similar levels of blurring, which allows the use of difference based dissimilarity measures in the alignment. Here we use a spatially weighted mean square intensity difference (MSD) of the signal between slices, unlike the normalized mutual information (NMI) in [18] or the cross-correlation (CC) in [20]. In this problem, the weighted MSD energy function provides more constraints by forcing similar intensity values in two images to correspond during registration, and hence can provide improved capture range and a reduced chance of falling into local minima.

A. Coordinate system

At the beginning of the procedure, one of the stacks is manually or automatically chosen to be the reference stack. The coordinate system of the center slice in the reference stack is used as the reference coordinate system, or the subject coordinate system. A floating coordinate system is defined for each of the slices. The floating frame of a slice is related to the reference frame by

$$\mathbf{X}=\text{R}\mathit{x}+\mathit{t}$$ (1)

where X represents the position vector in the floating frame, x the position vector in the reference frame, R a 3-by-3 rotation matrix, and t a translation vector, respectively (Fig. 2A). Since the plane of the slice is defined to be the XY-plane of its floating frame, the equation of the slice plane in the reference frame is obtained from the third component in (1),

Fig. 2.

(A) The reference frame, and the floating frame of one slice. The two coordinate systems are related by six motion parameters—three for translation (t) and three for rotation (R). (B) Parametrization of the intersection location. The parameter λ is 0 at the projection of the origin of the reference space onto the intersection line, and increases as it extends along the intersection direction, namely, v_12⊥

$${\mathit{v}}^{\top }\mathit{x}+t_3=0,$$ (2)

where v, the normal vector of the plane, is the transpose of the third row of R, namely, v^⊤ = R_3,*.

For any two non-parallel planes, there exists an intersection line that is a subset of both planes. The line can be parametrized by λ;

$$\mathit{x}=\lambda {\mathit{v}}_{12\perp }+{g\mathit{v}}_1+{h\mathit{v}}_2,$$ (3)

where v_12⊥ = v₁ × v₂/|v₁ × v₂| is the unit vector pointing to the direction of the intersection line (Fig. 2B). The mapping equations of this intersection line from the reference frame to the floating frames of the two slices are then

$${\mathbf{X}}_1={\text{R}}_1(\lambda {\mathit{v}}_{12\perp }+{g\mathit{v}}_1+{h\mathit{v}}_2)+{\mathit{t}}_1$$ (4)

$${\mathbf{X}}_2={\text{R}}_2(\lambda {\mathit{v}}_{12\perp }+{g\mathit{v}}_1+{h\mathit{v}}_2)+{\mathit{t}}_2.$$ (5)

Since X_1z and X_2z are 0 in the floating frames, the third components of (4) and (5) are,

$$X_{1z}=g+{h\mathit{v}}_1·{\mathit{v}}_2+t_{1z}=0$$ (6)

$$X_{2z}={g\mathit{v}}_1·{\mathit{v}}_2+h+t_{2z}=0$$ (7)

Thus, g and h of (3) are found;

$$g=\frac{t_{2z}{\mathit{v}}_1·{\mathit{v}}_2-t_{1z}}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2}$$ (8)

$$h=\frac{t_{1z}{\mathit{v}}_1·{\mathit{v}}_2-t_{2z}}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2}.$$ (9)

B. Energy Function

The fetal motion is estimated by finding the motion parameters (θ₁, ···, θ_N ) that minimize the global energy function E, which is defined by the combined dissimilarity of the intersection profiles between all combinations of any two orthogonally planned slices, e.g. an axially planned slice vs. a coronally planned slice, etc;

$$E(I_1,\cdots ,I_N;{\mathit{\theta }}_1,\cdots ,{\mathit{\theta }}_N)=\sum _{i=1}^N\sum _{j\in S_i}{\mathcal{D}}_{ij}$$ (10)

where N denotes the total number of slices, S_i the set of slices in the stacks orthogonally planned to the i-th slice, I_i the i-th slice image, and θ_i the set of the motion parameters of the i-th slice, namely, θ_i = [t_ix t_iy t_iz θ_ix θ_iy θ_iz ]^⊤, respectively. The dissimilarity measure D_ij is defined between two intersecting slices i and j, drawn from two orthogonally planned stacks,

$${\mathcal{D}}_{ij}\equiv {\mathcal{D}}_{ij}(I_i,I_j;{\mathit{\theta }}_i,{\mathit{\theta }}_j).$$ (11)

As mentioned earlier, the registration energy function to be minimized is chosen to be the MSD of the intensity profiles between two slices which are spatially weighted using a windowing function w_ij to exclude maternal tissues (described in further detail in the following sections) and averaged over all intersections of orthogonally planned slices,

$${\mathcal{D}}_{ij}=\frac{1}{N_w}\sum _{\lambda \in I_i\cap I_j}{w}_{ij}(\lambda ){\{I_i({\mathbf{X}}_i(\lambda ;{\mathit{\theta }}_i,{\mathit{\theta }}_j))-I_j({\mathbf{X}}_j(\lambda ;{\mathit{\theta }}_j,{\mathit{\theta }}_i))\}}^2,$$ (12)

where the vector X_i(λ; θ_i, θ_j) is a position vector in the i-th image, along the intersection between the i-th and j-th images parametrized by λ, as defined in (4) and (5), and N_w = Σ_i,jΣ_λ∈Ii∩Ij w_ij (λ) is the normalization factor. The complete energy function is, thus,

$$E(I_1,\cdots ,I_N;{\mathit{\theta }}_1,\cdots ,{\mathit{\theta }}_N)=\frac{1}{N_w}\sum _{i=1}^N\sum _{j\in S_i}\sum _{\lambda \in I_i\cap I_j}{w}_{ij}(\lambda )\times {\{I_i({\mathbf{X}}_i(\lambda ;{\mathit{\theta }}_i,{\mathit{\theta }}_j))-I_j({\mathbf{X}}_j(\lambda ;{\mathit{\theta }}_j,{\mathit{\theta }}_i))\}}^2.$$ (13)

The partial derivative of (12) with respect to a motion parameter, namely θ_i is

$$\frac{\partial {\mathcal{D}}_{ij}}{\partial {\theta }_i}=\frac{2}{N_w}\sum _{\lambda \in I_i\cap I_j}{w}_{ij}(\lambda )(I_i-I_j)\times \left(\frac{\partial X_i}{\partial {\theta }_i}\frac{\partial I_i}{\partial X_i}+\frac{\partial Y_i}{\partial {\theta }_i}\frac{\partial I_i}{\partial Y_i}-\frac{\partial X_j}{\partial {\theta }_i}\frac{\partial I_j}{\partial X_j}-\frac{\partial Y_j}{\partial {\theta }_i}\frac{\partial I_j}{\partial Y_j}\right).$$ (14)

Thus, ∂ /∂θ_i is a function of the two slice images, the spatial derivatives of the images, and the variation of the location of the intersection line within the two slices with respect to the motion parameter θ_i, which can be found in Appendix A.

C. Fetal Head Windowing

One of the challenges of the rigid fetal brain tissue registration problem is the influence of deforming maternal tissues that surround the fetal head. This is particularly important because these regions can contain high contrast boundaries (due, for example, to amniotic fluid) that can dominate the response of the registration measure and lead to poor alignment of slices that contain smaller proportions of brain tissue. The influence of these tissues can be addressed by delineating the brain from all slices beforehand [20], [22]. However, such a process is much more complex than skull striping in adult brain anatomy and is difficult to reliably automate for 2D T2W fetal slice data containing maternal tissues.

No spatial windowing was used in [18] except for an approximate rectangular bounding box. In [19], the intersection profile is weighted by a cosine bell window given by, $w_{ij}(\lambda )={cos}^2\{\frac{\pi }{2{\lambda }_1}(\lambda -{\lambda }_0)\}$, where λ₀ and λ₁ are adjusted to fit the bell shape to the intersection line segment. However, this approach does not account for the true 3D form of the fetal head, and still allows the contribution of matching non-fetus signals when matching two intersections at the edge of the brain. It also provides a poor window for highly misaligned slice pairs. Critically, being defined in the intersection coordinate frame, there is no way of enforcing anatomical consistency in the aperture between slice pairs.

Thus, it is a more natural choice to use the 3D windowing function to exclude the contribution of the non-fetal head tissues. In this work we have used a true 3D model based spatial windowing approach using an approximate model for the rigid fetal head anatomy. We define this spatial region of interest in a consistent 3D coordinate system related to the average space of the slice stacks as their alignment evolves, which provides a weighting to reduce the influence of deforming maternal tissues.

An ideal windowing function would be the exact aperture with the shape of the fetal head, or at least an average of fetal head images. However, the detailed shape and size of the brain can differ significantly between fetuses. As a reasonable approximation for the purpose of spatial windowing, we have used a parametric model consisting of an ellipsoid that is defined within the field of view of the subject;

$$w_{ij}(\lambda )=\mathcal{E}(\lambda {\mathit{v}}_{ij\perp }+{g\mathit{v}}_i+{h\mathit{v}}_j),$$ (15)

where (x) is the 3D ellipsoid volume to constrain the location of the fetal brain, and λ is related to the spatial location by (3). Figure 3 is an example of the intersection profile between an axial and a sagittal slice of a fetus at 22 weeks of gestational age, and the windowing function along the intersection.

Fig. 3.

The intersection between two orthogonally planned slices. (GA 22 weeks under normal development, TR/TE = 4500/90.) The bottom panel shows the two intersection profiles of the slices, and 1D profile of the 3D ellipsoidal windowing function.

D. Parametrized motion estimation

The goal of the registration is to recover the exact motion of the fetus imposed on each slice acquisition. A motion parameter value at a time point is strongly correlated with other parameter values in its temporal neighborhood, particularly when the fetus undergoes a smooth motion.

The work in [18] took this property into account by grouping temporally adjacent slices together, and splitting the groups into half in the process of registration. The benefit of this grouping includes the reduced dimensionality and the suppression of local minima. This grouping scheme, i.e., to group the parameters into one, two, then four groups etc., can be represented by the Haar basis set (Fig. 4A). The Haar basis set, however, is not the ideal function to represent the fetal motion, which is rather smooth. Even when sudden motion occurs, there is no guarantee that the moment of the sudden motion matches the splitting of the Haar basis functions.

Fig. 4.

Basis functions for motion estimation: (A) Haar basis functions and (B) discrete cosine basis functions.

In this work, accounting for slice interleave, we parametrize the motion using a discrete cosine basis set (Fig. 4B), and compare the performance with the existing grouping scheme. We use the type II discrete cosine basis functions;

$${\theta }_m^k=\sum _{n=1}^N{\alpha }_n^{k}cos\left\{\frac{n\pi }{N}\left(m+\frac{1}{2}\right)\right\},m=1,\cdots ,N,$$ (16)

or, in a vector-matrix formulation,

$${\mathbf{\Theta }}^k=\mathbf{B}{\mathit{\alpha }}^k,$$ (17)

where the B_m,n corresponds to the motion parameter of the m-th slice of the n-th discrete cosine basis function, and θ^k and α^k are the vectors of the motion parameters in temporal order and the coefficients of the discrete cosine representation, respectively, in the k-th translational or rotational direction.

The optimization starts with a small number of basis functions to provide an approximate estimate, and the number of the basis functions is gradually increased for higher accuracy. For the Haar basis functions, the number of the basis functions is doubled at the moment of basis increment, while the number of the discrete cosine basis functions can be increased one by one without losing symmetry. An example of parametrized motion compensation using the discrete cosine functions is viewed in Fig. 5, where the number of basis functions per stack was 1, 9 and 23 from the left to the right.

Fig. 5.

Visualization of motion parametrization using the discrete cosine functions. From the left, the number of basis functions for each stack is 1, 9 and 23, respectively.

III. Implementation

When only two slices are aligned by matching the intersection intensity profiles, the registration determines only 5 parameters out of 6, leaving one rotational degree of freedom undetermined. This is comparable to two hinge flaps free to rotate around the common joint. In order to achieve a unique registration of slices, at least 3 slices are needed, one orthogonally planned with respect to the others so that the singularity in determining the relative motion parameters is removed. The proposed approach, likewise, makes use of at least 3 stacks that are orthogonally planned with respect to one another.

A. Data preprocessing

Fetal MR images are acquired using a standard clinical imaging protocol which uses a conventional single shot fast spin-echo (SSFSE) T2W sequence. These may be gated for maternal breathing and include within-stack replanning by the radiologist to adjust the slice orientations using a real time planning tool when the fetus has moved severely during the acquisition of a stack of slices. Two or three stacks are acquired from each of three orientations—axial, sagittal and coronal. The original slice has 512×512 of voxels with 0.5×0.5 mm² in-plane dimensions. The fetal brain occupies about 100×100 voxels of area, and the rest is filled with maternal organs. A simple rectangular region encompassing the brain and surrounding tissues is manually selected in the reference stack (typically the first axial stack). The corresponding regions in other stacks are then located by matching the stack against the ROI in the reference stack through a global rigid body transformation [23]. The stacks are cropped around this rectangular reference volume, allowing a 20% of margin on each side to encompass the brain under severe motion. The motion parameters from the initial global rigid body transformations between stacks are used for the initial motion parameters for the following registration procedure. The intersection profiles of voxel intensity values are constructed by resampling the image slices using 2D B-spline interpolation within plane.

B. Motion parametrization during optimization

The energy function is minimized by the Levenberg-Marquardt method;

$${\mathrm{\Theta }}^{k+1}={\mathrm{\Theta }}^k-{[2({\nabla }_{\mathrm{\Theta }}D){\nabla }_{\mathrm{\Theta }}^{\top }D+aI]}^{-1}{\nabla }_{\mathrm{\Theta }}E,$$ (18)

where D is the lexicographical representation of the weighted intensity differences along the intersections between all the combinations of orthogonally planned slice pairs, θ the lexicographical representation of the motion parameters, namely, $\mathrm{\Theta }\equiv {[{\mathit{\theta }}_1^{\top },\cdots ,{\mathit{\theta }}_N^{\top }]}^{\top }$, and thus, ${\nabla }_{\mathrm{\Theta }}E=2{({\nabla }_{\mathrm{\Theta }}^{\top }D)}^{\top }D$.

Note that the first order approximation of the Hessian matrix in (18) is the covariance matrix of θ. Figure 6 is an example of the covariance matrix of the intersection profiles of three orthogonally planned stacks. This figure demonstrates the relationship of the dissimilarity measure of registration and the relative location and orientation of each slice. In this figure, it is identifiable that each slice transformation parameter is correlated with all other slice transformation parameters, including both across stack and within stack, directly or indirectly. For example, the X translation of the sagittal stack (X, sg) is positively correlated with Z translation of the coronal stack (Z, cr).

Fig. 6.

An example of the approximated Hessian matrix, calculated for the registration of three orthogonally planned stacks. Axial, sagittal and coronal stacks are denoted by ax, sc and cr, respectively. The translation of a slice in one stack is correlated with the translation of the slices from other stacks. The correlation between a translation and a rotation can be both (+) and (−) within one stack, depending on which side in the stack a slice is located with respect to the center slice of the stack. The rotation-rotation correlation can be identified likewise. Grid lines are added for reading.

When the motion is parametrized by (17), (18) can be rewritten as

$${\mathrm{\Theta }}^{k+1}={\mathrm{\Theta }}^k-{[2({\mathbf{B}}^{\top }{\nabla }_{\alpha }D){\nabla }_{\alpha }^{\top }D\mathbf{B}+aI]}^{-1}{\mathbf{B}}^{\top }{\nabla }_{\alpha }E.$$ (19)

In this formulation, one may drop higher degree dimensions of the basis function set in order to reduce the dimensionality of the Hessian matrix, and also to disregard physically unrealistic high frequency motion, as described in II-D. By implementing the Levenberg-Marquardt optimization for lower degree motion, and the conjugate gradient descent for higher degree motion, we achieve the optimal computational efficiency.

C. Exclusion of individual mis-matched slices for reconstruction

After the slice alignment optimization is complete, although most of the slices converge to their common consensus, some of the slices remain poorly matched. Those slices arise for cases of image degradation by fetal motion during the slice acquisition, partial volume effect (PVE), or from other imaging artifacts in the slice. Such slices are automatically detected by the abnormal level of their mean square intensity difference with respect to all other intersecting slices, evaluated in the spatial window region of the fetal head. A threshold level above which slices are classed as abnormal is defined with respect to the median mean square intensity difference (MSD) for all slices in a stack. The threshold level was empirically set to 1.25 times the median slice MSD in the stack. These rejected slices occurred in less than 1% of the clinical data.

D. Reconstruction by gradient-weighted averaging

While the possibility of super-resolution in MR imaging has been under debate [24]–[28], the reconstruction of a high resolution 3D volume from multiple 2D slices with anisotropic voxel dimensions, which is a super-resolution problem in the classical sense [29], is feasible. In this problem, the low through-plane resolution of one stack is complemented by the in-plane resolution of other stacks planned orthogonally to the stack. Since the in-plane images are sampled beyond the Nyquist rate of the through-plane acquisition in a typical multislice MR image acquisition, the reconstruction resolution in the through-plane dimension is restricted by the Nyquist limit of the in-plane resolution instead of the limit of the through-plane. This image fusion can be achieved by an interpolation of non-uniform sampling points.

In [20], a multilevel scattered data interpolation method using an iterative B-spline interpolation was employed. The point spread function (PSF) weighted averaging method in [18] is a non-iterative approach, which incorporates the information on the image acquisition model. In our work, the reconstruction in [18] is developed to selectively reduce the relative contribution of 2D slice pixels with greater PVE.

Since a typical 2D fetal MR image acquisition has anisotropic voxel dimensions, the amount of PVE varies depending on the relative angle of the image slice with respect to the direction of the underlying tissue boundaries. In general, the voxel of a 2D MR image is a cuboid elongated in the through-plane direction. Voxels in a slice plane parallel to a tissue boundary are more inclined to include multiple tissue types than those in a slice which is perpendicular to the boundary as depicted in Fig. 7.

Fig. 7.

A diagram depicting the through-plane partial volume effect. When the direction of tissue transition (g) is parallel to the normal vector of the slice (n), the voxel at that location may include contributions from different tissue types. When g is close to being perpendicular to n, the voxel at that location is less likely to include different tissue types.

In order to suppress PVE in the reconstructed volume, we penalize the voxels that do not distinguish different tissue types. The degree of PVE is evaluated from the relative angle of the voxel orientation and the direction of intensity gradient in the region of anatomy, namely θ_vg. Fortunately, although the true 3D intensity gradient at the acquired voxel location is unknown, one does not need the true 3D gradient to determine the selective weighting factor, if we weight the voxel on the slice in proportion to |sin θ_vg|, as shown in the following paragraph.

The relationship between the true 3D gradient g and the 2D gradient at its corresponding point on the i-th slice, ${\mathbf{g}}_i^{\prime }$, is given by

$$\mathbf{g}-{\widehat{\mathbf{n}}}_i({\widehat{\mathbf{n}}}_i·\mathbf{g})={\mathbf{g}}_i^{\prime },$$ (20)

where n̂_i is the normal vector of the i-th slice plane. Given the weight of the voxel on the i-th slice is proportional to |sin θ_vg| = |g ×n̂_i|, the gradient weighting of the corresponding voxel on the i-th slice is the magnitude of the 2D gradient;

$$\begin{array}{l}{w}_i=\mid \mathbf{g}\times {\widehat{\mathbf{n}}}_i\mid =\mid \{{\mathbf{g}}_i^{\prime }+{\widehat{\mathbf{n}}}_i({\widehat{\mathbf{n}}}_i·\mathbf{g})\}\times {\widehat{\mathbf{n}}}_i\mid =\mid {\mathbf{g}}_i^{\prime }\times {\widehat{\mathbf{n}}}_i\mid \\ =\left|\right|{\mathbf{g}}_i^{\prime }\left|\right|.\end{array}$$ (21)

Finally, the gradient weighting term is combined with the PSF weighted averaging method, and the resulting estimate of the intensity in the 3D reconstruction is given by

$$I(x)=\frac{{\sum }_i\mathit{PSF}({\mathit{T}}_i(x))w_i({\mathit{T}}_i^{\text{f}}(x))I_i({\mathit{T}}_i^{\text{f}}(x))}{{\sum }_i\mathit{PSF}({\mathit{T}}_i(x))w_i({\mathit{T}}_i^{\text{f}}(x))},$$ (22)

where I(x) is the reconstructed voxel value at x, PSF the point spread function in the 2D floating frame, and I_i the acquired slice image, and the subscript i denotes slice index. The spatial transformation is defined by T_i(x) = R_ix + t_i as in (1), and ${\mathit{T}}_i^{\text{f}}(x)$ represents the closest point on the i-th plane to T_i(x), the transformed location of x.

IV. Results

A challenge in testing the proposed methods is that there is no ground truth available in in utero fetal brain imaging. In earlier work [18] we made use of related premature neonatal imaging where a full T1W 3D acquisition was available (GA=28 weeks and older), from which simulated 2D acquisitions with motion artifacts could be created. This approach to validation was also replicated in the later work of [30] and then in [20] using T2W 3D images of a premature neonate (GA=30 weeks) and adult images. The key limitations of this approach to simulation are the difference in image contrast between typical T2 weighted fetal imaging and T1 weighted 3D neonatal imaging, the absence of confounding maternal tissues and also the differences in the developmental ages of the premature neonate and the fetus. To provide a more realistic evaluation of the fetal brain imaging problem, in this work we have made use of high quality post-mortem T2 weighted imaging of a fetal brain at an early stage of brain folding development to which we added 2D motion artifacts and contributions from surrounding maternal tissues acquired from in vivo studies. As in [18], a set of simulated motion-corrupted 2D slice stacks were formed using this data to simulate in utero imaging during fetal motion independent of maternal tissues, but with known slice motion parameters. In addition to this simulation, the methods were tested on clinical scans, and the reconstructed volumes were visually inspected.

A. Synthesizing clinical 2D studies from Post-mortem 3D MRI

True 3D fetal imaging data used to evaluate the accuracy of registration method consisted of a fast relaxation fast spin echo (FRFSE) sequence acquired as a 3D T2W volume with TR = 3000 ms and TE = 82.8 ms, and voxel dimensions = 0.59×0.59×1 mm³, using a 3T system (GE Medical Systems, Milwaukee, Wis.). The fetus had gestational age (GA) of 21.28 weeks.

In each simulated study, three orthogonal stacks are generated, and each stack consists of multiple slices of 1×1×3 mm³ voxel dimensions with no overlap, yielding 3-fold effective overlap from 3 stacks. Fetal motion was simulated by adding two motion points during the simulated acquisition of each stack, where the amount of motion is randomly chosen within a preset range. The motion was temporally smoothed using a Gaussian kernel for physical reality (Fig. 9). The air region of the post-mortem scan was replaced by maternal structures that were extracted from an in utero scan. Sets of 2D slices rotated and translated to simulate the head motion trajectory were generated and the maternal tissues surrounding the fetal head were kept fixed to provide a realistic confound for the registration algorithm. The PVE of the lower resolution clinical scans was simulated by applying spatial blurring the volume with a 3D Gaussian kernel, with σs proportional to the resampling voxel dimensions. A set of simulated stacks is shown in Fig. 8A–C.

Fig. 9.

Fetal motion was simulated by adding two motion points during the simulated acquisition of each stack, where the amount of motion is randomly chosen within a preset range. The motion was then temporally smoothed using a Gaussian kernel for physical reality (solid lines). The symbols indicate the estimated motion parameters obtained by registering slices using the proposed method.

Fig. 8.

Example motion corrupted slice stacks created from post-mortem fetal brain image data of a terminated pregnancy at GA 21.28 weeks, resampled in three different orientations with simulated fetal motion and maternal tissues added. Axial (A), sagittal (B) and coronal (C) stacks are generated with 22 slices in each orientation, and with voxel dimensions 1×1×3 mm³. (D) The 3D ellipsoidal spatial windowing function used select brain tissues during alignment.

In total, 100 data sets with different levels of fetal motion were generated. The simulated stack sets were then registered by the proposed methods, using three different approaches of spatial windowing and trajectory parametrization: (A) Intersection based 1D cosine bell spatial windowing along the intersection line and Haar basis functions for motion parametrization, (B) Full 3D ellipsoidal spatial windowing and Haar basis functions, (C) Full 3D ellipsoidal spatial windowing and the discrete cosine basis functions.

The registration accuracy against the ground truth for the synthesized data was evaluated by calculating the slice intersection error (SIE). This can be derived by considering two slices that intersect: With the correct estimate of the movement of the fetal brain, the sampling points on the two intersecting line segments have zero separation (Fig. 10A). If the estimated motion deviates from the ground truth, the sampling points on the intersection line segments will be separated from one another (Fig. 10B), where the separation between a pair of sampling points can be evaluated. The SIE between two slices i and j is derived from Eqs. (3)–(5) and defined by

Fig. 10.

Measurement of the registration error between two slices. (A) Two slices are correctly registered, the true intersection lines have zero separation. (B) The registration is incorrect, two intersection lines are separated. The slice intersection error (SIE) is the square of this separation, averaged over all sampling points.

$${\mathit{SIE}}_{ij}={\left|\right|{\widehat{\text{R}}}_i^{-1}({\text{R}}_i^{\ast }\mathit{x}(\lambda )+{\mathit{t}}_i^{\ast }-{\widehat{\mathit{t}}}_i)-{\widehat{\text{R}}}_j^{-1}({\text{R}}_j^{\ast }\mathit{x}(\lambda )+{\mathit{t}}_j^{\ast }-{\widehat{\mathit{t}}}_j)\left|\right|}^2,$$ (23)

where superscript * and ∧ denote the estimated transformation and the ground truth, respectively.

In order to evaluate the registration accuracy of the entire stack set, the mean slice intersection error (MSIE) was computed for all slice intersections. Figure 11 is the comparison of the MSIE before and after the registration. In this result, the proposed slice intersection motion correction (SIMC) method is shown to perform better than the existing reconstruction based registration (RBR) method. In particular in terms of accuracy, SIMC using a full 3D ellipsoid windowing approach together with a discrete cosine (DC) motion parametrization provides the lowest error. In terms of the robustness or capture range for larger fetal movements, this combination also outperforms other approaches. An illustration of one of the reconstructed volumes from the simulated test data sets is shown in Fig. 12.

Fig. 11.

The Mean Slice Intersection Errors (MSIEs) of the proposed registration method, using three different conditions (Full 3D ellipsoid/discrete cosine (DC), Full 3D ellipsoid/Haar, and 1D cosine bell weighted motion compensation/Haar motion basis functions) are plotted in mm² along with the existing reconstruction based registration (RBR) method. The x- and y-axes represent the initial and the final MSIE. Stacks of a post-mortem fetal brain image from a terminated pregnancy at GA 21.28 weeks were used, resampled in three different orientations with simulated fetal motion and maternal tissues.

Fig. 12.

A comparison between (A) original in-plane views of the stacks with simulated fetal motion of Fig. 8 and reconstructed volumes, with (B) ground truth motion parameters, (C) full 3D ellipsoid weighted motion compensation/discrete cosine motion basis functions, (D) full 3D ellipsoid weighted/Haar, (E) 1D cosine bell weighted/Haar, (F) reconstruction based motion compensation, and (G) no motion compensation. The mean slice intersection error was B:0.00, C:0.04, D:0.99, E:1.03, F:23.01 and G:34.87 mm², respectively.

B. Retrospective 3D image formation from clinical fetal MRI data

We applied the SIMC method to reconstruct 3D images retrospectively from clinically acquired fetal imaging studies carried out at UCSF. This is a challenging and realistic data set that, unlike the work of [20], does not make use of careful pre-processing to remove maternal tissues but instead focuses on the retrospective use of algorithms on clinically standard imaging protocols. The imaging data consisted of single-shot fast spin-echo (SSFSE) T2-weighted image sets of fetuses, acquired during normal maternal breathing using a 1.5T system (GE Medical Systems, Milwaukee, Wis.). Each stack typically consists of between 20 to 30 slices, acquired with an interleave of two. The repetition time (TR) ranged from 3000–11000 ms, and the echo time (TE) 86–100 ms. The in-plane resolution was approximately 0.5×0.5 mm², and the slice thickness was 3 mm. The subjects were 34 fetuses with gestational ages ranging from 20 to 34 weeks, under normal development. Eleven of the subjects had two scanning time points, so in total 45 separate data sets were used in the test. The mean and median of the GA at the moment of scan were 23.43 and 25.60 weeks, respectively. The original scans often include multiple stacks in each orientation, but up to 2 stacks were used in the test for a fair comparison. Each data set is preprocessed as described in subsection III-A. A full 3D ellipsoid function was employed for spatial windowing of the registration, and the discrete cosine basis functions were used for motion parametrization.

In total 45 data sets are visually inspected by a pediatric neuroradiologist with specific expertise in fetal MRI (OG) to confirm that gray matter (GM), white matter (WM), germinal matrix (GMat; a tissue type observed during a short period in development) and cerebrospinal fluid (CSF) are separable and their signal intensities fall into a consistent range over the entire image. The algorithm provided very accurate reconstructions in 24 cases without any form of manual intervention (Type i). On 10 other data sets, the algorithm provided a visually acceptable reconstructions, but with local regions of artifacts due to the presence of misregistered outlier slices (Type ii). For these data sets, the outlier slices are manually marked after investigating using a visualization tool instead of the slice rejection criterion described in subsection III-C. For the remaining 11 data sets, the algorithm did not provide a visually acceptable reconstruction automatically (Type iii). For these studies, it was found that for 9 of them, manual reinitialization of stack motion parameters which were highly misaligned, combined with a re-run on the algorithm, provided high quality 3D reconstructions. This indicated that for these small fraction of cases the capture range of the algorithm was not adequate to recover motion for some slices. The types of reconstruction are counted and presented in Table I, and examples of the types are shown in Fig. 13.

TABLE I.

Registration/reconstruction results of in utero fetal brain scans.

Type	Initialization	Slice Selection	Count
(i)	Automatic	Automatic	24
(ii)	Automatic	Manual	10
(iii)	Manual	Automatic	2
	Manual	Manual	7
	Failed		2
	Partially failed		2
	Total		45

Fig. 13.

Reconstruction results of subjects at similar ages. (A) GA = 22.3 wks (B–C) GA = 22.4 wks (D–F) GA = 22.7 wks (G) GA = 22.4 wks. The data set in G column completely failed even with the manual initialization. AA: Automatic initialization, automatic slice selection, AM: Automatic initialization, manual slice selection, MA: Manual initialization, automatic slice selection, MM: Manual initialization, manual slice selection.

Figure 14A–C display views in axial, sagittal, and coronal stacks, of one of the subjects at 22 weeks of GA, and Fig. 14D the reconstructed volume in the corresponding cross sections. Note that the through-plane distortion is removed, and the noise is reduced in the reconstructed volume. In addition a number of anatomical structures are more clearly defined in the reconstructed volume than in the original 3 mm slice data, for example, the gray matter region which appears degraded in the original stacks (arrows).

Fig. 14.

The original stacks in 3 different orientations, (A) axial, (B) sagittal, and (C) coronal. (GA 22 weeks under normal development, TR/TE = 4500/90.) (D) Reconstructed volume in the corresponding cross sections. (2 axial stacks (23 and 21 slices), 2 sagittal (19 and 19) and 2 coronal (23 and 23) are used. Automated initialization and slice selection.)

The behavior of individual slices is depicted in Fig. 15. Before the registration, the edges of the slices are aligned (Fig. 15A) while the brain anatomy and the eyeballs appear misaligned (Fig. 15B). After the registration, the tissue types are now aligned (Fig. 15D) returning the anatomy to correct correspondence (Fig. 15C).

Fig. 15.

Three dimensional visualization of the slices. (A) The edges of individual slices from six stacks are shown in the initial configuration after rigid stack alignment. (B) Three orthogonally planned slices in the initial configuration after rigid stack alignment. (C) The edges after motion compensation. (D) The same slices after motion compensation

After the registration, outlier slices that are either degraded or initially badly misregistered are excluded by the automated procedure described in III-C, using a thresholding factor=1.25. A few examples of degraded outlier slices in the coronal direction are depicted in Fig. 16 in comparison with a normal coronal slice. The normal slice in Fig. 16A shows clear separation between CSF (white), GM (black) and WM (gray), while these structures are blurred or darkened in the degraded slices. For 17 of the data sets in the experiment for which the automated procedure failed to reject all outlier slices, the slices were manually excluded using a 3D slice visualization tool.

Fig. 16.

Examples of a normal slice (A), a slice degraded by motion during the slice acquisition occurring in-plane (B), through-plane (C), and through-plane partial volume effect for a slice at the edge of the brain (D). In our experience, the occurrence of these artifacts is less than 1% of the acquired slices in our clinical studies. All planes shown are coronal.

Figure 17 depicts the amount of slice motion compensation in translation (top panel) and rotation (bottom panel) estimated for each subject as a function of the GA, after the outlier exclusion. The medians of the magnitude of the translation and rotation are marked by filled circles, along with the quartile range and the full range in a solid and a dashed line, respectively. Note that the proposed method recovers up to 15 mm (equivalent to 5 slices) of translation, and 30 degrees of slice rotation. However, no noticeable dependence of the amount of motion on GA was observed in this study.

Fig. 17.

The distribution of the magnitude of the recovered slice translation (top) in mm and rotation (bottom) in degrees for all studies, plotted as a function of the gestational age (GA). The median magnitude of each study is marked by a filled circle, along with the quartile range in a solid line and the full range in a dashed line. These results show the ability of the algorithm to recover significant motion of the fetal head during the imaging.

C. An Example of Clinical Applications

In order to provide an initial clinical evaluation of the utility of the approach, comparative image analysis procedures were applied to original and motion corrected data. Data from a study of a fetal brain with agenesis of the corpus callosum (ACC) (GA=24.57 wks, TR/TE=8000/91.168) and a normal fetal brain (GA=24.00wks, TR/TE=6666/90.432) at a similar GA were studied. The subjects were scanned using the scanner described in IV-B with the voxel dimensions of 0.47×0.47×3.00 mm³.

Figure 18 illustrates both the original data, the 3D reconstructions and the improvement in interpretation of the data made possible by a true 3D reconstruction. Acquisition of a true mid-sagittal slice with real time planning is a considerable challenge and depends on the movement of the fetus during the study. After the 3D reconstruction it is possible to easily select a mid-sagittal slice from the volume between the left and right lobes of the brain to inspect the midline anatomy as shown in Fig. 18. When such a slice is located, either CSF or GM and WM will be found in the mid-plane between the hemispheres indicating the severity of the corpus callosum abnormality as shown in Fig. 18B, D. Here, CSF filling the central fissure of the ACC brain is visualized as the bright region in the sagittal view in the top panel. In the normally developing anatomy, the corpus callosum, including fine detail of the connecting white matter and grey matter is visible in the slice in the 3D volume reconstruction of Fig. 18C. In particular, the same features are not visible in the original slice (Fig. 18D), due to both the greater noise and partial volume effects in the single acquired 2D slice data.

Fig. 18.

Sagittal and coronal views of the MR images of a fetal brain with agenesis of the corpus callosum (A,B; GA=24.57 wks, TR/TE=8000/91.168) and a normal control fetus (C,D; GA=24.00 wks, TR/TE=6666/90.432). (A) The motion compensated 3D reconstruction, and (B) the raw clinically acquired sagittal slice stack. Arrows indicate the central fissure filled with CSF. (C) The motion compensated 3D reconstruction with the corpus callosum marked with arrows, and (D) the raw clinically acquired sagittal slice stack, where arrows indicate GM on the mid-plane between the left and right lobes.

As an example of the utility of the 3D reconstructions to study tissue surfaces in 3D, Fig. 19 shows the result of automatic segmentation [16] applied before (top) and after motion compensation (bottom) of images of a normal subject scan, where each column corresponds to the volume rendering of GM, WM, GMat (germinal matrix) and the ventricle, respectively (GA=23.14 wks). While the surfaces without motion correction display shape distortion as well as tissue discontinuities in the GMat, the motion corrected volumes provide anatomically realistic smooth surfaces for this early stage of development. Such segmentation techniques allow the quantitative evaluation of surface folding in fetuses [31].

Fig. 19.

The result of automatic segmentation [16] before (top) and after the motion compensation (bottom). From left to right, each column corresponds to the volume rendering of GM, WM, GMat and the ventricle, respectively (GA=23.14 wks).

V. Discussion and Conclusion

In this paper, we presented a procedure to automatically register slice stacks of fetal MR images, and reconstruct a high isotropic resolution 3D volume that is consistent with the original slice stacks. Matching the intersections between slices provided a direct measure to evaluate the mismatch between slices. By defining an optimization energy function as the weighted mean square difference (wMSD) of intersection intensity profiles, we reformulated the registration problem into a conventional optimization problem, which was solved by combination of the Levenberg-Marquardt and conjugate gradient methods. The confounding influence of maternal tissues was suppressed by a spatial windowing function on the registration measure. A motion parametrization was introduced to reduce computational cost and constrain the registration to physically realistic fetal motion. Finally, we also proposed a novel volume reconstruction technique that adds a gradient based weighting to the existing Gaussian weighting method, in order to penalize voxels degraded by partial volume effect. This was achieved by comparing the relative orientation of voxel alignment with respect to the direction of tissue boundaries in the images.

The slice alignment method has been shown to provide improved accuracy in the recovery of known fetal motion, by using a high resolution post-mortem fetal brain image data set to which simulated motion and maternal tissue artifacts were added. The quantitative evaluation of the registration accuracy was achieved by comparing the slice intersection error (SIE), and this indicates that the proposed method consistently achieves the subvoxel accuracy. For example, when the initial MSIE was varied by up to 60 mm², the method recovered alignment to within 0.1 mm² or better of the ground truth in more than 75% of cases.

The method was then extensively evaluated on a large cohort of clinically acquired fetal brain data sets using visual inspection by an expert radiologist. The result of the test on the clinical data sets showed that the approach, combined with a quick slice initialization step before registration and a slice rejection step prior to reconstruction, provided better than 90% success when applied retrospectively to a range of clinical data. The algorithm recovered up to 15 mm of translation and 30 degrees of rotation for individual slices in this evaluation. For the most challenging studies with extreme motion, a semi-automatic procedure was applied which included stack location initialization, and slice rejection after alignment to remove a small number of unmatched slices after registration and prior to 3D volume reconstruction. The reasons for failure to align some individual slices include noise, inhomogeneous bias field, partial volume effect for slices cutting through the edge of the brain and fetal motion during the acquisition of a slice. These caused the algorithm to match incorrect tissue types together, for example, CSF-skull-amniotic fluid to WM-GM-CSF, because those tissue types have similar T2 response. However these cases can be easily excluded, either automatically or manually.

The proposed reconstruction method can achieve both improved signal to noise by averaging multiple 2D studies and improved resolution by increasing the through-plane resolution up to that of the in-plane resolution. This is achieved by penalizing the contribution of partial-volumed slices, i.e., slices that are parallel to a tissue boundary. The subvoxel motion correction accuracy provided by our new approach will allow the incorporation of improved 3D super-resolution approaches such as those of [32].

The proposed method provides two important advantages: improved 3D resolution to delineate finer scale structures of the fetal brain, and improved signal-to-noise ratio (SNR) by averaging multiple scans to provide a superior delineation of subtle contrasts between developing tissues. Currently, assessment of fetal structures using clinical fetal MRI can be challenging since it is extremely difficult to obtain true orthogonal planes that allow accurate assessment of important brain structures, due to fetal motion. Being able to better evaluate important structures, such as the midline, corpus callosum and cortex, will allow more accurate identification and quantification of fetal brain abnormalities and therefore result in improved prenatal counselling.

Appendix A

Partial Derivatives

The mapping from the floating frame of a slice to the reference frame is a function of the motion parameters of the slice, and this is mediated by g and h, which are also functions of the motion parameters as well, as formulated in (4) and (5). The partial derivatives of the mapping functions (4) and (5) with respect to the motion parameters are derived below.

For translations,

$$\frac{\partial g}{\partial t_{1x}}=\frac{\partial g}{\partial t_{1y}}=\frac{\partial g}{\partial t_{2x}}=\frac{\partial g}{\partial t_{2y}}=0,$$ (24)

$$\frac{\partial g}{\partial t_{1z}}=-\frac{1}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2},$$ (25)

$$\frac{\partial g}{\partial t_{2z}}=\frac{{\mathit{v}}_1·{\mathit{v}}_2}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2}.$$ (26)

For the rotation around the k-axis,

$$\frac{\partial g}{\partial {\theta }_{1k}}=\frac{{\mathit{v}}_1^{\prime }·{\mathit{v}}_2}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2}(2{g\mathit{v}}_1·{\mathit{v}}_2+t_{2z})$$ (27)

$$\frac{\partial g}{\partial {\theta }_{2k}}=\frac{{\mathit{v}}_1·{\mathit{v}}_2^{\prime }}{{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }^2}(2{g\mathit{v}}_1·{\mathit{v}}_2+t_{2z}),$$ (28)

where the apostrophe (′) denotes the partial derivative in the k-direction. The change of the mapping position in the floating frame caused by the change of the motion parameters is,

$$\frac{\partial {\mathbf{X}}_1}{\partial t_{1x}}={[\begin{array}{ccc}1& 0& 0\end{array}]}^{\top },\frac{\partial {\mathbf{X}}_1}{\partial t_{1y}}={[\begin{array}{ccc}0& 1& 0\end{array}]}^{\top },$$ (29)

$$\frac{\partial {\mathbf{X}}_1}{\partial t_{1z}}=\frac{{\text{R}}_1{\mathit{v}}_{2\perp }}{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }+{[\begin{array}{ccc}0& 0& 1\end{array}]}^{\top },$$ (30)

$$\frac{\partial {\mathbf{X}}_1}{\partial {\theta }_{1k}}={\text{R}}_1^{\prime }({\mathit{v}}_{12\perp }\lambda +{g\mathit{v}}_1+{h\mathit{v}}_2)+{\text{R}}_1({\mathit{v}}_{12\perp }^{\prime }\lambda +g^{\prime }{\mathit{v}}_1+{g\mathit{v}}_1^{\prime }+h^{\prime }{\mathit{v}}_2),$$ (31)

$$\frac{\partial {\mathbf{X}}_1}{\partial t_{2x}}=\frac{\partial {\mathbf{X}}_1}{\partial t_{2y}}={[\begin{array}{ccc}0& 0& 0\end{array}]}^{\top },$$ (32)

$$\frac{\partial {\mathbf{X}}_1}{\partial t_{2z}}=\frac{{\text{R}}_1{\mathit{v}}_{1\perp }}{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid },$$ (33)

$$\frac{\partial {\mathbf{X}}_1}{\partial {\theta }_{2k}}={\text{R}}_1({\mathit{v}}_{12\perp }^{\prime }\lambda +g^{\prime }{\mathit{v}}_1+h^{\prime }{\mathit{v}}_2+h{\mathit{v}}_2^{\prime }),$$ (34)

where v_1⊥ is a unit vector that is normal to both v₁ and v_12⊥, namely,

$${\mathit{v}}_{1\perp }\equiv \frac{({\mathit{v}}_1·{\mathit{v}}_2){\mathit{v}}_1-{\mathit{v}}_2}{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }$$ (35)

$${\mathit{v}}_{2\perp }\equiv \frac{({\mathit{v}}_1·{\mathit{v}}_2){\mathit{v}}_2-{\mathit{v}}_1}{\mid {\mathit{v}}_1\times {\mathit{v}}_2\mid }.$$ (36)

The partial derivatives of X₂ are derived in the same manner.