MRI Distortion Correction and Robot-to-MRI Scanner Registration for an MRI-Guided Robotic System

Abstract

Magnetic resonance imaging (MRI) guided robotic procedures require safe robotic instrument navigation and precise target localization. This depends on reliable tracking of the instrument from MR images, which requires accurate registration of the robot to the scanner. A novel differential image based robot-to-MRI scanner registration approach is proposed that utilizes a set of active fiducial coils, where background subtraction method is employed for coil detection. In order to use the presented preoperative registration approach jointly with the real-time high speed MRI image acquisition and reconstruction methods in real-time interventional procedures, the effects of the geometric MRI distortion in robot to scanner registration is analyzed using a custom distortion mapping algorithm. The proposed approach is validated by a set of target coils placed within the workspace, employing multi-planar capabilities of the scanner. Registration and validation errors are respectively 2.05 mm and 2.63 mm after the distortion correction showing an improvement of respectively 1.08 mm and 0.14 mm compared to the results without distortion correction.

I. INTRODUCTION

The development of MRI-guided robotic systems is complicated by the need to track the position and the orientation of these instruments in real-time within the MRI scanner as well as accurately localizing the desired target. The clinically-desired instrument to target accuracy may be defined as the maximum error that can be allowed during an intervention without putting the effect of the therapy in jeopardy or endangering the patient, where such tolerances are application specific [1]. For instance, in the case of an intracardiac ablation procedure a 3 mm instrument to target accuracy might be a clinically sufficient goal, given the ablation catheter can be manipulated with enough precision [2]. Thus, in order to achieve a clinically-desired accuracy, it is necessary to register the robot space to the scanner’s image space. In addition, system-related geometric distortions in MR images could degrade the accuracy of instrument tracking and target localization. This paper investigates the preoperative correction of site-specific MRI distortions and the preoperative registration of robotic tools to the MRI scanner to employ them jointly with real-time high speed MRI image acquisition and reconstruction methods in real-time interventional procedures.

MRI is susceptible to patient and scanner related spatial distortions [3]. Scanner related geometric distortion arises from multiple sources, including gradient coil nonlinearities, magnetic field inhomogeneities and eddy currents [4], [5]. Although slight distortions in MR images do not affect significantly regular radiological examinations, geometric distortion poses serious challenges in certain MRI applications such as image-guided interventions, where precision is a primary concern and high geometric accuracy is required [6]. In the MRI-guided robotic procedures, without correction, this distortion might lead to inaccurate tracking of the robotic device as well as imprecise target localization.

The characterization and measurement of the geometric distortion has been extensively studied by specifically designed phantoms. Initial studies focused on using 2D measurements for the characterization by employing square grids [7], [8] and cylindrical rods [9], [10]. Yet, solely using 2D measurement does not completely solve the problem [5]. 3D phantoms have since been employed for characterizing the distortion [4], [5], [6], [11], [12], [13], [14], [15], [16], [17]. Once the distortion is quantified, two main approaches have been followed for correction; (i) Applying a transformation to the geometric displacements due to distortion measured via the 3D phantom and (ii) Representing the gradient fields via spherical harmonics and calculating the 3D geometric displacements by this expansion [11], [18], [19].

Various robot to image registration methods have been previously developed for MRI-guided robotic interventions. These approaches include using joint encoding [20], passive MRI fiducial markers [21], [22], [23], [24], optical position sensors [25], gradient field sensing [26], and micro tracking coils [27], [28].

Most common of these methods are using either micro-tracking coils or passive MRI fiducial markers. Although the coil-based approaches provide real-time registration and have high accuracy, they require custom scanner programming for each scanner and thus are not easily applicable from one scanner to another [21]. Fiducial marker-based methods are scanner independent thus providing portability. Their performance depends on the number and configuration of the markers. The existing methods also rely on specific designs of fiducial frames or MRI sequences [29], [30].

The goal of this work was to develop a preoperative robot-to-MRI scanner registration approach, while concurrently addressing the scanner related geometric distortions with the future goal of utilizing this method for real-time interventional robotic procedures under MR image guidance [31]. Here, the aim of the preoperatively estimated distortion field is to prioritize using it in real-time MRI-guidance as opposed to a detailed offline distortion analysis.

Although the clinical MRI scanners have distortion correction capabilities [32], they are provided by the proprietary software of the vendors and the resulting distortion map is not directly accessible. The vendor specific distortion correction algorithms can be used solely via vendors’ own user interfaces. Therefore, they are not suitable for real-time interventional procedures employing custom real-time high speed MRI image acquisition and reconstruction methods [33] that are not provided by the vendors. The proposed approach is intended to be utilized in real-time interventional procedures by calculating a distortion map in a prior offline step, and then subsequently use the estimated distortion field in the ensuing images collected in real-time, thereby bypassing the scanner’s proprietary distortion correction feature.

In the presented work, the geometric distortion in the MR images is identified by utilizing a grid-based, custom-built 3D phantom (Fig. 1a). A CT scan of the phantom is acquired to establish ground truth data. Morphological operations are applied to localize the control points both in CT and MRI images of the phantom. The CT and MRI control points are registered to a common coordinate system via coherent point drift algorithm. The corresponding point sets are then used to determine the distortion map. The underlying distortion is modeled and corrected by employing thin-plate splines.

FIGURE 1.

(a) Schematic of the 3D grid phantom used for mapping the MRI distortion field. (b) Schematic of the registration prototype used for the proposed approach. Purple circles indicate the fiducial coils used for registration and validation. (c) Schematic of joint MRI scanner distortion and robot-to-MRI scanner registration method.

A differential image based registration algorithm is presented for the novel steerable robotic catheter system proposed by Liu et al. [34], [35]. A set of active fiducial coils embedded along a registration frame (Fig. 1b) are used for the registration of the MRI image space with the physical robot space. The locations of these coils are known with respect to the robot base from the CAD file that is used to manufacture the frame structure. When the coils are inactive, a multislice image is acquired and used as the background (static) model. Then, the coils are activated by passing electric currents through the coils and the multi-slice image acquisition is repeated using the same MRI sequence parameters. This multi-slice image is used as the foreground (moving) model. By subtracting the background model from the foreground model, a set of circular imaging artifacts are obtained, which give the location of the fiducial coils along the pillars. The multi-slice imaging approach takes the full advantage of the 3D information to provide more accurate fiducial coil detection as opposed to using a single slice.

For each coil, its location is computed by a weighted average of detected artifact centroids among slices. The weights correspond to the artifact area in each slice. Artifact centroid and radius in each slice are estimated via Hough transform. Finally, the registration between these two sets of 3 degrees of freedom (DOF) data, namely, coil locations with respect to the scanner image coordinates and coil locations with respect to the physical robot coordinates, is performed by least-squares fitting.

The distortion correction and registration procedures are validated by various target coils placed inside the workspace. In order to eliminate any other errors, such as robot kinematic modeling inaccuracies, and focus only on the registration error, a set of coils with known baseline coordinates were used as the validation targets. The distortion corrected images provided by the proprietary software of the MR scanner were used to evaluate the baseline performance.

During an interventional procedure, an MR image plane in a specific orientation could provide more information about the environment compared to an image plane in a different orientation [36]. Thus, it is important for the performance of a robot-to-MRI registration approach to be agnostic of the image plane orientation. For this purpose, validation target coils are scanned in various orthogonal and oblique orientations, in addition to the image plane orientation used during the registration step.

The differential image based registration algorithm was originally presented in [37], where the geometric distortion in the MR images was not considered. In order to use this registration approach jointly with real-time high speed MRI image acquisition and reconstruction methods [33] in real-time interventional procedures, this study extends the work in [37] and investigates the effects of the geometric distortion in robot to scanner registration through distortion analysis with additional validation experiments.

MRI distortion mapping and robot-to-MRI scanner registration problems were studied separately by several groups previously. Here, they are investigated jointly within a concise framework, where the presented approach does not require custom scanner programming, receiver channel reservation, or complicated instrument design as in the previous studies. Fig. 1c shows the workflow of the proposed method.

Even though the presented registration approach is planned to be used with the robotic catheter system proposed in [35] for interventional cardiac procedures, it is agnostic of the choice of robotic hardware. It also does not rely on any custom MRI sequences as well as vendor specific algorithm and hardware capabilities. The multi-slice sequences employed for imaging are many singe-slice experiments, where the slice position is adjusted incrementally. Thus, the approach is not restricted to a particular scanner. It is possible to acquire many slices with any scanner by incrementally moving the slice position between scans [38].

To the best of authors’ knowledge, this is the first study that utilizes active fiducial coils in the context of robot-to-MRI scanner registration, where a background subtraction method is employed for coil detection.

The related studies regarding robot-imaging modality registration and MRI distortion correction are given in Section II. The distortion correction and differential registration approaches are respectively described in Section III and Section IV. The experimental setup and procedures are explained in Section V. The results are presented in Section VI. The discussion and conclusions are given in Section VII.

II. RELATED STUDIES

A. MRI DISTORTION CHARACTERIZATION AND CORRECTION

Several previous work studied system related distortions, which are often performed with purpose built phantoms. These phantoms are utilized for control points localization followed by quantitative characterization and correction of site-specific image distortions.

Most of the studies rely on locating control points; i.e. the grid intersections or other markers in the MRI images and comparing the locations to a ground truth. The ground truth data is usually acquired via a CT acquisition of the same phantom [4], [12], [13], [14], [39]. The control points are localized through image-processing tools. After associating the corresponding control points in the MRI image and the ground truth, a displacement field is constructed via interpolation or spherical harmonics [11], [12], [40], which is then used for distortion correction.

Wang et al. [5] built a phantom from grid sheets that is immersed in an MRI-visible solution. The control point positions are extracted by deriving approximate first derivatives in three dimensions via 3D Prewitt operators. It was not mentioned how the ground truth positions of the control points were determined. The authors assumed perfectly spaced regular grid sheets in each of the three dimensions without considering any manufacturing imperfections and employed this assumption to define true positions of the control points.

Baldwin et al. [4] employed a similar phantom and control points extraction technique used by [5] to investigate system-related distortions. They separately assess the distortions due to background inhomogeneities, gradient nonlinearities, and phantom-related susceptibility artifacts, They used CT data as the ground truth and performed the distortion correction via elastic body spline based registration.

Doran et al. [12] utilized a custom built phantom comprised of three orthogonal interpenetrating arrays of water-filled tubes. The MRI control points were localized via a single level intensity threshold, whereas the baseline CT control points were extracted manually. They computed the first order approximation of the 3D distortion field via calculating the mean value of the distortions measured from two orthogonal datasets.

Satenscu et al. [14] used the same phantom in [4]. They localized the control points by an adaptive method to reduce the effects of MRI image intensity inhomogeneities. After generating the distortion map from the displacements of corresponding CT and MRI control points, they performed distortion correction via spatial interpolation.

In [15], Walker et al. constructed a phantom comprised of refined plastic layers, which are inserted with Vitamin E capsules. They used deformable B-splines to register the MRI images to the ground truth CT images, where the resulting deformation field provides the distortion information. Likewise, Nousiainen et al. [13], studied a B-spline based nonrigid CT–MRI image registration method to determine the geometric distortions using a grid-based phantom. For validation, control points in the corresponding CT and MRI images were located via a semi-automated method utilizing template matching and manual adjustments.

Mangione et al. [40] used a 3D lattice phantom comprised of cylinders. They extracted the control points via searching maxima of the cross covariance between images and 3D cross prototypes. They estimated spherical harmonic coefficients to perform distortion correction.

In this paper, the MRI system related geometric distortion is studied by identifying pairs of control points in the corresponding MRI and ground truth CT images of a grid-based, custom-built 3D phantom. The control points are localized by applying a sequence of morphological operations. Thin plate splines are used to model the underlying geometric distortion. [14] and [12] presented similar spline interpolation based methods for generating the distortion maps within the volume of interest. Whereas, those studies performed an extensive 3D distortion characterization by analyzing multiple orthogonal 2D distortion datasets, this study aims to present a concise framework for mapping the distortion field with a single orientation distortion dataset, where the determined distortion field is used in the subsequent robot to MRI scanner registration approach. The resulting concise framework would enable the presented approach to be utilized in real-time interventional robotic procedures [31], [33].

B. ROBOT TO IMAGE MODALITY REGISTRATION

Reliable instrument tracking and precise targeting during image-guided interventions require accurate image-to-robot registration. Several research groups studied the problem of spatial registration of robotic and manual instruments within imaging modalities and more specifically in MRI-guided procedures. Two most-common approaches are utilizing passive and active fiducial markers for registering instrument coordinate system to the image coordinate system.

The passive fiducial marker based registration methods use markers filled with contrast agents, which are detectable by the corresponding imaging modality. Susil et al. [24] presented a registration method for CT-guided interventions based on a Z-frame comprised of nine passive fiducial marker tubes and uses one single image for computing the parameters. The method was generalized by Lee et al. [23] to an algorithm that allows using multiple images.

Tokuda et al. [30] proposed a fiducial frame registration method based on automatic extraction of linear features to mitigate dependence of Z-frame registration to the thresholding performance. In [22], Shang et al. developed multi-image registration method for tracking MRI-guided robots and showed multi-slice method improves accuracy compared to single slice method [24]. This approach is later employed by [41]. Z-frame based instrument to MRI scanner registration algorithm is further utilized in [42], [43], [44], [45], and [46].

Other configurations of MRI-visible markers are used in [21], [47], [48], [49], [50], [51], [52], [53] to register needle-guided systems to the image coordinate system. The marker locations in the needle guidance system are known. The instrument coordinate system are registered to the MRI coordinate system after the segmentation and detection of the markers in the MRI images.

The active fiducial-based registration method employs a number of micro-tracking coils [27], [54], [55] embedded in the end-effector of the device, which locates their spatial position in the MRI scanner. It provides high-accuracy and fast registration. However, it requires custom scanner programming and dedicated scanner channels. Currently, a limited number of scanners support micro-tracking coils as a default capability making it challenging to be portable from one scanner to another.

In this paper, an alternative robot to MRI scanner registration approach is presented. Unlike the Z-frame method, it does not rely on the specific geometric shapes for the fiducial frames and unlike the micro-tracking method, it does not require custom scanner programming, receiver channel reservation, or complicated instrument design. Eight active fiducial coils are embedded on a frame and placed inside the workspace. The locations of these coils in MRI images are detected by background subtraction and using standard imaging pulse sequences. The background subtraction for coil detection utilizes simple thresolding as such the proposed method does not rely on custom thresholding used for detecting markers filled with contrast agents in the Z-frame method. The detected coils are registered to their known ground truth locations in robot base coordinate system by least square fitting. The coicustom thresholding

The method is validated on a set of fixed target coils inside the workspace. These fixed coils are used instead of the actual robot prototype to mitigate the effect of other sources of error such as kinematic modeling uncertainties. The validation approach is different from the previous studies, where the performance of the registration algorithm is investigated via multi-planar imaging capabilities of MRI. DiMaio et al. [43] presents the closest validation approach, as in that study the imaging plane is adapted to automatically follow the motion of the fiducial frame in the scanner. In [43], only a single slice was used for registration, whereas this study utilizes multi-slice images to take full advantage of the 3D information in the registration.

III. CORRECTION OF DISTORTION IN MRI IMAGES

The proposed method for assessing and correcting geometric distortion in MRI images is presented in this section.

A 3D custom-built phantom comprised of parallel grids is utilized to map the geometric distortion. The displacement field representing the geometric distortion is generated by localizing the grid intersections in the corresponding CT and MRI acquisitions of the phantom, where the CT data represents the ground truth.

A. PROBLEM DESCRIPTION

Suppose the control points localized in the ground truth CT images and the MRI images are transformed into the same coordinate system and one-to-one correspondence between the CT and MRI control points are established. Let $P=\{p_1,\dots ,p_N\}$ and $\tilde{P}=\{{\tilde{p}}_1,\dots ,{\tilde{p}}_N\}$ be respectively the CT and MRI point sets with P, $\tilde{P}\subset {ℝ}^d$ is the total number of control points.

The goal of the distortion correction procedure is to recover the optimum transformation f (·) that maps one control point set to the other, such that the distance between these two sets of corresponding points is minimized in the least-square sense:

$$\underset{f(\cdot )}{\text{argmin}}\sum _{i=1}^N{\Vert p_i-f({\tilde{p}}_i)\Vert }^2.$$ (1)

Any point ${\tilde{p}}_j$ in the distorted domain could be corrected via the estimated mapping function f(·); i.e. $p_j\approx f({\tilde{p}}_j)$. Fig. 2 depicts the distortion correction process for ${ℝ}^2$.

FIGURE 2.

The distortion correction process for a single grid. (a) Blue squares show the localized ground truth grid intersections (x_i, y_i). (b) Red circles show the localized grid intersections (${\tilde{x}}_i$, ${\tilde{y}}_i$) in the distorted domain. (c) Shows aligned control points after correction. Any point (${\tilde{x}}_j$, ${\tilde{y}}_j$) in the distorted domain could be transformed with the correction mapping function f (·).

The localization of the control points in CT and MRI acquisitions is explained in Section III-B. The rigid registration of CT and MRI control points into the same coordinate system is described in Section III-C. The correction procedure of the geometric distortion is given in Section III-D.

B. LOCALIZATION OF CT AND MRI CONTROL POINTS

The control points are determined by a semi-automated approach. First, morphological operations are applied to both CT and MRI images to localize grid intersections and identify control points. The resulting control points are visually inspected and any offset points are manually corrected.

Fig. 3 shows the automated steps for a distorted MRI image. First, each image (Fig. 3a) is binarized (Fig. 3b). Adaptive thresholding is used in the binarization steps of each image to account for inter-slice intensity variations across the volume. Then, closing and dilation operations are applied respectively to eliminate any isolated pixels (Fig. 3c) and remove any holes (Fig. 3d). This is followed by skeletonization (Fig. 3e). After obtaining the grid skeleton, the branch points are detected based on the pixel connectivity (Fig. 3f). The branch points are dilated (Fig. 3g) and their centroids are extracted to get the control point locations (Fig. 3h).

FIGURE 3.

Sequence of morphological operations showing the detection of the control points on the phantom for a distorted MRI image. (a) Original image. (b) Binarization. (c) Closing. (d) Dilation. (e) Skeletonization. (f) Finding branch points. (g) Dilation of branch points. (h) Centroids of the dilated branch points superimposed on the original image.

Multiple image slices correspond to a single grid in both CT and MRI images. A weight is assigned to the each detected control point for every slice of a particular grid. This weight is equal to the total number of control points detected in that slice over the total number of grid intersections. Then a weighted average is computed across slices to get the final locations of the control points on that grid.

C. RIGID-REGISTRATION OF CT AND MRI CONTROL POINTS

The rigid registration of the localized CT and MRI control point sets to a common coordinate system and establishing correspondences are achieved via the Coherent Point Drift (CPD) algorithm [56]. In the CPD algorithm, the alignment of two point sets is formulated as a probability density estimation problem, where one point set is considered as the centroids of Gaussian Mixture Models and the other point set is regarded as observations. The CPD algorithm is preferred due to its advantages over other state-of-the-art algorithms such as Iterative Closest Point (ICP), which requires the initial positions of the two point sets to be sufficiently close.

D. CORRECTION OF THE GEOMETRIC DISTORTION VIA THIN-PLATE SPLINES

Thin-plate splines [57] are chosen to model f (·) in (1). They are effective tools for modeling the coordinate transformations. Thin-plate splines have a natural representation as a linear combination of radial basis functions (RBFs), φ, plus a linear polynomial. For ${ℝ}^d$, the coordinate mapping is given by [58]:

$$f(y)=\sum _{i=1}^N{a}_i\phi \left(\Vert {\alpha }_i-y\Vert \right)+\sum _{j=1}^N{b}_j{s}_j(y),$$ (2)

where ${\alpha }_i\in {ℝ}^d$ are the centers for the basis functions in the RBF interpolant and v = f (y) is the target function value at $y\in {ℝ}^d$. a_i’s and b_j’s are respectively the weights of the RBFs and the polynomial. The RBF has the form $\phi =r^2\ln (r)$. s₁(y) = 1 and s_j(y) corresponds to the coordinates of y for j = 2, … , M [59]; i.e. for $y={(y_1,\dots ,y_d)}^{\text{T}}\in {ℝ}^d$, $s_j(y)=y_{j-1}$ for j = 2, … , M with M = d + 1.

Given the CT and MRI control point sets, $\{P,\tilde{P}\}$, the distortion correction requires computing an appropriate mapping function f (·) satisfying $f({\tilde{p}}_i)=p_i$ for all 1 ≤ i ≤ N. This results in a system of N linear equations in N + M unknowns a_i’s and b_j’s (2). The additional unknowns due to the polynomials are eliminated by introducing the orthogonality conditions to ensure a unique solution [58]:

$$\sum _{i=1}^N{a}_i{s}_j\left({\tilde{p}}_i\right)=0,j=1,\dots ,M.$$ (3)

Together, these yield an equivalent system of linear equations [58]:

$$\left[\begin{array}{cc}Q& L\\ L^{\text{T}}& 0\end{array}\right]\left[\begin{array}{c}a\\ b\end{array}\right]=\left[\begin{array}{c}V\\ 0\end{array}\right]$$ (4)

where

$$Q_{i,n}=\phi (\Vert {\tilde{p}}_i-{\tilde{p}}_n\Vert ),i,n=1,\dots ,N,$$ (5a)

$$L_{i,j}=s_j({\tilde{p}}_i),i=1,\dots ,N,j=1,\dots ,M,$$ (5b)

$$a={[a_1,\dots ,a_N]}^{\text{T}},b={[b_1,\dots ,b_M]}^{\text{T}},$$ (5c)

$$V={[p_1,\dots ,p_N]}^{\text{T}},$$ (5d)

which is solved to compute unknowns a_i’s and b_j’s. Once the unknown weights are computed, any point $\tilde{p}\in {ℝ}^d$ could be evaluated with the spline function to correct the distortion. A more thorough treatment of the radial basis functions and thin-plate splines are given in [59] and [60].

IV. ROBOT-TO-MRI SCANNER REGISTRATION

In this section, the approach employed to address the scanner to robot registration is presented. Table 1 and Fig. 4 define the coordinate systems used in the registration process.

TABLE 1.

The notation for the coordinate systems, their corresponding acronyms and origins used in the document.

Frame Origin	Acronym	Coordinate System
S	SCS	Slice Coordinate System
G	GCS	Gradient Coordinate System
D	DCS	Device Coordinate System
P	PCS	Patient Coordinate System
B	BCS	Base Coordinate System

FIGURE 4.

The coordinate systems defined for the registration process. DCS and PCS are drawn outside the bore for a cleaner visualization.

In the proposed scheme, the registration will be performed with the help of a set of active artificial fiducials created in the MRI images. These active artificial fiducials will be in the form of localized MRI image artifacts created by passing electric currents through a set of electromagnetic coils, which will be referred to as “active fiducial coils.” These coils act as “active” fiducial markers, rather than passive fiducial markers, because, it is possible to actively turn them on and off, allowing to control if and when these fiducial artifacts are created in the MRI images.

A. PROBLEM DESCRIPTION

Suppose the ground truth locations of the fiducial coils in robot base coordinate system (BCS) are known from the CAD design file and represented as ${\overline{q}}_B^j$ for j ∈ {1, … , 8}. The same corresponding coils are detected in MRI scanner’s device coordinate system (DCS) and are represented as ${\overline{q}}_D^j$ for j ∈ {1, … , 8}. The transformed coil location from the device coordinate frame (DCS) to the base coordinate frame (BCS) are represented as:

$${\overline{q}}_B^j=R_{BD}{\overline{q}}_D^j+t_{BD}\text{with}j\in \{1,\dots ,8\},$$ (6)

where R_BD represents the orientation of the DCS with respect to BCS and t_BD is the location of the origin of DCS with respect to BCS.

Then, the goal of the registration procedure is finding the optimum rigid transformation, which minimizes the distance between these two sets of corresponding points in the least-square sense:

$$\underset{\left[R_{BD},t_{BD}\right]}{\text{argmin}}\sum _{j=1}^{N_C}{\Vert {\overline{q}}_B^j-{\overline{g}}_{BD}{\overline{q}}_D^j\Vert }^2,$$ (7)

where ${\overline{g}}_{BD}$ is the homogeneous representation of $g_{BD}=(t_{BD},R_{BD})\in SE(3)$¹:

$${\overline{g}}_{BD}=\left[\begin{array}{cc}{R}_{BD}& t_{BD}\\ 0& 1\end{array}\right].$$ (8)

This homogeneous transformation matrix represents the mapping from scanner’s device coordinate system to robot’s base coordinate system. In (7), N_C is the number of coils used; i.e. N_C = 8 if all the fiducial coils are utilized.

The first step of registering the MRI scanner to the robot base is detecting the fiducial coils in the scanner images. Fig. 5 shows the flowchart of the registration procedure.

FIGURE 5.

Flowchart of the proposed registration method.

B. ACTIVE FIDUCIAL COIL DETECTION IN MRI IMAGES

First, a multi-slice image is acquired when the coils are inactive. This image provides the background model. Then, the coils on top side of the pillars are activated by passing currents through coils.² Another multi-slice image is acquired using the same imaging sequence. This image provides the foreground model. As the only change in the scene is the activated coils, performing background subtraction gives an artifact, which corresponds to coil locations among the slices. This step is then repeated for the coils located at the bottom side of the pillars. The order of activating top or bottom coils is of no significance to the presented method. One such sequence is followed here. The images for the background, foreground, and the result of background subtraction for the same slice is shown in Fig. 6.

FIGURE 6.

Foreground (upper left), background (upper right), background subtracted (lower left), and coil detection (lower right) images (coronal orientation) for the same slice.

For each coil, the size of the artifact is largest in the image slice that intersects the center of the coil, and the artifact size gradually decreases as the slices move away from center towards the edges of the coil. For each slice, the centroid and radius of the artifact are calculated with a Hough Transform [61]. The fiducial artifacts for a set of multi-slice image are shown in Fig. 7. With this information, the area of the artifact in each slice is calculated and these areas are used to compute a weighted average of the centroids among slices to get the final detected coil location.

FIGURE 7.

Active fiducial coil artifacts for consecutive slices in a multi-slice image (coronal orientation). Artifact size changes throughout the slices.

C. MAPPING DETECTED FIDUCIAL COILS TO SCANNER DEVICE COORDINATE SYSTEM

The detected coil location from the weighted average computation is then mapped from slice coordinate system (SCS) to scanner’s device coordinate system (DCS) by a sequence of homogeneous transformations given in (9):

$${\overline{q}}_D={\overline{g}}_{DP}{\overline{g}}_{PG}{\overline{g}}_{GS}{\overline{q}}_S.$$ (9)

In (9), ${\overline{q}}_S$ is the detected coil in SCS in homogeneous coordinates and ${\overline{q}}_D$ is the detected coil mapped to DCS. The descriptions of the coordinate systems corresponding to these transformations are given in Table 1. Plugging (9) in (7), ${\overline{g}}_{BD}$ and thus mapping from DCS to BCS can be represented as:

$${\overline{g}}_{BD}={\overline{g}}_{BS}{\overline{g}}_{SG}{\overline{g}}_{GP}{\overline{g}}_{PD},$$ (10a)

$${\overline{q}}_B={\overline{g}}_{BS}{\overline{g}}_{SG}{\overline{g}}_{GP}{\overline{g}}_{PD}{\overline{q}}_D.$$ (10b)

Applied magnetic field gradient coordinate system (GCS) is always at the center of the SCS and has a fixed orientation with respect to SCS. Thus, ${\overline{g}}_{SG}$ is constant. Patient (sagittal, coronal, transverse) coordinate system (PCS) origin matches the origin of the DCS and has a fixed orientation with respect to DCS, and thus ${\overline{g}}_{PD}$ is also constant. Only ${\overline{g}}_{GP}$ changes with each slice in a multi-slice image in the direction of imaging plane with a magnitude of slice thickness. The transformations ${\overline{g}}_{SG}$, ${\overline{g}}_{GP}$, and ${\overline{g}}_{PD}$ are known and provided by the scanner. Only unknown in (10a) is ${\overline{g}}_{BS}$ and estimated via (7):

$${\widehat{g}}_{B{S}_{opt}}=\underset{\left[R_{BS},t_{BS}\right]}{\text{argmin}}\sum _{j=1}^{N_C}{\Vert {\overline{q}}_B^j-{\overline{g}}_{BS}{\overline{g}}_{SG}{\overline{g}}_{GP}{\overline{g}}_{PD}{\overline{q}}_D^j\Vert }^2.$$ (11)

Then, the registration from scanner’s device coordinate system to robot’s base coordinate system is given by:

$${\widehat{g}}_{B{D}_{opt}}={\widehat{g}}_{B{S}_{opt}}{\overline{g}}_{SG}{\overline{g}}_{GP}{\overline{g}}_{PD}.$$ (12)

The detected coil location converted from pixel values to Cartesian coordinates based on the pixel resolution. As there are four fiducial coils located on the top pillars, this would give ${\overline{q}}_D^j$ for j ∈ {1, … , 4}. Repeating the same procedure for the four bottom fiducial coils would give ${\overline{q}}_D^j$ for j ∈ {5, … , 8}. Thus, all the fiducial coils are mapped to DCS.

D. LEAST-SQUARE BASED REGISTRATION

Once all the fiducial coils are mapped to DCS, the least-squares problem given in (7) can be solved. Registering two corresponding sets of three DOF data is a well-studied problem. In [62], Eggert discusses and compares previously introduced four algorithms [63], [64], [65], [66] to solve this problem.

In this study, the singular value decomposition (SVD) based least-squares fitting algorithm proposed by Arun [65] is employed to find the optimum transformation. In short, first the centroids of both dataset are computed and then both datasets are centered around the origin. Optimal rotation R_BD in (8) is computed via applying SVD to the covariance matrix of the centered datasets. Plugging the optimal rotation and the centroids into the (6) gives the optimum translation t_BD. Sorkine [67] gives a concise summary of the algorithm.

V. VALIDATION EXPERIMENTS

A. EXPERIMENT SETUP

1). DISTORTION CORRECTION PHANTOM

The distortion cube phantom is 3D printed with PLA material. The dimensions of the cube are 171 × 171 × 171 mm³. Fig. 8 shows a detailed schematic of the distortion phantom, providing its measurements. Each square on a single grid has a dimensions of 15 × 15 mm².

FIGURE 8.

Schematic of the distortion correction phantom used for the proposed approach.

2). REGISTRATION STRUCTURE

The registration prototype (Fig. 9a) is embedded with 29 current-carrying coils (24 of them along outer, middle, and inner pillars and 5 of them on the LEGO® made tree-structure, located at the middle), each of which is a 20-turn coil for pillars (Fig. 9b) and a 16-turn coil (Fig. 9c) for the tree structure. The coils are made of heavy insulated 30-gauge AWG magnet wires (Adapt Industries, LLC, Salisbury, MD, USA). The prototype itself is 3D printed with ABS-M30™ (Stratasys, Ltd, 7665 Prairie, MN, USA) material.

FIGURE 9.

(a) Registration frame prototype used in the experiments. (b) Registration coil. (c) Validation coil.

The experiments are conducted in a 3T clinical scanner (Vida, Siemens Healthineers, Erlangen, Germany), as shown in Fig. 10. The distortion correction phantom and the registration prototype are mounted vertically inside an aquarium tank (254 × 254 × 267 mm³) and immersed in distilled water doped with a gadolinium-based contrast agent. This particular solution was used to load the radio-frequency coils of the scanner appropriately and have the relaxation properties be more realistic.

FIGURE 10.

Experiment setup inside a clinical MRI scanner. The Registration frame prototype is immersed in a phantom filled with distilled water doped with a gadolinium-based contrast agent. Phase array RF coils are placed on top of the prototype.

The aquarium tank is centered along the central axis of the MRI scanner and a phase array RF coil is placed on top of the prototype. For the registration prototype, the cables of the coils are connected to a transconductance amplifier controller which stays inside the MRI room outside the 5 Gauss line. The controller box sets the coil currents using a micro-controller which communicates with a PC located outside the MRI room through a USB serial link over fiber.

B. EXPERIMENTAL METHODS

The thin-plate spline based distortion correction algorithm presented in Section III utilizing a custom-built phantom and the differential multi-slice image registration algorithm presented in Section IV are experimentally validated as follows:

A CT scan of the distortion phantom is acquired to collect ground truth data for the control points. For the purposes of this study, it is assumed that these images are distortion free. The data acquisition is performed via a clinical CT scanner (Somatom, Siemens Healthineers, Erlangen, Germany) in the transverse orientation with the following parameters: number of slices = 319; slice thickness = 0.6 mm; field of view = 216 × 216 mm²; matrix size = 512 × 512.

The procedure for the MRI acquisition of the distortion phantom and the registration prototype is described below.

1. An initial scout image acquisition with a coarse resolution is performed to determine distortion phantom location inside the bore.

2. The phantom is scanned in coronal orientation. A gradient echo acquisition with the following parameters is performed: Echo time (TE) = 2.98 ms, slice thickness = 1 mm; field of view = 256 × 256 mm²; matrix size = 256 × 256; flip angle = 20 degrees; bandwidth = 260 Hz/pixel; repetition time (TR) = 5000 ms.

3. The distortion phantom is switched with the registration prototype without moving the aquarium tank and an initial scout image acquisition with a coarse resolution is performed to determine registration prototype location inside the bore.

4. The coils located on the top side of all the lattice pillars are scanned in coronal orientation. Mid pillar coils are used for registration. Inner pillar coils are reserved to use later for validation. A gradient echo acquisition with the following parameters is performed: Echo time (TE) = 4.70 ms, slice thickness = 1.5 mm; field of view = 300 × 300 mm²; matrix size = 192 × 192; flip angle = 20 degrees; bandwidth = 260 Hz/pixel. This gives the background model explained in Section IV. The acquisition is then repeated by activating the coils with a current of 75 mA.

5. Step 4 is repeated for the bottom coils.

6. Five coils, embedded in the tree structure, are used for validation in addition to mid and inner pillar coils. This structure replaces the actual robotic prototype to minimize the effect of the uncertainties in the kinematic model of the robot in validation. In order to investigate how using different imaging plane orientations affect the accuracy of target localization, multiple datasets are collected with different image plane orientations. For each dataset, the number of slices and image plane orientations are given in Table 2. For each set, acquisition is performed with same background/foreground sequence in Steps 2-3.

7. Depending on the number of slices and slice orientation, 26-34 receiver coils were used and repetition time (TR) was 528, 880, or 1060 ms depending on the number of slices.

TABLE 2.

The image plane orientations used in acquisition and the number of slices collected for each dataset.

Experiment	Imaging Plane	Number of Slices
Registration Mid Pillar	Coronal	Top Coils: 60 Bottom Coils: 60
Localization Inner Pillar	Coronal	Top Coils: 60 Bottom Coils: 60
Localization Tree	Coronal	60
Localization Tree	Sagittal	100
Localization Tree	Transverse	120
Localization Tree	Double Oblique	120

The double oblique orientation corresponds to T>C38.3>S10.0 meaning transverse (T) tilted towards coronal (C) by 38.3° then towards sagittal (S) by 10°. The fiducial coil registration algorithm is implemented in MATLAB® and the analysis is performed offline. The registration step takes approximately 1.75 seconds after the image acquisition step on an Intel® 3.40GHz quad-core CPU with 16GB RAM under Linux operating system.

VI. RESULTS

A. GEOMETRIC DISTORTION CORRECTION

The contrast solution that the phantom was immersed in may contain air bubbles, which could generate false control points due to the artifacts created by the air bubbles. This is a typical issue with the fluid filled phantoms. The automatic step of the control point detection was predominantly insensitive to such artifacts. Any remaining localization errors were manually corrected as described in Section III-B. The control point localization was achieved with an accuracy of one voxel (1 × 1 × 1 mm³), which was determined by superimposing the control points on the images (Fig. 3).

Initial distortion as well as the residual distortion after the correction were computed to evaluate the performance of the distortion correction algorithm. The distortion along each axis is calculated as the difference between the coordinates of each corresponding CT and MRI control point after they are registered to common coordinate system. For ${ℝ}^3$:

$$e_{i_x}={\tilde{p}}_{i_x}-p_{i_x},e_{i_y}={\tilde{p}}_{i_y}-p_{i_y}{e}_{i_z}={\tilde{p}}_{i_z}-p_{i_z},$$ (13)

where i = 1 … N. The overall distortion for each point is then given by $e=\sqrt{e_{i_x}^2+e_{i_y}^2+e_{i_z}^2}$. The residual distortion along each axis and overall residual distortion are calculated after the distortion correction is applied once the correction function is computed via (4):

$${\overline{e}}_{i_x}=p_{i_x}-f({\tilde{p}}_{i_x}),$$ (14a)

$${\overline{e}}_{i_y}=p_{i_y}-f({\tilde{p}}_{i_y}),$$ (14b)

$${\overline{e}}_{i_z}=p_{i_z}-f({\tilde{p}}_{i_z}),$$ (14c)

$$\overline{e}=\sqrt{{\overline{e}}_{i_x}^2+{\overline{e}}_{i_y}^2+{\overline{e}}_{i_z}^2}.$$ (14d)

In order to validate the proposed method, the detected control points on the ground truth CT and MRI grid data are grouped into two subsets. First, for each grid, every other detected control point is reserved to respectively calculate the unknown weights a_i’s and b_j’s of the radial basis functions and the polynomial in the underlying distortion field (2) via (5a) to (5d). Once the unknown weights are computed, every skipped point in each grid, which was not used in the calculation of the weights, is evaluated with the spline function (2) to verify the distortion compensation.

Table 3 shows the mean and maximum absolute distortion values of each axis and overall distortion for both before and after performing the distortion correction. Standard deviations are also provided. The mean, standard deviation, and maximum initial distortions were found to be 1.20 mm, 0.7 mm, and 4.14 mm. The mean, standard deviation, and maximum residual distortion results were calculated as 0.29 mm, 0.20 mm and 1.47 mm, which indicate the presented method provides good performance for correcting the underlying geometric distortion.

TABLE 3.

Mean and maximum absolute distortion values of each axis and the overall distortion before and after performing the correction.

		Initial Distortion			Residual Distortion
Axis		Mean [mm] (Std Dev [mm])	Max [mm]		Mean [mm] (Std Dev [mm])	Max [mm]
X		0.68 (0.66)	4.13		0.17 (0.16)	1.29
y		0.59 (0.44)	2.36		0.10 (0.09)	0.96
z		0.51 (0.49)	2.57		0.15 (0.15)	1.22
e		1.20 (0.72)	4.14		0.29 (0.20)	1.47

Fig. 11a and Fig. 11b respectively show the mean and maximum distortions of each axis and the overall distortion for each image plane before distortion correction. Image plane position is changing along the z-axis, which corresponds to the out-of-plane axis for the coronal orientation that the phantom was imaged. It can be observed that the distortion is increasing towards the edges of the volume of interest (VOI). Fig. 12 displays the sample distortion maps for the image planes located at z = 18 mm and z = −89 mm from the isocenter. For both image planes, distortion is higher towards the edges of the field-of-view (FOV) and the distortion map is smoother for the plane closer to the isocenter.

FIGURE 11.

The mean (a) and maximum (b) distortions of each axis and the overall distortion for each image plane before distortion correction. Image plane position is along z-axis.

FIGURE 12.

The distribution of overall distortion for the image planes located at; (a) z = 18 mm, (b) z = −89 mm.

During the robot to scanner registration stage, all of the detected control points on the CT and MRI grids are utilized for estimating the overall distortion field.

B. ROBOT TO SCANNER REGISTRATION

Target registration error (TRE) [68] is used to evaluate the system performance as it is the quantity of clinical interest. Fiducial registration error (FRE) values are also presented to verify the system is functioning properly [69]. FRE is computed by using the cost function of the optimization problem, (7), in the root mean square error formulation:

$$FRE=\sqrt{\frac{{\sum }_{j=1}^{N_C}{\Vert {\overline{q}}_B^j-{\widehat{g}}_{B{D}_{opt}}{\overline{q}}_D^j\Vert }^2}{N_C}},$$ (15)

where ${\widehat{g}}_{B{D}_{opt}}$ is the result of registration step (12). TRE is used when a new set of target points (${\overline{p}}_B^j$, ${\overline{p}}_D^j$ for j ∈ 1, … , M_C) are used for validation:

$$TRE=\sqrt{\frac{{\sum }_{j=1}^{M_C}{\Vert {\overline{p}}_B^j-{\widehat{g}}_{B{D}_{opt}}{\overline{p}}_D^j\Vert }^2}{M_C}}.$$ (16)

For the methods presented in this study, a baseline registration performance is provided by using the images whose distortions were corrected with the proprietary software of the scanner. Table 4 shows the results for the baseline registration errors. FRE is given in Row 1 for the registration result and TRE is given in Rows 2-6 for the validation results. The mean FRE value and TRE value are respectively computed as 1.91 mm and 2.53 mm. For each step, imaging out-of-plane errors averaged over all coils for that step are also reported. Image out-of-plane axis corresponds to z-axis for the coronal, x-axis for the sagittal, and y-axis in Cartesian coordinates for the transverse orientations. For coronal imaging orientation and a set of target points (${\overline{p}}_B^j$, ${\overline{p}}_D^j$ for j ∈ 1, … , M_C), the out-of-plane mean error (OPME) is given by:

$${\overline{p}}_B^j={[{\overline{p}}_{B_x}^j,{\overline{p}}_{B_y}^j,{\overline{p}}_{B_z}^j]}^{\text{T}},$$ (17a)

$${\widehat{p}}_B^j={\widehat{g}}_{B{D}_{opt}}{\overline{p}}_D^j,$$ (17b)

$${\widehat{p}}_B^j={[{\widehat{p}}_{B_x}^j,{\widehat{p}}_{B_y}^j,{\widehat{p}}_{B_z}^j]}^{\text{T}},$$ (17c)

$$OPME={\sum }_{j=1}^{M_C}\left|{\overline{p}}_{B_z}^j-{\widehat{p}}_{B_z}^j\right|/M_C.$$ (17d)

TABLE 4.

Baseline results. FRE of registration step (Row 1) and TRE of validation step (Rows 2-6). Out-of-plane mean error across all coils for each step and standard deviations are also shown.

Experiment	Imaging Plane	3-D Error [mm] (Std Dev [mm])	Out-of-Plane Mean Error [mm] (Std Dev [mm])
Registration Mid Pillar	Coronal	1.91 (1.07)	1.53 (0.31)
Localization Inner Pillar	Coronal	2.17 (0.94)	1.60 (0.88)
Localization Tree	Coronal	2.39 (0.36)	1.57 (0.26)
Localization Tree	Sagittal	2.70 (1.08)	1.65 (1.21)
Localization Tree	Transverse	2.61 (0.59)	1.82 (1.16)
Localization Tree	Double Oblique	2.82 (0.80)	-

Out-of-plane errors for other imaging orientations and registration coils are computed similarly. Out-of-plane errors are within 0.5 mm of the slice thickness (1.5 mm). The validation experiments performed with the same imaging plane orientation as in the registration step (coronal), has higher accuracy.

Table 5 shows the results before and after applying the distortion correction. FRE is given in Row 1 registration results and TRE is given in Rows 2-6 for validation results. Before the distortion correction, the registration and the mean validation errors are respectively computed as 3.09 mm and 2.78 mm. After the distortion correction, the registration and validation performances are improved respectively to 2.05 mm and 2.64 mm. The validation experiments performed with the same imaging plane orientation as in the registration step (coronal), has higher accuracy. The out-of-plane errors are comparable; coronal = 1.62 mm, sagittal = 1.71 mm, transverse = 1.87 mm.

TABLE 5.

Results before and after applying distortion correction. FRE of registration step (Row 1) and TRE of validation step (Rows 2-6). Out-of-plane mean error across all coils for each step and standard deviations are also shown.

Distortion Correction			Before			After
Experiment	Imaging Plane		3-D Error [mm] (Std Dev [mm])	Out-of-Plane Mean Error [mm] (Std Dev [mm])		3-D Error [mm] (Std Dev [mm])	Out-of-Plane Mean Error [mm] (Std Dev [mm])
Registration Mid Pillar	Coronal		3.09 (1.45)	1.98 (0.60)		2.05 (1.21)	1.61 (0.41)
Localization Inner Pillar	Coronal		2.66 (1.03)	1.80 (0.95)		2.25 (1.08)	1.68 (0.87)
Localization Tree	Coronal		2.57 (0.71)	1.69 (0.64)		2.49 (0.49)	1.62 (0.41)
Localization Tree	Sagittal		2.83 (1.20)	1.76 (1.15)		2.77 (1.03)	1.71 (1.18)
Localization Tree	Transverse		2.80 (0.59)	1.93 (1.10)		2.73 (0.64)	1.87 (1.11)
Localization Tree	Double Oblique		3.06 (0.91)	-		2.95 (0.78)	-

It can be noted that the distortion has stronger effect on the registration results. The pillar coils used for registration are located towards edges of the volume of interest. Fig. 13 shows the planar view of distorted control points overlaid on the detected registration and validation coils for the coronal orientation. The distortion mapping results given in Figs. 11 and 12 showed that the distortion is more prominent towards the edges of volume of interest as well as the field of interest for a given image plane. As the magnitude of gradient nonlinearity increases with distance from isocenter, it is expected the distortion to be more pronounced towards the edges of the FOV [70]. This explains the higher increase in the FRE from the baseline performance compared to the TRE. As the validation coils located around the center of VOI (Fig. 9a), the distortion has a less noticeable effect on the TRE and thus correction has a more subtle improvement.

FIGURE 13.

Shows the planar view of distorted control points overlaid on the detected registration and validation coils for the coronal orientation.

This study utilizes multi-slice images to take full benefit of the 3D information in the registration. Multi-slice validation is compared to the single slice method to analyze effectiveness of this approach. For the single slice method, instead of performing a weighted average, the fiducial artifact with the largest area across the multi-slice image is selected for coil localization. Table 6 shows the experiment results for single slice method and Table 7 presents the results before and after applying the distortion correction. The multi-slice method improve the registration and validation compared to single slice method respectively by 0.34 mm and 0.53 mm for the baseline performance, and 0.41 mm and 0.64 mm for the experiments after applying the distortion correction.

TABLE 6.

Baseline results for single slice method. FRE of registration step (Row 1) and TRE of validation step (Rows 2-6). Standard deviations are also reported.

Experiment	Imaging Plane	3-D Error [mm] (Std Dev [mm])
Registration Mid Pillar	Coronal	2.25 (1.50)
Localization Inner Pillar	Coronal	2.58 (1.14)
Localization Tree	Coronal	3.03 (0.59)
Localization Tree	Sagittal	3.52 (1.33)
Localization Tree	Transverse	3.37 (0.86)
Localization Tree	Double Oblique	3.29 (1.02)

TABLE 7.

Single-slice results before and after applying distortion correction. FRE of registration step (Row 1) and TRE of validation step (Rows 2-6). Standard deviations are also reported.

Distortion Correction			Before		After
Experiment	Imaging Plane		3-D Error [mm] (Std Dev [mm])		3-D Error [mm] (Std Dev [mm])
Registration Mid Pillar	Coronal		3.51 (1.60)		2.46 (1.51)
Registration Mid Pillar	Coronal		3.05 (1.12)		2.71 (1.18)
Localization Inner Pillar	Coronal		3.28 (0.74)		3.18 (0.67)
Localization Tree	Sagittal		3.72 (1.49)		3.64 (1.39)
Localization Tree	Transverse		3.61 (0.90)		3.50 (0.83)
Localization Tree	Double Oblique		3.47 (0.94)		3.36 (0.97)

VII. DISCUSSION AND CONCLUSION

This study presents a differential multi-slice image based robot to MRI scanner registration approach utilizing active fiducial coils, while taking into account the scanner related geometric distortion. A grid-based, custom-built 3D phantom was utilized to map the distortion. Grid vertices were used as control points and identified in MRI and ground truth CT images via morphological operations. The mean and maximum overall distortions were 1.20 mm and 4.14 mm, whereas the mean and maximum residual distortions were 0.29 mm and 1.47 mm. Thin-plate splines were used to model the underlying distortion field. It was shown that over a volume of 171 × 171 × 171 mm³, the distortions could be successfully corrected with an average accuracy of less than 1/3 voxel size (1 × 1 × 1 mm³) and maximum accuracy of less than 1.5 voxel size. The distortions are more prominent around the edges of the studied volume and the field-of-view of a given image plane.

The baseline FRE and TRE were respectively computed as 1.91 mm and 2.53 mm by using the images whose distortions were corrected with the proprietary software of the scanner. The FRE and TRE were respectively computed as 2.05 mm and 2.63 mm after the distortion correction, which are an improvement of respectively 1.08 mm and 0.14 mm compared to results without distortion correction. To the best of authors’ knowledge, this is the first study that utilizes active fiducial coils in the context of robot-to-MRI scanner registration, where a background subtraction method is employed for coil detection.

Multi-slice images in different orientations were acquired to explore the effects of using different imaging plane orientations on the accuracy of target localization. The out-of-plane localization was within 0.5 mm of the slice thickness (1.5 mm). Multi-slice approach was also compared to single-slice approach and shown to have better registration performance.

In the registration analysis, to eliminate any other errors; such as robot kinematic modeling inaccuracies, and focus only on the registration error, a set of coils embedded in a LEGO® structure, with known baseline coordinates, were used as validation targets. When the proposed registration approach is incorporated in an actual robotic system, inaccuracies both in the robot kinematic model and robot tracking algorithm would affect the robotic guidance performance. It takes approximately 1.75 seconds to perform registration after the image acquisition, which would provide good performance for the preoperative registration step during an interventional procedure. This also makes it possible to swiftly re-register the robot to the scanner during the procedure if needed. In the future, we are planning to implement the proposed methods in C++ to be used with the robotic catheter system [35], which would provide further performance improvements in temporal resolution.

In this study, outer pillar registration coils were not used as they remained outside the volume where the site-specific distortion was investigated (Fig. 13). It can be reasonably expected that the distortion would be more pronounced for the larger volume which would contain the outer pillar coils based on the results presented in Section VI-A. The distortion over a larger volume will be studied with a new, wider phantom, which would allow to better sample the MRI bore, investigate the distortion effects towards the edges, and map the distortion more completely.

During the interventional procedure, the magnetic susceptibility within the MRI scanner might change as a result of any potential anatomical motions, which in turn might affect the distortion measurements performed preoperatively using the custom-built phantom and consequently the registration performance. The effects of the patient on the distortion measurements and the accuracy of the registration will be investigated in future tests.

For the presented work here, the distortion field was determined with a single orientation distortion dataset. Investigating the distortion that relies on the analysis of multiple datasets in arbitrary orientations [12] might be a future avenue for having a more detailed distortion map. A more accurate distortion characterization via determination of the 3D distortion field by using an iterative process as in [14] is another potential direction moving forward. The registration validation experiments performed with different orientations showed the resulting distortion map was not biased towards a direction within the experiment space.

The future tests would include performing repeated trials to verify the reproducibility of the distortion and the registration analysis. Another insightful future work would be implementing the fully automatic extraction of the control points and investigating the effect of no manual control point correction on the performance of distortion correction algorithm. The registration analysis based on image acquisitions with different slice thickness would also be useful. It could be assumed the FRE and thus TRE would increase with the increased slice thickness. Though, relative TRE performance with respect to the slice thickness size would be more informative. Thin slices need a long duration to acquire to obtain a good signal to noise ratio. Therefore, it is desirable to achieve accurate registration also with thick slices [29]. This analysis together with the impact of image quality on the accuracy of registration will be investigated in the future. As the registration step would be performed once at the beginning of a clinical procedure. A thinner slice could be selected to achieve better performance, while accepting longer-acquisition times as a trade-off.

The direct comparison of the presented active fiducial coil based registration method with the passive fiducial marker and micro tracking coil based methods remains future work. The microcoils need dedicated scanner channels and require tailored programmed sequences for tracking, which are atypical in commercial MRI scanners. Passive fiducials, due to their homogeneous nature, also require careful programming of pulse sequences to ensure marker contrast and detection. When these disadvantages considered, the active fiduical coil based method is expected to be more robust and practical for widespread clinical deployment. Therefore, a direct comparison of different registration methods would be informative, though it is outside the scope of the presented study.

The proposed registration approach is intended to be used with the robotic catheter system proposed in [35], where the catheter prototype is mounted vertically to an aquarium tank and immersed in distilled water doped with a gadolinium-based contrast agent. For this purpose, the same aquarium tank setup is utilized in this study to preserve the experimental setup of the robotic catheter system. Despite this, the active fiducial coil based registration approach is not inherent to this particular setup and could be employed without the aquarium tank. For instance, the coils could be installed directly along the instrument, such as to the base of the robotic catheter, where each coil could be individually immersed into separate pouches that are filled with the water solution.

The registration performance of the differential robot-to-MRI scanner approach after distortion correction is within 96% accuracy of the baseline. This suggests the proposed distortion correction method could be a substitute to the propriety software of the scanner for the real-time interventional robotic procedures. The proposed registration approach when used in conjunction with the authors’ previous work [71] paves the way for achieving clinically-desired instrument to target accuracy of under 3 mm during image guidance.

Biographies

E. ERDEM TUNA (Member, IEEE) received the B.S. and M.S. degrees in electrical and electronics engineering from Bilkent University, Ankara, Turkey, and the Ph.D. degree in electrical engineering from Case Western Reserve University, in 2021. He was a Translational Research Fellow of Case Western Reserve University, from 2020 to 2021, and is currently a Postdoctoral Researcher. His current research interests include robotics and control theory with emphasis on medical robotics and haptics, robotics and machine learning to biomedical, and biologically inspired engineered systems.

NATE LOMBARD POIROT received the B.S.E. degree in mechanical and aerospace engineering from Case Western Reserve University, in 2014, where he is currently pursuing the Ph.D. degree in electrical engineering. His research work includes prototyping and validating/optimizing catheter prototypes via fluid and mechanical analysis. As an Ames Space Academy Alum (’15), his research interests include the growing private space industry and hopes to apply his degree to this field.

DOMINIQUE FRANSON received the B.S. degree in bioengineering from the University of Maryland, College Park, in 2010, and the Ph.D. degree in biomedical engineering from Case Western Reserve University, in 2022. She was a Catheter Engineer at the National Institutes of Health, Bethesda, MD, USA, from 2010 to 2014. She is currently a Research Scientist in MR at Siemens Healthineers. Her research interests include real-time and interventional magnetic resonance imaging.

JUANA BARRERA BAYONA received the B.S. degree in biomedical engineering from Case Western Reserve University, in 2019. She is currently pursuing the M.D. degree with the University of California at San Francisco, San Francisco. She was a Research Analyst at Galvani Bioelectronics, from 2019 to 2021.

SHERRY HUANG received the B.S. degree from Case Western Reserve University, in 2017, with a double major in electrical engineering and biomedical engineering and the Ph.D. degree in biomedical engineering from Case Western Reserve University, in 2022. She was a NSF Graduate Research Fellow. She is currently a Clinical Research Scientist at GE Healthcare.

NICOLE SEIBERLICH received the B.S. degree in chemistry from Yale University, in 2001, and the Ph.D. degree in physics from the University of Wuerzburg, in 2008. She was an Associate Professor in biomedical engineering at Case Western Reserve University, until 2019, with secondary appointments in radiology and electrical engineering and computer science. She is currently an Associate Professor in radiology at Michigan Medicine, University of Michigan. Her research interests include fast magnetic resonance imaging, signal processing and image reconstruction, quantitative MRI, and cardiac MRI. She is currently an Associate Editor of IEEE Transactions on Medical Imaging.

MARK A. GRISWOLD received the bachelor’s degree in electrical engineering from the University of Illinois Urbana–Champaign and the Ph.D. degree in biophysics from the University of Wuerzburg, in 1999.

He is currently a Professor in radiology, biomedical engineering (BME), electrical engineering and computer science (EECS), and physics at Case Western Reserve University. Following this, he moved to the Harvard Medical School, in 1993, where he led the MRI Detector Laboratory, Beth Israel Hospital. He joined the Case Western Reserve Faculty, in 2005. He is also a Recognized Pioneer in the development of several innovative approaches for magnetic resonance imaging (MRI). In particular, his parallel MRI methods have been translated into FDA-approved products by several MRI system manufacturers. These methods have led to almost an order of magnitude increase in MRI acquisition speed over the last decade. More recently his laboratory has focused on translating the increase in imaging speed into improved quantitative imaging, especially imaging of tissue/organ function. This has culminated in the development of magnetic resonance fingerprinting (MRF) that promises another order of magnitude in the speed of quantitative MRI.

M. CENK CAVUSOGLU (Senior Member, IEEE) received the B.S. degree in electrical and electronic engineering from Middle East Technical University, Ankara, Turkey, in 1995, and the M.S. and Ph.D. degrees in electrical engineering and computer science engineering from the University of California at Berkeley, Berkeley, in 1997 and 2000, respectively. He is currently a Professor in electrical engineering and computer science, biomedical engineering, and mechanical and aerospace engineering with Case Western Reserve University, Cleveland, OH, USA. He was a Visiting Researcher with the INRIA Rhones-Alpes Research Center, Grenoble, France, in 1998, a Postdoctoral Researcher and a Lecturer with the University of California at Berkeley, from 2000 to 2002, and a Visiting Associate Professor with Bilkent University, Ankara, from 2009 to 2010. His current research interests include robotics, systems and control theory, and humanmachine interfaces, with emphasis on medical robotics, haptics, virtual environments, surgical simulation, biosystem modeling and simulation, robotics, and control engineering to biomedical and biologically inspired engineered systems.

He has previously served as an Associate Editor of the IEEE Transactions on Robotics and as a Technical Editor of the IEEE/ASME Transactions on Mechatronics.