Beating-heart registration for organ-mounted robots

Abstract

Background:

Organ-mounted robots address the problem of beating-heart surgery by adhering to the heart, passively providing a platform that approaches zero relative motion. Because of the quasi-periodic deformation of the heart due to heartbeat and respiration, registration must address not only spatial registration but also temporal registration.

Methods:

Motion data were collected in the porcine model in vivo (N = 6). Fourier series models of heart motion were developed. By comparing registrations generated using an iterative closest-point approach at different phases of respiration, the phase corresponding to minimum registration distance is identified.

Results:

The spatiotemporal registration technique presented here reduces registration error by an average of 4.2 mm over the 6 trials, in comparison with a more simplistic static registration that merely averages out the physiological motion.

Conclusions:

An empirical metric for spatiotemporal registration of organ-mounted robots is defined and demonstrated using data from animal models in vivo.

1 |. INTRODUCTION

It is desirable to avoid cardiopulmonary bypass during minimally invasive surgery due to its concomitant morbidity,¹ but doing so greatly increases the difficulty of accurate manipulation. Cardiac stabilizers have been developed to address this, but they do not hold the heart surface completely motionless,² and the force required to hold a portion of the surface nearly motionless brings some morbidity as well.^3–5 In recent years, as an alternative to alleviate these problems, our group has developed an approach that may be referred to by the term organ-mounted robots. These are highly flexible systems, designed for subxiphoid access, that adhere to the exterior of the heart (or other moving organ) and move freely with it, thereby providing passive compensation of physiological motion in order to serve as a stable platform for intervention. Examples include Lamprey, a repositionable passive device,⁶ HeartLander, a crawling robot that moves in inchworm fashion,⁷ and Cerberus, a deployable flexible triangular manipulator with a moving injector head.⁸ Mapping, myocardial injection, epicardial placement of pacing leads and tissue ablation have been demonstrated in animal models in vivo with these devices.^6,9

This robotic approach offers benefits such as passive compensation of heartbeat and respiratory motion, but it also creates certain difficulties. The subxiphoid approach used to access the heart (in contrast to an intercostal approach) has the benefit of avoiding the pleural space, but it restricts vision, requiring model-guided techniques for navigation. In this paradigm, electromagnetic trackers are registered to models constructed from preoperative medical images to provide a virtual view of the operating site.¹⁰ In most virtual image-guided systems, registration consists of only the rigid transform that aligns the real and virtual space. This spatial registration provides full knowledge of the system, because the system can be described by a 3-link closed kinematic chain. If we are given a measurement of the robot in the world frame, z, and we know the ground-truth registration, T*,which aligns the maps, S, with the actual surface, H, then the location of the robot in the map frame is simply found by transforming the measurement by the rigid registration transformation, T*z. The underlying assumption is that the dimensionality of the map matches that of the world, i.e. the static map represents a static organ. However, this assumption does not hold on the beating heart. Organ-mounted robots thus require new methods for registration and localization on the beating heart. A previous paper presented methods for modelling the periodic motion of robots on the epicardial surface.⁶ This paper presents methods for registration for such robots, where the intraoperative information inherently includes physiological motion.

1.1 |. Formulation of the problem

The surface of the heart, as previously described, is a dynamic environment that deforms periodically due to both heartbeat and respiration. If we assume that we have a function that describes the periodic motion of the surface of the heart,

$$\mathcal{H}=(u,v,{\varphi }_c,{\varphi }_R)$$

then preoperative medical imaging provides a snapshot of the deforming heart at an instant,

$$\mathcal{S}(u,v)=T*\mathcal{H}(u,v,{\varphi }_c^{*},{\varphi }_R^{*})$$

where u and v are coordinates describing a location on the surface, ∅^*_c and ∅^*_R are the ground-truth cardiac and respiration phases at the instant the images were taken, and T* is once again the ground-truth registration that aligns the map and image frames.

The second assumption is that if an organ-mounted robot remains stationary on the heart, it follows the path of the underlying point on the surface of the heart,

$${\mathcal{P}}_i({\varphi }_c,{\varphi }_R)=T*\mathcal{H}(u,v,{\varphi }_c^{*},{\varphi }_R^{*})$$

where P_i is the function describing the path in 3D space of the ith point on the surface of the heart. This implies that the robots do not deform the surface of the heart, which, for systems such as HeartLander and Lamprey, we believe is a valid assumption, due to their small size and high flexibility.⁶

Combining ² and ³ provides the basis for the approach of registration for organ-mounted robots,

$$\mathcal{S}(u_i,v_i)=T*{\mathcal{P}}_i({\varphi }_c,{\varphi }_R);$$

that is, the points on the surface of the map are given by the path of the robot at the instant the images used to create the map were captured. This view of registration for organ-mounted robots is equivalent to the case of a static world and static map if we collect measurements only at the ground-truth physiological phases. If the surface consisted of only a single periodic component this may be a reasonable approach; however, with multiple periodic components one could wait arbitrarily long before observing the correct phase combination.

Because of these periodic components of the deformation of the heart and the mismatch in dimensionality of the real world and the map, registration for organ-mounted robots requires knowledge of the physiological phases when the medical images were captured, which we refer to as temporal registration, as well as the rigid spatial registration described above. The next section describes our approach, which uses a collection of learned periodic motion models to do these two aspects of registration for organ-mounted robots in conjunction with one another.

1.2 |. Approach

In order to solve the registration problem for organ-mounted robots, we first must be able to estimate or predict where the robot will be at the ground-truth physiological phases. Given a collection of measurements, $\left\{z^{{\varphi }_1},z^{{\varphi }_2},\dots ,z^{{\varphi }_{\mathrm{k}}}\right\}$, where $z^{{\varphi }_i}$ is a 6-degree-of-freedom (6- DOF) pose of the robot at a particular set of phases, ∅_i, we first learn frequency-based models that accurately describe the periodic motion of points on the surface of the heart, $\mathcal{P}(\cdot )$. Using these models, we can then predict where the robot will be at any phase, or, more importantly, at the ground-truth map phases,∅* .

Next, using a collection of these periodic models, $\left\{{\mathcal{P}}_1(\cdot ),{\mathcal{P}}_2(\cdot ),\dots ,{\mathcal{P}}_n(\cdot )\right\}$, the ground-truth phases and the rigid transformation that minimizes the registration error,

$${{\displaystyle \sum _i\left(\mathcal{S}(u_i)-T{\mathcal{P}}_i(\varphi )\right)}}^2,$$

are estimated simultaneously. This estimation of parameters that aligns the real world and map, in both space and phase, may be termed spatiotemporal registration.

1.3 |. Related work

Methods for registering and localizing in and around the heart are highly dependent upon the instrument or robot being used. Tully et al. use an electromagnetic tracker located at the distal end of a highly articulated snake-like robot to estimate the shape of the robot using an extended Kalman filter framework.¹¹ If the pose of the robot is found to violate geometric constraints, namely intersecting the preoperative surface models, the registration and world frame localization parameters are updated using inequality- or equality-constrained Kalman filtering to project the system state into the feasible space.^12,13 In the inequality-constrained case the model is assumed to be rigid, thereby restricting the robot to lie entirely outside the surface, while the equality-constrained case incorporates a stiffness model and force measurement to more accurately model the surface as a deformable body.

Therapies that target the endocardium using intracardiac echocardiographic (ICE) catheters use ultrasound images in conjunction with an electromagnetic tracker. One method generates a point cloud by extracting heart surface points from 2-dimensional ultrasound images at the catheter tip by rotating the catheter about its longitudinal axis. After sufficient points have been collected, the cloud is registered to the preoperative model of the left atrium using an iterative closest point (ICP) method to determine the pose of the catheter in the model frame directly.¹⁴ Electromagnetic position measurements can then be used for registration.

Another method uses a particle filter to recursively estimate the map frame pose of the ICE catheter in the left atrium.¹⁵ In this work the probability of a catheter tip pose is calculated by comparing virtual ultrasound images constructed from a preoperative model with the actual ultrasound images. The predicted catheter pose is then determined as the weighted sum of the particles, and the registration parameters are then calculated using this pose estimate and measurements from an electromagnetic position tracker. Again, the use of ultrasound allows direct estimation of the map frame pose.

2 |. MATERIALS AND METHODS

2.1 |. Formulation of the problem

The problem addressed herein is the registration problem of aligning two point clouds, but with the added complication of one of the sets involving periodic variation, as shown diagrammatically in Figure 1. Registration is defined as

$$\mathcal{S}(u_i)=R^{*}{p}_i^{{\varphi }^{*}}+t,$$

FIGURE 1.

Registration of the periodic models of motion and the surface heart model requires simultaneously estimating (A) the phases corresponding to the map, ${\varphi }^{*}=\left({\varphi }_C^{*},{\varphi }_R^{*}\right)$, as well as (B) the rigid registration that spatially registers the data, T*

where

$$p_i^{{\varphi }^{*}}={\mathcal{P}}_i\left({\varphi }_C^{*},{\varphi }_R^{*}\right),$$

S is the surface mesh of the heart, u_i are the map coordinates of the point on the surface, ${\mathcal{P}}_i$ is the function defining the trajectory of the ith point, ${\varphi }_C^{*}$ and ${\varphi }_R^{*}$ are the ground-truth cardiac and respiration phases of our map, and R* and t* are the rigid rotation and translation that align S(u_i) and $p_i^{{\varphi }^{*}}$.

This formulation assumes that the spatial registration is a rigid transformation, meaning that the robot does not deform the surface of the heart and the set of poses of the robot at the image phases, $\left\{p_1^{{\varphi }^{*}},p_2^{{\varphi }^{*}},\dots \right\}$, is the same shape as the map. We believe this assumption is justified because of the flexibility and small form factor of organ-mounted robots.

If the ground-truth image phases are known, the spatial registration parameters can be found using ICP,¹⁶ which minimizes the distance function

$$T^{*}=\underset{T}{\arg \min }{\displaystyle \sum }_i{\left(\mathcal{S}(u_i)=T{p}_i^{{\varphi }^{*}}\right)}^2,$$

where we have combined rotation and translation into a rigid transformation, T*, and assumed homogenous coordinates for p.

If the imaging modality used to construct the anatomical models used a gating technique, it is reasonable to assume that the cardiac phase, ${\varphi }_C^{*}$, of the image set is known. It is less likely, however, that the respiration phase of the image set is known. Image sets are generally collected during a breath hold¹⁷ without measurement of phase. Identification of the respiration phase corresponding to the image set is the temporal component of the spatiotemporal registration.

The initial instinct to solve for temporal registration may be to minimize the previous metric over the physiological phases,

$${\varphi }_C^{*},{\varphi }_R^{*}=\underset{{\varphi }_C,{\varphi }_R}{\arg \min }\left(\underset{T}{\min }{\displaystyle \sum }_i{\left(\mathcal{S}(u_i)=T{p}_i^{\varphi }\right)}^2\right).$$

If the shape of the heart is unique at the image phases, then, in theory, optimizing over the phases solves for temporal registration. Based on data from live-animal experiments, however, this premise does not hold. Plots showing RMS error for registration experiments conducted on data from 6 live-animal experiments are shown in Figure 2. Average RMS error for these trials, in which we iterate over one phase while holding the other constant at the known image phase, shows that the temporal registration method given by ⁹ does not result in reliable global minima.

FIGURE 2.

Registration RMS error across (A) cardiac phase and (B) respiration phase for all 6 trials. Average RMS error across all trials is shown by the bold dashed line

The remainder of this paper presents an empirical method for registering in phase because of the lack of a theoretical optimization metric that can be defined a priori. We first describe the data acquisition, then present an empirical optimization metric based on the experimental data.

2.2 |. Data acquisition in porcine model in vivo

Data from animal experiments in vivo (N = 6) conducted under a protocol approved by the Institutional Animal Care and Use Committee (IACUC) of the University of Pittsburgh were collected and analyzed retrospectively as described below.

2.2.1 |. Fiducial placement

External skin markers were placed on the chest for use as initial estimates of spatial registration. The markers used were Weck Visistat 35 W 6.5 mm × 4.7 mm staples (Ref. #528235, Teleflex Medical, Research Triangle Park, NC). A 3 × 3 grid of markers was used, with a spacing of approximately 4 cm, with the central marker placed at the midpoint of the line from the xiphoid cartilage and the manubrium of the sternum, as shown in Figure 3(A).

FIGURE 3.

A, Initial patient registration is achieved using a matrix of 9 fiducial markers placed on the chest wall. A reference probe placed against the chest wall is used as the base frame to account for relative motion between the patient and measurement frame. B, The Lamprey organ-mounted passive robot consists of a single foot that is manually manipulated by an operator using a flexible catheter-like device

2.2.2 |. Cardiac CT imaging

ECG-gated cardiac CT image sets were collected prior to each surgery using a LightSpeed VCT imaging system (GE Medical Systems). Image sets were collected during a breath hold after inspiration (i.e. a halt of the forced respiration system), which corresponds to a ground-truth respiration phase ∅_R* = 0.3.

The desired cardiac phase of the image sets was ∅_C = 0.7, corresponding to the most common motion-free imaging window during diastole.¹⁷ The cardiac phase for each slice in the image sets, as shown in Figure 4, was estimated using image metadata, which included the time each image was collected as well as the time of every detected QRS complex in the ECG. Cardiac phase for each slice was calculated as the proportion of time between successive QRS detections. While 3 of the image sets do have jumps in cardiac phase, the portion of the cardiac cycle ranging from 0.6–0.8 is relatively motion-free.¹⁸ Cardiac phase for the entire image volume, ∅_C*, was estimated as the average phase of the image slices, and ranged from 0.61 to 0.67. Image sets had an axial slice thickness of 0.63 mm, with in-plane pixel spacing of 0.49 mm.

FIGURE 4.

Cardiac phase for each image slice from each gated CT scan from each of the 6 trials. Cardiac phase can be found using the timestamp recorded as each image was acquired, in conjunction with the detected QRS complexes in the image metadata

2.2.3 |. Construction of the epicardial model

CT volumes were manually segmented using OsiriX® software (Fondation OsiriX, Geneva, Switzerland) to produce triangle-mesh surface models of the heart, fiducial markers and various other anatomical structures for purposes of visual feedback, including the ribs, coronary arteries and endocardial surfaces. These surface models were then further processed using MeshLab¹⁶ to smooth, close holes and decimate the surface models. Models were then transformed into a heart-centric reference frame. This frame is defined by first recursively fitting ellipses to the vertices of the heart model in 10 mm slices in the z-direction, then aligning the new z-axis with the centroid of these ellipses. The x-axis is then defined as the weighted average, by eccentricity, of the minor axes of the ellipses.

Examples of the segmented models, ellipses fitted to the heart model and the heart-centric reference frame are shown in Figure 5. This heart-centric frame is analogous to standard medical imaging views of the heart where slices parallel to the X-Y, X-Z and Y-Z planes give views along the short axis, vertical long axis and horizontal long axis respectively.

FIGURE 5.

Approximately coronal (A) and approximately sagittal (B) views, in image coordinates, of the segmented anatomical structures. Ellipses fit to the heart surface model define the heart-centric reference frame, with the Z-axis defined by their centres, and Y- and X- axes defined by their major and minor axes respectively

Locations of the fiducials in the map frame were also collected to provide initial patient registration.

2.2.4 |. Intraoperative data collection and processing

Prior to the procedure, a reference probe was placed on the chest wall (Figure 3(A)) and used as the base frame for all measurements to ensure that motion of the patient or measurement frame does not corrupt data.

Access to the intrapericardial space was achieved through a subxiphoid skin incision and an incision in the pericardium near the apex of the heart. The initial registration was performed with the chest fiducials, using singular-value decomposition.¹⁹ Lamprey (Figure 3(B)) was then introduced and manually guided over the surface of the heart. At approximately 100 points spread over the surface of the heart, the robot was attached using vacuum pressure, and data were logged for 30 s at each point. Measurements, sampled at 100 Hz, include the 6-DOF pose of the robot and the reference probe, respiration flow rate and ECG. The sensors used were a microBird magnetic tracker, 2 MEMS airflow sensors (Omron D6F-50A-000) placed in line with the inspiration and expiration lines of the forced respirator, and ³ 0–1 V analogue output from a LIFEPAK 12 defibrillator/monitor (Medtronic, Minneapolis, MN) using a Physio-Control LIFEPAK 12/15 Analog ECG Output Cable (Part #: 3010484–00).

Measurements were processed to obtain the position and orientation in quaternion form of the robot in the frame of the reference probe, $t_H^P$ and $q_H^P$. Cardiac phase, ∅_C was extracted from the raw ECG signal by first using a Pan-Tompkins QRS detection algorithm²⁰ to detect the QRS complexes and then linearly interpolating between them. Respiration phase, ∅_R, is similarly extracted from flow rate measurements where the start of expiration is detected.

Information on each of the datasets is shown in Table 1. Average heart rates for the 6 trials ranged from 1.03 to 1.59 Hz (62 to 95 bpm). Forced respiration was used in all cases, with the first two trials using respiration frequencies of 0.2 Hz (12 bpm) and the remaining four 0.17 Hz (10 bpm). The rotations of the heart-centric frames with respect to the CT image coordinates are given as X-Y-Z Euler angles.

Table 1.

Data recorded from porcine trials

Trial	Points	Heart Ratea	Resp. Ratea	Image Phase	$R_M^I$ b
		(s−1)	(s−1)	Φ*	𝛼 (deg)	β (deg)	γ (deg)
1	101	1.08	0.20	0.66	−22.9	2.2	−43.8
2	108	1.36	0.20	0.61	−14.6	5.1	−63.5
3	98	1.59	0.17	0.64	−19.5	11.2	−59.0
4	44	1.42	0.17	0.66	−26.0	12.7	−86.0
5	84	1.37	0.17	0.64	−27.6	14.0	−55.6
6	95	1.03	0.17	0.67	−25.7	16.1	−78.4

^aCalculated as average over all points.

^bRotation of map frame relative to image frame, given as X-Y-Z intrinsic Euler angles R(α,β,γ) = R_x(α)R_y(β)R_z(γ)).

2.2.5 |. Physiological motion model

The 30 s time traces of robot pose were fitted using amplitude-modulated (AM) Fourier series models presented by Wood et al.⁶ These models capture the arbitrary cyclical motion in 6 degrees of freedom of each point on the surface of the heart. Collective motions of the heart, such as expansion and contraction or gross translation and rotation are not explicitly included in the model; however, these whole-organ motions are captured when considering all of the point models collectively. The models provide a least-squares fit of the quasi-periodic motion that the points undergo due to heartbeat and respiration.

$$t_H^P({\varphi }_C,{\varphi }_R)={\mathcal{M}}_q^R\left(q_{\overline{H}}^P\right)W_{\overline{t}}^T\varphi +t_{\overline{H}}^P$$

$$q_H^P({\varphi }_C,{\varphi }_R)=q_{\overline{H}}^P\otimes {\mathcal{M}}_{\omega }^{\mathrm{q}}\left(W_{\omega }^T\varphi \right)$$

where $t_{\overline{H}}^{\mathrm{P}}$ and $q_{\overline{H}}^{\mathrm{P}}$ are the mean translation and rotation, ${\mathcal{M}}_q^R$ is the mapping from quaternion to rotation matrix and ${\mathcal{M}}_{\omega }^{\mathrm{q}}$ is the mapping from exponential coordinates of rotation. Matrices $W_{\overline{t}}$ and W_ω are Fourier series parameter matrices for translation and rotation, respectively, and ∅ is the vector of sines and cosines of the cardiac phases given by

$$\mathrm{W}=\left[\begin{array}{c}{\mathrm{t}}_0^{\mathrm{T}}\\ {\left[\begin{array}{c}{\mathrm{a}}_i^T\\ {\mathrm{b}}_i^T\end{array}\right]}_{\mathrm{i}=1:{\mathrm{H}}_R}\\ {\left[\begin{array}{c}{\mathrm{c}}_i^T\\ {\mathrm{d}}_i^T\end{array}\right]}_{\mathrm{i}=1:{\mathrm{H}}_{\mathrm{c}}}\\ {\left[\begin{array}{c}{\mathrm{e}}_{i\mathrm{j}}^T\\ \begin{array}{l}{\mathrm{f}}_{i\mathrm{j}}^T\\ {\mathrm{g}}_{i\mathrm{j}}^T\\ {\mathrm{h}}_{i\mathrm{j}}^T\end{array}\end{array}\right]}_{\begin{array}{l}\mathrm{i}=1:{\mathrm{H}}_{\mathrm{c}}\\ \mathrm{j}=1:{\mathrm{H}}_{\mathrm{R}}\end{array}}\end{array}\right],\varphi =\left[\begin{array}{c}1\\ {\left[\begin{array}{l}\sin (\mathrm{i}{\varphi }_R)\\ \cos (\mathrm{i}{\varphi }_R)\end{array}\right]}_{i=1:H_R}\\ {\left[\begin{array}{l}\sin (\mathrm{i}{\varphi }_C)\\ \cos (\mathrm{i}{\varphi }_C)\end{array}\right]}_{i=1:H_c}\\ {\left[\begin{array}{l}\sin (\mathrm{i}{\varphi }_C-j{\varphi }_R)\\ \cos (\mathrm{i}{\varphi }_C-j{\varphi }_R)\\ \sin (\mathrm{i}{\varphi }_C+j{\varphi }_R)\\ \cos (\mathrm{i}{\varphi }_C+j{\varphi }_R)\end{array}\right]}_{\begin{array}{l}i=1:H_c\\ j=1:H_R\end{array}}\end{array}\right]$$

These matrices are the linear form of the AM Fourier series model introduced by Bachta et al.²¹ This model is an extension to dual Fourier series models for cardiac cartesian point motion, which directly accounts for the coupling between cardiac and respiration motion.

$$t({\varphi }_C,{\varphi }_R)=t_0+t_R({\varphi }_R)+t_C({\varphi }_C)+t_{CR}({\varphi }_C,{\varphi }_R)$$

$$t_R={\displaystyle \sum _{i=1}^{H_R}{a}_i\sin (i{\varphi }_R)+b_i\cos (i{\varphi }_R)}$$

$$t_C={\displaystyle \sum _{i=1}^{H_C}{c}_i\sin (i{\varphi }_C)+d_i\cos (i{\varphi }_C)}$$

$$t_{CR}={\displaystyle \sum _{i=1}^{H_{MC}}{\displaystyle \sum _{j=1}^{H_{MR}}{e}_{ij}}\sin (i{\varphi }_C-j{\varphi }_R)+f_{ij}\cos (i{\varphi }_C-j{\varphi }_R)+}{g}_{ij}\sin (i{\varphi }_C+j{\varphi }_R)+h_{ij}\cos (i{\varphi }_C+j{\varphi }_R)$$

where a, b, c, d, e, f, g and h are vectors of Fourier series coefficients, and H_R, H_C, H_mc and H_mrH_r are the numbers of respiration, cardiac, and coupled cardiac and respiratory harmonics, respectively. The values used for each trial were H_c = 5, H_r = 4, H_mc = 2 and H_mr = 1.

2.3 |. Registration experiments

2.3.1 |. Respiration phase registration

Using the maps of the heart surfaces and the models of periodic motion, spatial registration parameters were solved for ${\varphi }_C={\varphi }_C^{*}$, with ∅_R ranging from 0 to 1 in increments of 0.01 using the ICP algorithm.¹⁶ Spatial registration estimated using fiducials, T₀, was used as the seed for ICP. RMS error from each point to the nearest point on the map as well as the registration parameters for each test were recorded. Various distance metrics were used for the point-to-surface correspondence, including the use of surface normals extracted from the periodic rotation model,⁶ and surface normals estimated from the surface mesh; however, they were found to have minimal effect.

RMS registration error over the entire respiration cycle for each of the trials was shown previously in Figure 2. There is no discernible global minimum in the overall error, as it is nearly constant over the respiration cycle at 2 mm RMS. Registration parameters for each trial are shown in Figure 6(A) for translation and Figure 6(B) for rotation. These parameters are given in the heart-centric reference frames with rotation given in X-Y-Z Euler angles.

FIGURE 6.

Registration parameters across respiration phase for all trials for (A) translation and (B) rotation. Average registration transformations are shown as bold dashed lines

Inspection of the translation registration parameters, shown in Figure 6(A), reveals the first discernible trend. As the registration provided by the fiducials is used as the seed in the ICP implementation, the registration parameters found essentially yield the error in the fiducial registration. In all 3 directions, the mean registration reaches a global minimum at a respiration phase of approximately 0.3. This effect is amplified when looking at the total 3D translation and rotation for these trials, shown in Figure 7. Total translation is calculated as the norm of the translation vector, while total rotation is the geodesic distance.

FIGURE 7.

Magnitude of registration distances for (A) translation and (B) rotation. Average distances are shown as bold dashed lines

In 5 of 6 trials, the global minimum registration translation distance occurs at a respiration phase of approximately 0.3. The mean translation reaches a minimum of 9.9 mm at a respiration phase of 0.27. A similar trend does not hold for rotation, as the mean remains fairly constant on average. The mean rotation distance at the phase corresponding to minimum translation is 8.2°. From these results, we can define an empirical registration metric for respiration phase as the phase that minimizes the magnitude of the translation from the initial fiducial registration:

$${\varphi }_R^{*}=\underset{{\varphi }_R}{\arg \min d}\left(\underset{T}{T_0,\arg \min }{\displaystyle \sum }_i{\left(\mathcal{S}(u_i)=T{p}_i^{\varphi }\right)}^2\right).$$

2.3.2 |. Comparison with spatial registration

In order to determine the effect of the use of the periodic motion of the heart in registration, we compared the results obtained using the previously described spatiotemporal registration approach to a simpler spatial registration where point motion is treated as noise. The de-noised estimate of registration using the mean position of each point, ${\overline{p}}_i$, is given by

$$\overline{T}\underset{T}{=\arg \min }{\displaystyle \sum }_i{\left(\mathcal{S}(u_i)=T{\overline{p}}_i\right)}^2.$$

The points transformed by the estimated registration parameters were then projected to the closest point on the heart surface. Using the notation ${\overline{u}}_i$ and $u_i^{*}$ for the map frame coordinates of these projected points on the heart surface, the distance between where the two methods predict the robot is on the heart, δ_i, can be calculated as

$${\delta }_i=\Vert \mathcal{S}(u_{{}_i}^{*})-\mathcal{S}({\overline{u}}_i)\Vert .$$

Because the surface models are convex, the Euclidean distance between the points will underestimate the distance along the surface between them. However, due to the small curvature of the heart relative to the distances between points, this effect will be minimal.

3 |. RESULTS

3.1 |. Registration of respiration phase

Results of registration can be seen qualitatively in Figure 8, with the initial fiducial registration shown in Figure 8(A) and the final registration in Figure 8(B). The arrows represent the predicted surface normals from the motion models at the estimated image phases. Visually, the registered points more accurately fit the surface model of the heart and the predicted surface normals match the surface well. These images also confirm that the initial fiducial registration is a reasonable estimate for anchoring the empirical registration metric.

FIGURE 8.

A, The initial registration found using fiducials has noticeable error. B, Spatiotemporal registration considerably reduces point-to-surface error

3.2 |. Comparison of spatial registration

A plot of the difference in predicted location on the surface of the heart is shown in Figure 9. For each trial a single point error is shown as a dot, while the average over all points in a trial is shown as a square. Mean error per trial ranges from 1.7 mm to 8.8 mm, with a mean error of 4.2 mm across trials.

FIGURE 9.

Difference in estimated position between the spatiotemporal registration method described in this paper and a more simplistic method that uses averaging for each data point to filter out physiological motion

4 |. DISCUSSION

The results presented in this work have several important implications, as well as limitations, which must be accounted for when interpreting their meaning. First, an assumption of the presented methods is that the difference in shape of the heart intraoperatively and preoperatively, when imaging was conducted, is sufficiently small. Several factors may contribute to a difference between the shape of the heart as observed by the robot and the models produced from imaging, including the presence of the robot in the intrapericardial space, errors in extracting the model from preoperative imaging, and physiological changes in the patient due to surgical preparation and access. In the data presented, no evidence was observed of any these factors affecting performance. The presented work may be extended to account for these effects using more powerful statistical or nonrigid registration methods in place of ICP.

Next, temporal registration cannot be achieved by optimizing over the error in the fit between the data and model. This means that the shape of the heart at the image phases is not unique to a degree that can overcome the noise in the surface and motion models. Due to this, registration in cardiac phase is not feasible without further information. This is not a significant limitation, however, because preoperative imaging generally includes a cardiac-gated image set from which ground-truth cardiac phase can be extracted.

Motion of the heart due to respiration has been shown to be, in part, a rigid motion applied to the entire heart.²² This finding is consistent with our observation of invariance of error with respiration phase, and provides grounds for the empirical respiration-phase registration metric presented. Minimizing the magnitude of the translational component of spatial registration simply finds the rigid motion that most closely agrees with fiducial registration.

Next, we showed that the mean difference between the robot location predicted using the periodic motion of the heart and a simplistic method involving merely filtering it out is approximately 4 mm; presumably this represents a reduction in registration error by the spatiotemporal method compared to the simpler method. While the acceptable positioning error will be intervention-specific, identifying the required clinical accuracy is difficult. Estimates of clinically required accuracy are often ad hoc and based on the best guess of the clinician. An ‘acceptable’ error sometimes mentioned is 5 mm.¹⁰

Several factors must be noted here. First, achieving this level of accuracy while ignoring the motion requires an accurate estimate of the mean position of the robot. Because of the low frequency of the respiration motion, this still requires significant time to observe, time that could be used learning the actual motion of the heart. Finally, we note our expectation that the primary benefit of this work will be to push the envelope of what is possible in minimally invasive surgery as a means to enable new interventions