MRI - 3D Ultrasound - X-ray Image Fusion with Electromagnetic Tracking for Transendocardial Therapeutic Injections: In-vitro Validation and In-vivo Feasibility

Abstract

Myocardial infarction (MI) is one of the leading causes of death in the world. Small animal studies have shown that stem-cell therapy offers dramatic functional improvement post-MI. An endomyocardial catheter injection approach to therapeutic agent delivery has been proposed to improve efficacy through increased cell retention. Accurate targeting is critical for reaching areas of greatest therapeutic potential while avoiding a life-threatening myocardial perforation. Multimodal image fusion has been proposed as a way to improve these procedures by augmenting traditional intra-operative imaging modalities with high resolution pre-procedural images. Previous approaches have suffered from a lack of real-time tissue imaging and dependence on X-ray imaging to track devices, leading to increased ionizing radiation dose. In this paper, we present a new image fusion system for catheter-based targeted delivery of therapeutic agents. The system registers real-time 3D echocardiography, magnetic resonance, X-ray, and electromagnetic sensor tracking within a single flexible framework. All system calibrations and registrations were validated and found to have target registration errors less than 5 mm in the worst case. Injection accuracy was validated in a motion enabled cardiac injection phantom, where targeting accuracy ranged from 0.57 to 3.81 mm. Clinical feasibility was demonstrated with in-vivo swine experiments, where injections were successfully made into targeted regions of the heart.

1. Introduction

Cardiovascular disease is one of the primary sources of mortality in the world, especially in industrialized nations. In particular, myocardial infarction (MI), which is usually caused by an abrupt thrombotic occlusion of a coronary artery that results in myocardial necrosis, is responsible for nearly 50% of deaths globally [1]. The American Heart Association estimates 600,000 Americans will experience new MI and about 320,000 will experience recurrent MI [2]. Eighteen percent of men and 23% of women over the age of 40 will die within a year of their first MI [2]. Those who present late or fail early treatment often develop progressive ventricular enlargement (also called adverse remodeling), heart failure, and sudden death. Patients who survive an MI but develop heart failure are 10 times more likely to die than other MI survivors [3]. Novel biologic agents such as stem cells represent an exciting and potentially revolutionary advance in post-MI therapy; however, little is known about the optimal delivery method of these agents. In this paper, we present a multi-modal image fusion system for catheter based delivery of theraputic agents in the beating heart. The system, the first-of-its-kind for this application, fuses 3D echocardiography (3DEcho), X-ray fluoroscopy (XRF), and pre-operative magnetic resonance (MR) image based anatomical models with electromagnetic (EM) tracking to create a versatile imaging platform. It allows the physician to perform endomyocardial injections in situations where targeting accuracy may be crucial from the standpoint of safety and efficacy, with the potential for reducing patient and staff exposure to ionizing radiation.

Open chest, direct intramuscular injection of bone marrow stem cells into infarct borders results in functional myocardial recovery in mice post-MI [4]. However, human studies that infused cells into coronary arteries have not shown robust benefits [5, 6, 7, 8, 9, 10, 11]. Contributing to this discrepancy may be the delivery method and target location which influences cell retention and engraftment. For example, intra-coronary cell infusion requires a patent infarct related artery and is associated with low acute retention [12, 13, 14, 15]. Open chest procedures are impractical and risky in this clinical population because they require general anesthesia, thoracotomy, and long recovery times. Minimally invasive, controlled intramuscular catheter injection directly into infarct borders from the inside surface (transendocardial) is conceptually safer and would be associated with faster patient recovery than an open chest approach. In addition, the transendocardial approach offers improved cell retention in animal studies, and therefore, may offer enhanced therapeutic effect. However, there are safety issues involved with transendocardial injections, namely the risk of myocardial perforation by interventional devices.

Clinically, the risk of spontaneous ventricular wall, papillary muscle, and septal rupture occurs 3–5 days post-MI due to tissue fragility, and the risk diminishes thereafter. Coincidently, this time frame may be the most optimal for stem cell transplants to offer the greatest efficacy [16]. Although newly infarcted myocardium remains thick, it is “friable,” meaning the tissue is mechanically weak and prone to perforation, especially if “prodded” by stiff endomyocardial catheters. Consequently, there remains a serious concern of cardiac perforation caused by a catheter if pressed against the new, friable infarct. Furthermore, both acute and chronically infarcted myocardium is not perfused and therefore transplanted cells are not likely to thrive within the infarcted region. This observation underlies the hypothesis that the infarct border zone is the optimal location for transplantation of therapeutic agents. An acceptable distance-to-infarct accuracy threshold has yet to be defined and is probably dependent upon the biologic mode of action of the cell or biologic agent. Tools that can predictably map injection sites will allow investigators to study the effect of injection distribution or redundant injection locations. Therefore, from the standpoints of safety and efficacy, accurate image guidance is a crucial component of catheter-based targeted endomyocardial injection therapy.

Catheter-based transendocardial delivery of various biologic agents are being investigated in human trials using several catheter delivery systems for mostly chronically ischemic or non-ischemic cardiomyopathy [17, 18, 19, 20, 21, 22]. These procedures are conducted in mild to moderately sedated, non-intubated, free-breathing patients lying supine on a cardiac catheterization table. Two primary interventional imaging modalities have been used to guide these procedures: i) XRF [17, 23], and ii) electroanatomical mapping (EAM) (also known as electromechanical mapping, EMM) [18, 19, 24, 20, 25, 21, 26, 27, 28, 22]. XRF uses the principle of projection imaging and therefore offers poor depth perspective and poor soft tissue contrast. For example, XRF cannot itself provide visualization of the precise infarct boundary. Despite being the de facto imaging modality for guiding nearly all catheter-based cardiac interventions, XRF has limited usefulness as a stand-alone imaging modality for targeted transendocardial therapy.

NOGA (Cordis Corporation, Bridgewater, NJ) is a clinical EAM-based navigation system that has been used to perform transendocardial stem cell injections in human studies [18, 19, 24, 20, 25, 21, 26, 27, 28, 22]. 3D maps are acquired by moving a flexible, EM-tracked catheter into the left ventricle and recording numerous locations of the catheter tip, along with displacement and voltage measurements. This technique was originally validated in [29]. Precision and accuracy measurements were defined as the stability of repeated measurements (“loop stability”), and small relative catheter displacements compared to known values. Targeted injection accuracy relative to known landmarks has not been investigated. The clinical studies that used NOGA to perform stem-cell injections targeted the entire ischaemic region, so accuracy was not as critical as when targeting the discrete infarct border zone. As previously mentioned, the safety of this approach for acute MI cases is questionable due to the fragile nature of newly infarcted tissue, with limited experience showing that the procedure is safe[25]. Aside from safety concerns, one of the major drawbacks of the NOGA system is the time it takes to perform the mapping, resulting in long procedures (reported as 81 ± 19 min. in [24], 62 ± 18 min. in [21], and 54 ± 15 min. in [25]). Furthermore, the ability of EAM to properly detect the actual infarct border zone location has been brought into question [30]. The authors found that infarct border localization using EAM does not always correspond to that obtained from delayed enhancement MR, the clinical gold standard [31, 32].

Attempts have been made to overcome the limitations of X-ray imaging by integrating pre-procedural MR images. This technology is often referred to as XFM, and the first such system for endomyocardial injections was proposed in [33]. Point based registration was performed by using X-ray/MR visible markers (“fiducials”) placed on the chest. Contours of the left ventricular (LV) epicardium, LV endocardium, right ventricular (RV) endocardium, and infarct were used to generate 3D anatomical models of the heart, which were then overlaid on top of the biplane XRF images for improved intraprocedural catheter guidance. Injections containing tissue dye and MR visible contrast were made into the border zone and interior of the infarct in a swine model. A similar XFM system was presented in [34], where the major difference was that the MR derived 3D models were registered to biplane XRF by visual alignment with a ventriculogram. Similar systems have been proposed for electrophysiology procedures [35][36], some of which support integration of EAM generated maps [37].

A common limitation of the previously discussed XFM and NOGA systems is the lack of real-time cardiac motion information. Injections are typically performed only at diastole, minimizing mis-registration due to beating motion. However, respiratory motion can result in tissue displacements of up to 2.0 cm [38]. Interventional MRI based procedures have been demonstrated [39] and are capable of providing the interventionalist with dynamic, high-resolution images of the heart that can capture this motion. A few studies [40, 41, 42, 43, 44, 45] focused on demonstrating safety and feasibility of targeted endomyocardial injections using real-time MRI in swine or canine models. The ability to perform injections into correct regions was shown, but targeting accuracy was not quantified as a distance error measure. No human clinical trials using interventional MRI for transendocardial therapy have been conducted. A comprehensive review of these procedures [46] noted severe limitations, including the ability to monitor patients, risk of device heating causing burns, lack of commercial testing and development of compatible devices, and lack of dedicated combined multimodal interventional suites, all contributing to a lack of acceptance for clinical use.

Echocardiography (Echo) can also provide real-time images of the heart, without many of the aformentioned issues related to real-time MRI. Echo is relatively inexpensive, provides real-time images at high frame-rates, and offers soft-tissue contrast that isn’t possible with XRF. The recent availability of live 3DEcho offers volumetric imaging, enabling visualization of complex cardiac anatomy and valvular function. Limitations do exist, however, such as small field-of-view (FOV), poor signal-to-noise ratio, presence of artifacts, and operator and patient dependent image quality. In [47], swine experiments testing the feasibility of echo-guided endomyocardial injections were conducted. A catheter was directed toward three anatomical targets in the heart and injections were performed under purely 3D and biplane echo guidance with the aid of echo contrast. From post-mortem examination, 95% of the attempted injections were confirmed, but only 83% were made into the correct zone.

Echo imaging has also been used in recent multimodal image fusion prototypes for cardiac interventions. In [48], direct access and targeted catheter injection accuracy comparisons were made between endoscopic (considered as a control), 2DEcho guided, and pre-operative model enhanced 2DEcho guided imaging within a beating heart phantom. For catheter injections, it was shown that model-enhanced echo guided RMS targeting errors were much smaller than 2DEcho-only guided errors.

We present an innovative solution to address precise targeting for transendocardial therapeutic delivery. Delayed enhancement MRI is the clinical gold standard for infarct imaging and therefore a pertinent imaging modality for MI applications. Real-time 3DEcho can image cardiac motion directly, without relying on ECG gating or respiratory bellows that track irregular heart and respiratory motion poorly in the clinical setting. XRF allows for confirmation of device locations and a familiar imaging modality for the interventionalist. Finally, EM-tracking provides accurate, real-time catheter tip definition in 3 dimensions without additional radiation exposure. The EM system also serves as an intermediate framework to register the MR and XRF images to 3DEcho. By combining all three modalities with EM-tracking, we achieve i) superior cardiac tissue and infarct definition, ii) full cardiorespiratory motion visualization and iii) 3D catheter tip tracking. The significance is that this system is the first flexible, truly multi-modal image fusion system implemented in the clinical setting for transendocardial injections. This has tremendous long-term potential to reduce X-ray and contrast dose and improve procedural safety, efficiency, and outcomes. We hypothesize that such a system is accurate and feasible, and demonstrate this with in-vitro and in-vivo experiments.

2. Methods

The overall approach to achieve intra-operative registration and fusion with the aid of EM-tracking is shown in Fig. 1. The TEE probe and catheter are independently pre-calibrated to the tracking system. Pre-operative MR is registered to the EM-tracking system using patient fiducial markers in the operating room. XRF is registered to the EM system using a calibration phantom. This binds registration between MRI, XRF, catheter, and 3DEcho, providing the capability for the visualization of the pre-operative plan, interventional devices, and live anatomy. The proposed EM-based framework has the following advantages:

• An easy plug-and-play framework to replace different probes/tools.

• A pre-calibrated suite to minimize the number of intra-operative registrations that need to be performed.

• Real-time tracking of echo probe/ interventional tools.

• Minimal dependence on any one particular modality.

Figure 1.

Approach for integrating MRI, 3DEcho, and XRF using an EM-tracking framework.

The platform allows the use of the system in a variety of different combinations, enabling interventional navigation using (i) multiplanar 2D views showing echo overlaid onto MR images with EM based catheter positioning and automatic slice selection; (ii) a 3D heart model segmented from MR with EM-tracked catheter positioning and biplane echo overlay, and (iii) an MR based anatomical model with 3DEcho projected onto XRF views.

2.1. Registration Workflow

2.1.1. 3DEcho to EM Calibration

To track the 3DEcho volumes within the EM framework, it was necessary to calibrate the tracked TEE probe to the 3DEcho volume. A method and toolkit [49] was developed using MATLAB (Mathworks, Natick, MA) to facilitate this calibration (Fig. 2). It allowed a user to simultaneously acquire 3DEcho volumes and the locations of EM-tracked tools. 3DEcho data was acquired using proprietary LAN-based streaming software and hardware for the Philips iE33 ultrasound machine. The 3DEcho was tracked by rigidly attaching EM sensors onto the TEE probe with a coating of curable polyurethane plastic (Smooth-Cast 300 Series, Smooth-On, Easton, PA, USA) (Fig. 2). Echo images were calibrated at a 10 cm imaging depth for both the Live3D and Thick Slice acquisition modes. This calibration needed to be done only once in the laboratory, and was therefore not part of the clinical workflow.

Figure 2.

Close-up of the TEE probe after the sensors have been mounted. The white material is the plastic layer covering the sensors (left). Physical setup for EM/3DEcho calibration, showing the EM-tracked needle with the TEE probe in a water bath, and the corresponding echo image (right).

Two algorithms were used, a point-to-point (P2P) based calibration and a point-to-line (P2L) based calibration. For the P2P method, an EM-tracked needle tip was simultaneously recorded and imaged within the 3DEcho volume, allowing calibration by point cloud registration [50]. The P2L method was investigated because the needle tips were difficult to visualize in Thick Slice mode. The P2L-based calibration was found by minimizing the distance between the EM-tracked needle, represented as a line in the EM coordinate system, and points lying on the needle in the in the echo image. Further details about the methods can be found in [49].

For analysis, the registration was calculated under two scenarios: using all available points and using only inlier points. Outlier detection was employed because of the inherent uncertainty in segmenting objects in US volumes. Outliers were found using the MLESAC [51] algorithm (a variation of RANSAC). Throughout the rest of this paper, we refer to the accuracy metrics fiducial registration error (FRE) and target registration error (TRE). Unless otherwise stated, fiducial registration error (FRE) is the RMS error between the registered point clouds, and target registration error (TRE) is the distance between registered data points that were not themselves used to calculate the registration. FRE and TRE were used to characterize registration accuracy for the EM-3DEcho calibration.

2.1.2. XRF to EM and XRF to 3DEcho Registration

XRF was registered to the EM space using two custom-built “prism” and “planar” phantoms (Fig. 3), designed for pre-procedural and intraprocedural calibration, respectively. The prism phantom’s geometry was more optimal for calibration, but the planar phantom was easier to integrate into the clinical workflow because it had a smaller spatial footprint. The phantoms contained metal beads and grooves that were clearly visible in the XRF images, and were machined with a high degree of precision so that the geometry and location of the features were precisely known within an arbitrarily defined phantom coordinate system P. Both also had a rigidly mounted EM sensor attached. Finding the registration transformation between XRF and EM-tracking, ^XRFT_EM, was performed in two steps: calibration between the EM space and the phantom space (^PT_EM, once in the lab), and registration between the phantom space and the C-arm space (^XRFT_P, intra-operatively). Knowing the overall registration ^XRFT_EM allowed for fusion between the 3DEcho and XRF images (eq. 1), since 3DEcho was calibrated to the EM system (sec. 2.1.1). Further details concerning the XRF to EM registration step are located in [52].

$${}^{XRF}{T}_{3D\mathit{\text{Echo}}}={}^{XRF}{T}_P\times {}^P{T}_{P\mathit{\text{sensor}}}\times {}^{P\mathit{\text{sensor}}}{T}_{\mathit{\text{Probe}}}\times {({}^{3D\mathit{\text{Echo}}}{T}_{\mathit{\text{Probe}}})}^{-1}$$ (1)

Figure 3.

The prism (left) and planar (right) XRF to EM calibration phantoms.

XRF to EM registration accuracy was determined by segmenting an EM-tracked needle in the XRF space and comparing its location to the transformed EM location measurement. This was performed 25 times at 3 table heights and 3 horizontal positions within the XRF field of view. 3DEcho to XRF registration accuracy was measured by placing a needle in a water tank, simultaneously imaging the needle with XRF and 3DEcho, segmenting the needle tip in both modalities, and measuring the distance between the two locations. This was done 10 times with different needle poses for each phantom.

2.1.3. MR to EM and MR to 3DEcho Registration

Registration between the MR and EM coordinate systems was performed using external fiducials. Fiducials were manually segmented from the MR images, resulting in a set of points p_MR. The fiducials were then touched with an EM-tracked pointer to acquire their locations in the EM coordinate space, p_EM. The registration transformation that aligned the two point sets, ^TEET_MR, was found using using the algorithm from [50]. Because 3DEcho was calibrated to the EM space, it was also possible to register 3DEcho to MR (eq. 2).

$${}^{MR}{T}_{3D\mathit{\text{Echo}}}={}^{MR}{T}_{\mathit{\text{Probe}}}\times {}^{\mathit{\text{Probe}}}{T}_{EM}\times {}^{EM}{T}_{3D\mathit{\text{Echo}}}$$ (2)

Registration accuracy for MR to EM and MR to 3DEcho was performed using a cardiac phantom. Seven MR compatible fiducials (Beekley Corporation, Bristol, CT) were attached to the phantom, which was then scanned to generate an MR image. Thread like fiducials that were visible in MR and 3DEcho were also attached for validation of the MR to 3DEcho registration.

Because a minimum of 3 fiducials were needed to register the images, a statistical analysis was performed that used all possible combinations of fiducials to find the transformation matrix. For example, if there were 7 total fiducials, all $\left(\begin{array}{c}\hfill 7\hfill \\ \hfill 3\hfill \end{array}\right)$ possible combinations of 3 fiducials were tested, as well as all all $\left(\begin{array}{c}\hfill 7\hfill \\ \hfill 4\hfill \end{array}\right)$ possible combinations of 4 fiducials, and so on. This was done to analyze the effect of fiducial count on the overall registration accuracy. FRE was calculated for each combination, and TRE was measured for each fiducial that was not used in the registration. To calculate TRE for the MR to 3DEcho registration, the distance between the tips of the thread-like fiducials in MR and in 3DEcho were measured.

For pre-operative registration during the in-vivo experiments, fiducials were stuck on the surface of the animal’s chest before the MR scan, manually segmented from the images after the scan, and touched with an EM-tracked pointer once the animal was on the operating table. A typical fiducial placement configuration, as well as the location of the fiducials relative to the heart, is shown in Fig. 4. FRE and TRE were calculated as in the phantom experiments.

Figure 4.

Volume renderings of the swine MRI with chest fiducials highlighted in blue. Left: Image of the skin surface. Middle: Internal image of the heart from the same point of view as the surface image. Right: Internal image of the heart rotated about the inferior-superior axis to show the distance of the fiducials from the heart. The spatial configuration of the fiducials relative to the heart has an effect on the overall targeting accuracy.

2.1.4. Catheter Design, Tracking, and Calibration

While EM-tracked catheters exist for some proprietary clinical systems, off-the-shelf models for research do not. For example, the NOGA system uses the EM-tracked “Myostar” catheter, but it is designed to be used only once and is not compatible with the NDI Aurora system. We therefore designed an EM-tracked injection catheter by mounting a 5DOF EM sensor into the interior of a steerable catheter sheath (Channel catheter, Bard Medical, Covington, GA) (Fig. 4). We used a sharpened nitinol hypotube (Johnson Matthey Medical, West Chester, PA) for the injection needle. Off the shelf calibration software (NDI ToolViewer, Waterloo, ON, CA) was used to calibrate the distance from the EM sensor to the catheter tip.

2.2. Cardiac Injection Phantom

Targeting experiments were carried out in a realistic, respiratory motion enabled cardiac phantom. The phantom was made from poly-vinyl alcohol (PVA), a hydro-gel material with tissue-like echo and MR imaging properties. It was modeled from a cadaver heart and realistically shaped. For targeting experiments, it was desirable to have easily removable targets that could be replaced rapidly to facilitate numerous injection experiments. This was accomplished by removing the apical half of the phantom and replacing it with a removable “slab.” Within this slab were 4 circular target holes, each 9 mm in diameter, where targets could easily be added and removed (Fig. 6).

Figure 6.

(Left) A custom validation phantom designed for in-vitro experiments. (Right) The removable “slab” and injection targets, with dye from catheter injections.

The PVA phantom was mounted on a stage, itself mounted on a ramp within a water tank. This enabled the phantom to move back and forth with a small rotation in the sagittal plane, similar to the way the human heart moves with respiration. An electronically controlled pneumatic pump system was used to move the phantom back and forth. A pressure cylinder attached to a differential pressure pump, which was controlled by a micro-controller (RAPU, Brookshire software, Springfield, VA), was used to drive motion of the phantom. The microcontroller allowed for millimeter resolution positioning as a function of time for realistic breathing motion of the phantom. MR to EM registration was performed as described in section 2.1.3.

A series of injections were performed with our prototype EM-tracked catheter to test the accuracy of the system under different amounts of respiratory motion. Food dye was injected into the targets to localize the actual injection site. Based on previous reports of typical human respiratory motion [38], we tested targeting accuracy under 0 mm, 5.0 mm and 20.0 mm of inferior-superior translation. All experiments were performed at a respiratory rate of 15 breaths per minute, a typical rate for a sedated patient. We defined “injection error” as the distance between the target centers and the actual injection locations.

2.3. In-vivo Experiments: System Feasibility

We performed a total of four in-vivo swine experiments to test feasibility. Procedures were approved by the local Institutional Animal Care and Use Committee. Injections were performed in the fourth experiment. The experimental workflow is shown in Fig. 7.

Figure 7.

In-vivo experimental workflow.

MR (1.5T, 8 channel cardiac phased array coil, Signa HDx, GE Healthcare, Milwaukee, WI) was performed before and after the procedures. Left ventricular (LV) short axis oblique slices were obtained using balanced steady state free procession (bSSFP) imaging. Delayed enhancement (DE) imaging was performed 15 minutes following administration of 0.15 mmol/kg Gd-BOPTA (Multihance, Bracco Diagnostics, Princeton, NJ) using a standard inversion recovery based sequence. The DE scans were gated at mid-diastole. All MR imaging was performed at end expiration by temporarily suspending mechanical ventilation. Scan parameters can be found in [34].

Following MR, the swine was moved to a cathlab and placed on the operating table. Care was taken to make sure that the position of the animal did not change between MR imaging and transport to the cath-lab. Once on the table, the EM field generator was mounted next to the swine so that the center of the field generator was aimed at the heart. The EM-tracked TEE probe was introduced and maneuvered until the highest quality image of the heart was attained. EM-tracked sensors were introduced through the femoral artery and maneuvered through the aorta into the left ventricle.

For the fourth experiment the 3D workstep was used, where the devices were viewed within a three-dimensional mesh and/or surface model of the cardiac anatomy (Fig. 8). Injections were performed using this workstep. Five proposed injection sites were segmented from the MR images and included in the 3D model. During the procedure, the location and orientation of the catheter tip was displayed within the anatomical mesh models (Fig. 8). Biplane ultrasound was also available for display. The animal was injected with co-localizing agents for visualization in MR and tissue slices. Ferex (BioPal, Worchester, MA) was used to visualize the injection locations in post-op MR, and tissue dye (Bradley Products, Bloomington, MN) was used to confirm the injection locations in the ex-vivo heart. Following the injections, MR imaging was again performed according to the same protocols as the pre-procedural images in order to locate the injections. The animal was then sacrificed and the heart was removed, fixed in formalin, and stored for later imaging and examination.

Figure 8.

3D MR/Echo/EM workstep. (Left) Infarct and ventricle surface models are displayed simultaneously with 3DEcho images. This image was from a practice run where a clot caused a double infarct. (Right) The ventricle (green mesh), infarct (yellow), injection target zone (red), and catheter pose are both shown for the actual injection experiment. Note that rendering properties and colors changed between experiments as the interventionalist’s preferences evolved.

3. Results

3.1. Registration Workflow

3.1.1. 3DEcho to EM Calibration

Results for the 3DEcho to EM calibration are shown in Table 1. TRE was calculated only on random inlier points that were not included in the calibration calculation. For the Live3D mode, the TRE values were marginally better for the point to line calibration method. Live3D was more accurate than Thick Slice mode, presumably due to the larger field of view allowing more accurate segmentation of the catheter.

Table 1.

Accuracy of the Ultrasound to EM calibration. P2P = Point to Point calibration method, P2L = Point to Line calibration method

Calibration Method	FRE (all points, mm)	FRE (inliers, mm)	TRE (mm)
P2P, Live3D	1.55	1.37	1.48
P2L, Live3D	1.22	1.30	1.45
P2L, Thick Slice	1.66	1.54	1.63

3.1.2. XRF to EM and XRF to 3DEcho Registration Results

TRE mean and standard deviation are reported for the XRF to EM registration accuracy experiments in Table 2. XRF to 3DEcho registration accuracy was measured for both phantom designs. For the prism phantom, the TRE was 2.38 ± 0.96 mm, while for the planar phantom, it was 3.72 ± 0.48 mm (n=10 for both phantoms). In general, the planar phantom was less accurate than the prism phantom, which can be explained by increased localization error in the direction of the X-ray beam resulting from low variance of feature locations in that direction.

Table 2.

XRF to EM registration accuracy for different table heights and positions. Average TRE ± St. Dev. for each table height and position are shown (n=25 trials each)

		Position
Type	Table Height (cm)	1	2	3
	39	0.99 ± 0.39	0.88 ± 0.44	0.68 ± 0.36
Prism	36	0.87 ± 0.38	0.87 ± 0.39	0.88 ± 0.39
	33	1.45 ± 0.34	0.68 ± 0.21	0.81 ± 0.23
	39	1.37 ± 0.38	1.00 ± 0.31	1.59 ± 0.31
Planar	36	1.44 ± 0.60	1.23 ± 0.54	1.60 ± 0.66
	33	1.92 ± 0.48	1.67 ± 0.31	1.65 ± 0.41

3.1.3. MR to EM and MR to 3DEcho Registration Results

The results of the statistical analysis based on different fiducial combinations for the phantom and in-vivo MR to EM registrations are shown in Table 3, while results for MR to 3DEcho registration for the phantom experiments are shown in Table 4. Notice that increasing the number of fiducials increased the FRE, but lowered the TRE.

Table 3.

MR to EM registration accuracy for the phantom and in-vivo experiments. FRE was calculated for varying numbers of fiducials used in the registration, and TRE was calculated using fiducials that were left out of the registration.

Phantom
Fiducials	Trials	FRE Mean (mm) ± St. Dev.	TRE Mean (mm) ± St. Dev.
3	35	1.21 ± 0.38	3.10 ± 0.90
4	35	1.47 ± 0.29	2.55 ± 0.57
5	21	1.60 ± 0.22	2.35 ± 0.65
6	7	1.68 ± 0.16	2.15 ± 0.92
7	1	1.73	-
In-vivo
Fiducials	Trials	FRE Mean (mm) ± St. Dev.	TRE Mean (mm) ± St. Dev.
3	84	3.11 ± 1.13	7.80 ± 5.15
4	126	3.76 ± 0.75	6.27 ± 2.45
5	126	4.07 ± 0.51	5.89 ± 1.95
6	84	4.26 ± 0.36	5.67 ± 1.59
7	36	4.39 ± 0.24	5.52 ± 1.32
8	9	4.47 ± 0.15	5.46 ± 1.14
9	1	4.54	-

Table 4.

MR to 3DEcho registration accuracy for the phantom experiments. FRE was calculated for varying numbers of fiducials used in the registration MR to EM component of the registration, and TRE was calculated using fiducials that were left out of the MR to EM component of the registration.

Fiducials	Trials	TRE Mean (mm) ± St. Dev.
3	35	3.23 ± 0.49
4	35	3.02 ± 0.30
5	21	2.96 ± 0.20
6	7	2.93 ± 0.14
7	1	2.91

3.2. Phantom Experiments: Catheter Injection Accuracy

Results for the phantom catheter injection experiments are shown in Table 5. Injection error is the distance between the center of the phantom target and the actual injection location marked by food dye. Not surprisingly, the data showed that respiratory motion increased targeting error.

Table 5.

Mean values of FRE and injection error for each set of phantom injection experiments.

n	Motion (mm)	Mean FRE (mm)	Mean Injection Accuracy ± Std. Dev.
15	00.0	0.76	1.36 ± 0.79
16	05.0	0.86	2.23 ± 1.33
10	20.0	0.85	2.54 ± 1.27

3.3. In-vivo Experiments: System Feasibility

For the MR to 3DEcho fusion workstep, the myocardial borders in the 3DEcho images aligned well with the those from the MR volumes and the devices (Fig. 9). On several occasions, we noticed registration errors when the devices were in regions near the boundaries of the EM field generator FOV. For example, when the catheter was moving through the descending aorta, the tip appeared to be outside the 3D vessel model, presumably due to the location of this structure with respect to the EM FOV. We also noticed that bringing the X-ray C-arm near the EM field generator caused mis-registration errors. For example, the 3DEcho images were well aligned with the MR images following registration, but when the C-arm was brought closer, the 3DEcho image “drifted” out of registration, returning to the correct location when the C-arm was again removed.

Figure 9.

Long (left) and short (right) axis slices of the pre-operative MR image (grayscale) with overlaid 3DEcho (colored). Notice the general agreement between anatomical features in the two images. The green lines reveal the location of the image slice in the other view.

A qualitative analysis showed that it was possible to make relatively accurate injections using the 3D workstep. Four out of five injections were confirmed from post-op MR, ex-vivo MR, and tissue examination (Fig. 10). It was not possible to determine an exact correspondence between pre- and post-operative injection locations due to relative proximity, but the MR and tissue images showed that there were three injections confirmed in the anterior and septal wall and one confirmed near the apex. Two injections were at the infarct border (Fig. 10 col. 1,3), while one was in the infarct near the border (Fig. 10 col. 2). The location of the apical injection appeared to be just outside the infarct border in the tissue sample (Fig. 10 col. 5). MR and tissue samples did not reveal the location of one of the injections (Fig. 10. col. 4).

Figure 10.

In-vivo injection results. The corresponding points are labeled the same color. The infarct scar is visible as bright areas in the ex-vivo MR and brown areas in the tissue samples, and is outlined in pink. Columns 4 and 5 have missing images because the 4th injection was never confirmed, and the 5th injection was not visible in the ex-vivo MR images. Top Row: Pre-operative MR with proposed injection sites. 2nd Row: Ex-vivo high-resolution MR. 3rd Row: Post-op in-vivo MR. 4th Row: Tissue slices with tissue dye showing injections. * Injection not identified in the image series.

4. Discussion

In this paper, we presented a system that combines XRF, MR, and real-time 3DEcho within an EM-based framework. The system and algorithms developed in this research have focused on the need for an interventional cardiologist to perform these injections as accurately as possible in a beating heart. Calibration and registration accuracies were demonstrated for MR to EM and 3DEcho, and XRF to EM and 3DEcho. We also demonstrated injection accuracy in a cardiac injection phantom under differing amounts of respiratory motion, and feasibility in animal experiments.

Each of the sub-system registrations were found to have a TRE of less than 2 mm. In general, the most accurate registration was between XRF and EM. This is probably because it is possible to more accurately segment fiducials in XRF compared to other modalities due to lack of artifacts, excellent contrast, and high image resolution. Another major factor was the ability to incorporate a large number of data points into the registration. The 3DEcho to EM calibration was presumably less accurate because of the inability to accurately segment the EM-tracked needles in the US images. Collecting as many data points as possible was needed to compensate for this. For example, even though three points were in theory suffcient to compute the 6-DOF calibration, in practice at least 30–35 were needed to converge to a stable FRE and TRE.

XRF and MR to 3DEcho registration showed the largest overall TRE in the phantom experiments. This can be explained by the fact that these registrations were highly indirect, involving a number of registration steps, each having its own source of errors. For example, XRF to 3DEcho registration involves four registration steps, which results in an overall registration error of almost 4.0 mm. A solution to this may be to incorporate image-based registration solutions between XRF and 3DEcho, such as proposed in [53, 54]. MR to 3DEcho registration is also possible [55], but challenging due to the nature of echo images.

There were some interesting take away points from the results of the MR to EM registration. First, increasing the number of fiducials decreased the TRE, but increased the FRE. This is in line with the theoretical results on point based rigid registration from [56], where it was shown that lowering FRE does not necessarily result in lower TRE. This is an important lesson to remember, as our team was often tempted to re-do the registration with less fiducials to get a “better” (lower) FRE value, even though the post-experiment analysis showed that this actually increased the TRE. As with the EM to 3DEcho and EM to XRF registrations, increasing the number of fiducials helped to converge to the lowest possible TRE. Second, there was a large increase in error for the in-vivo experiments. This can be explained by the electromagnetic environment of the cath-lab having more EM distortions than an ideal laboratory environment. Also, the fiducial configuration for the in-vivo experiment (Fig. 4) was less optimal than for the phantom. Results from [56] show that TRE depends on the distance of the target point from the centroid of all the fiducials and the localization error of the fiducials, among other factors. Localization error is probably higher in the cathlab for a chest fiducial configuration, because the fiducials are more spread out within the FOV of the EM field generator, and EM field distortions are generally higher in that environment.

For the in-vitro injection accuracy, we demonstrated accurate catheter injection accuracy in a phantom that was able to mimic respiratory motion, using the 2D MR/3DEcho fusion workstep with EM-tracked catheter guidance. The catheter injection accuracy was 1.36 mm when no respiratory motion was present, and roughly doubled for both 5.0 and 20.0 mm of respiratory motion. The data suggested that respiratory motion increased targeting error, which was expected, but that increased amplitude of respiratory motion had little effect on the errors. We hypothesize that accurate targeting in the presence of motion is a function of knowing the respiratory and cardiac timing correctly, or having accurate, real-time motion compensation.

For the in-vivo injections, we showed from visual inspection that four out of five injections were near their proposed injection sites. Three of the injections were made very close to the infarct border zone, while one of the injections was in the infarcted region near the border zone. It should be noted that the injections were performed with a sub-optimal prototype catheter that had limited range and steerability, as well as a cumbersome injection mechanism. It was reported by the physician performing the injections that commercial catheter models designed for these procedures are easier to use and would result in better outcomes. Some of the proposed sites were impossible to maneuver to, and the physician was forced to settle for “as close as possible.” The sub-optimal design may also have been a reason that one of the injections could not be confirmed, as the relatively dull needle may not have penetrated the myocardium.

One of the major sources of error in our approach was the existence of cardiorespiratory motion. We demonstrated the ability to fuse 3DEcho with MR in both the 2D and 3D worksteps (Figs. 9,8), but we found that the echo images were difficult to interpret in the 3D workstep (Fig. 8). This highlights the need for real time tracking of cardiorespiratory motion implemented into the fusion system. A motion compensation algorithm was demonstrated by our group [57] and others [58]. These methods have been designed and validated using offline in-vitro and in-vivo data. However, a reliable clinical solution remains a challenging technical problem due to variations in 3DEcho image quality and the inherent added complexity of the clinical environment. An important issue which is not often addressed is correcting for the inherent system latency of streaming 3DEcho data, which is the topic of further research. Management of latencies of different parts of the system will be crucial to translate this technology to human clinical studies. Forward looking motion estimation such as presented in [59] is a potential option. Furthermore, quality control of the clinical workflow is of interest. For example, if EM field distortions cause mis-calibration of the image fusion system, or if there is a break down in the registration, can this be detected? Tools to automatically detect these distortions and compensate for them have been proposed [60], and need to be tested in a clinical scenario.

Our calibrations and validation studies were performed using commercially available imaging tools (i.e. XRF, Echo, EM, MR); however, several steps are still required before this combination multimodality interventional image registration workflow can be implemented clinically. A U.S. Food and Drug Administration (FDA) Investigational Device Exemption (IDE) is required on the registration software, the EM tracked TEE probe and the EM tracked transendocardial injection catheter before this approach can be tested in real-time in humans. From a technical perspective, placing EM tracked coils on the TEE probe and the transendocardial interventional device is relatively straightforward. There are several examples in the field of electrophysiology and arrhythmia ablation that employ such EM tracked tools. Future steps include software risk assessment and human cardiac interventional validation studies that will test the workflow and registration capabilities using off-line image reconstructions.

5. Conclusion

The purpose of this study was to investigate the feasibility of EM tracking based multimodal image fusion for catheter based endomyocardial injections. The system fused XRF, real-time streaming 3DEcho, and MR images within an EM-tracking framework. It was able to be used in multiple modes where different combinations of modalities were available. We demonstrated that the system registration and calibrations are accurate and resulted in an efficient clinical workflow, that the entire system is accurate for catheter based injections with in-vitro experiments, and that this system can be used to accurately guide catheters and perform injections in an in-vivo scenario. Image guided catheter injection procedures present unique clinical and technical challenges, and we believe that this system provides an innovative and practical approach to solving these problems. With potential future advances in 3DEcho imaging in the form of capacitive micro-machined ultrasound transducers and improved EM sensor accuracy, multi-modal image fusion systems like the one presented here show great promise in applications where targeting accuracy in the presence of organ motion is critical and reduced radiation exposure is desired.

Figure 5.

Diagram of the EM-tracked injection catheter. The EM sensor is fixed in place at the tip the catheter, while the sharpened hypotube is able to move freely within the lumen.