Accuracy Considerations in Image-guided Cardiac Interventions: Experience and Lessons Learned

Abstract

Motivation

Medical imaging and its application in interventional guidance has revolutionized the development of minimally invasive surgical procedures leading to reduced patient trauma, fewer risks, and shorter recovery times. However, a frequently posed question with regards to an image guidance system is “how accurate is it?” On one hand, the accuracy challenge can be posed in terms of the tolerable clinical error associated with the procedure; on the other hand, accuracy is bound by the limitations of the system’s components, including modeling, patient registration, and surgical instrument tracking, all of which ultimately impact the overall targeting capabilities of the system.

Methods

While these processes are not unique to any interventional specialty, this paper discusses them in the context of two different cardiac image-guidance platforms: a model-enhanced ultrasound platform for intracardiac interventions and a prototype system for advanced visualization in image-guided cardiac ablation therapy.

Results

Pre-operative modeling techniques involving manual, semi-automatic and registration-based segmentation are discussed. The performance and limitations of clinically feasible approaches for patient registration evaluated both in the laboratory and operating room are presented. Our experience with two different magnetic tracking systems for instrument and ultrasound transducer localization is reported. Ultimately, the overall accuracy of the systems is discussed based on both in vitro and preliminary in vivo experience.

Conclusion

While clinical accuracy is specific to a particular patient and procedure and vastly dependent on the surgeon’s experience, the system’s engineering limitations are critical to determine whether the clinical requirements can be met.

1 Introduction

While many diseases can be treated via non-invasive approaches such as drug therapy, a large number of conditions require therapeutic intervention. In the case of cardiac disease, the intervention may consist of the replacement or repair of a malfunctioning valve, restoration of myocardial perfusion by inserting a stent, or grafting an obstructed vessel, or electrical isolation of tissue regions that give rise to arrhythmia. Therapeutic interventions often expose the patient to additional risks arising from the approach taken to access the target tissue, as opposed to the therapy itself. As long as patient trauma, risk of complications and recovery times can be reduced — the driving factors in the development of minimally invasive therapy approaches — surgical intervention may remain the preferred treatment.

The use of medical imaging has enabled the performance of minimally invasive procedures, providing a means for visualization and guidance during interventions where direct visual feedback could not be achieved without significant trauma: such procedures are commonly referred to as image-guided interventions (IGI). A recent review [1] provided the following statement to serve as a potential definition of IGI: “Image-guided interventions are medical procedures that use computer-based systems to provide image overlays that help the physician precisely visualize and target the surgical site.” In simpler terms, within the IGI community, an image-guided procedure is any minimally invasive intervention performed using imaging for guidance.

An image guidance system typically employs pre-operative images, surgical tracking, a data integration method, and a visualization and display platform (Fig. 1), as described by in [2]. Pre-operative data is usually in the form of high-quality computed tomography (CT) or magnetic resonance (MR) images. Optical or magnetic spatial localization systems, depending on the application, are employed to identify the position and orientation of the surgical instruments. The pre-operative plan is then registered to the patient, ultimately integrating the pre- and intra-operative images and surgical tool representations into a common coordinate system. The guidance environment is made available to the surgeon for intra-operative navigation via either head-mounted displays or overhead monitors traditionally available in the operating rooms (OR) [3].

Fig. 1.

Typical IGI workflow: pre-operative imaging and planning, surgical instrument tracking, patient registration, and surgical environment visualization and display.

While these processes have received extensive attention over the past two decades, there is one aspect of the IGI framework — accounting for differences between the pre-operative data and intra-operative reality — that has yet to be further explored. This “missing link” leads to a frequently asked question with regards to any IGI system or procedure: how accurate is it?

From a clinical stand-point, the success of an intervention is judged according to its therapeutic outcome. From an engineering perspective, navigation accuracy is constrained by the inherent limitations of the IGI system. The overall targeting error within an IGI framework is dependent on the uncertainties associated with each of the components, emphasizing the requirement that a proper overall validation of image-guided surgery systems should estimate the errors at each stage of the IGI process [4] and study their propagation through the entire workflow [5]. Following from these expert suggestions, the accuracy challenge can be posed as a series of questions: How accurate is the pre-operative modeling and planning? How accurate is the image-/model-to-patient registration? How accurate is the surgical tracking? What is the overall targeting accuracy of the IGI system? Lastly, what is the tolerable clinical error associated with the procedure?

Since the answers to the above questions are not universally applicable, but rather application and system dependent, this paper attempts to address them according to the experience with two different image guidance platforms for cardiac interventions: a model-enhanced ultrasound-assisted guidance platform developed at the Imaging Research Laboratories (Robarts Research Institute, London, ON, Canada) and a prototype system for advanced visualization for image-guided left atrial ablation therapy developed at the Biomedical Imaging Resource (Mayo Clinic College of Medicine, Rochester, MN, USA).

The former surgical guidance environment integrates magnetically tracked trans-esophageal ultrasound (US) imaging for real-time visualization, augmented with pre-operative models of the cardiac anatomy and virtual representations of the delivery instruments tracked using the NDI Aurora^™ magnetic tracking technology (Northern Digital Inc., Waterloo, ON, Canada) [6]. This system represents one of the first attempts toward bridging surgical planning with interventional guidance within a framework that allows the interpretation of the 2D intra-operative US data within the 3D context provided by the pre-operative models used for surgical planning [7].

The latter guidance platform is built on an architecture designed to integrate information from pre-operative imaging, intra-operative imaging, left atrial electro-physiology, and real-time positioning of the ablation catheter into a single user interface [8]. The system interfaces to a commercial cardiac mapping system (Carto XP, Biosense Webster Inc., Diamond Bar, CA, USA), which transmits the location of the tracked catheter to the system. While sufficiently general to be utilized in various catheter procedures, the system has been primarily tested and evaluated for the treatment of left atrial fibrillation. The user interface displays a surface-rendered, patient-specific model of the left atrium (LA) and associated pulmonary veins (PV) segmented from a pre-operative contrast-enhanced CT scan, along with points sampled on the endocardial surface with the magnetically tracked catheter. These points are incorporated into a surface-based registration algorithm, and as a result, the model-to-patient registration is continuously updated during the procedures, as additional “intra-operative” locations are sampled within the left atrium [9].

This paper provides an overview of the aforementioned accuracy considerations as learned from our in vitro and preliminary in vivo experience with these image guidance platforms. We hope this discussion will stimulate further interest from the IGI community in the joint effort to establish standardized protocols for procedure validation and accuracy assessment in image-guided interventions.

2 Clinical Accuracy Constraints

2.1 Tentative Clinical Accuracy Definition

While a formal definition is currently lacking, clinical accuracy may be tentatively defined as the maximum error that can be tolerated during an intervention without compromising therapy or leading to increased risks to the patient. Such tolerances are difficult to define, as they are procedure and patient specific. Moreover, to identify a robust measure of the clinically-imposed accuracy, in vivo experiments are required, where all variables must be closely controlled — a very challenging task for most in vivo interventions. Instead, researchers often follow the “ad-hoc” approach and make statements such as “according to the expertise of our collaborating clinicians, an overall accuracy on the order of 5 mm is considered suitable for this application [10,11].” While this may be an adequate “rule of thumb” for some applications, it may lead to significant inappropriate constraints for others.

2.2 Examples of Accuracy Expectations in the Clinic

Using the resection of a cancerous tumour as a first example, one may argue that the required clinical accuracy is dictated by the size of the smallest tumour that needs to be removed. Provided the average size of such malignant tumours is less than 5 mm, this rule of thumb may constitute a realistic constraint.

In the case of an intracardiac ablation procedure, the clinician aims to electrically isolate a region of tissue by forming a closed loop around the arrhythmia foci via either radio-frequency, cryo or thermal energy delivery. The left atrium (LA)/pulmonary vein (PV) junction is one region known to initiate arrhythmia, where a 5 mm radius loop may provide a clinically adequate goal. On the other hand, provided a smaller and more localized arrhythmia focus is identified, the size of the electrically isolating loop may be reduced, provided the ablation catheter permits such fine manipulations.

Another type of intervention in need of better guidance is the robot-assisted coronary artery bypass grafting (RA-CABG) procedure. As many as 10–20% of the robot-assisted procedures are converted to traditional open-chest surgeries [12] due to the inability to reach the target vessel with the robotic instruments, despite the port placement configuration determined based on a pre-operative plan. To avoid such situations, a better prediction of the intra-operative target vessel location is needed. Since the trocar can be repositioned in one rib space increment, a clinically-imposed accuracy on the order of one intercostal space (10–15 mm) may be deemed sufficient [13].

Lastly, based on our in vivo experience on mitral valve implantations and atrial septal defect repairs in porcine models, an overall ~ 5 mm pre- and intra-operative anatomical alignment at the target region may be sufficient for navigation. According to the navigation-positioning paradigm defined in the context of the model-enhanced US guidance platform [14], the main contribution of the pre-operative models, together with the virtual instrument representations, is to facilitate the tool-to-target navigation. The real-time positioning of the instrument onto the target — the actual therapy delivery — is confirmed under real-time imaging. While this platform relies on real-time tracked transesophageal echocardiography (TEE) for feedback, other intra-operative imaging modalities could be employed, X-ray fluoroscopy being another common approach [15,16].

The translation of clinical accuracy expectations into engineering accuracy constraints may be straight forward as in the cases described above. In other situations, it may be difficult to identify exactly when accuracy errors in the image-guidance platforms begin to affect clinical performance. For example, in percutaneous aortic valve implantation, it is difficult to estimate exactly how accurately the coronary ostia need to be localized to allow optimal stent placement. However, the accuracy requirement will dictate the engineering approach used - magnetic tracking of the TEE probe may enable the localization of the ostia to within 5 mm, but not within 0.5 mm, in which case alternative approaches need to be investigated.

3 Engineering Accuracy Considerations

3.1 Identifying Engineering Accuracy Constraints

According to Jannin’s recommendations [4], the overall system accuracy is dependent on the limitations of its integrated components. Here two cardiac image guidance platforms are discussed in terms of their integrated processes — modeling, registration, surgical tracking, and overall targeting.

3.2 Modeling Accuracy

Within the framework of an IGI environment, static and dynamic pre-operative models of the cardiac anatomy are generated by segmenting high-quality CT or MR images. The accuracy of such anatomical models can be interpreted in terms of their faithful representation in representing the subject specific organ. The accuracy of the models generated via manual segmentation (Fig. 2) is difficult to assess, as manual segmentation performed by expert clinicians is considered the gold-standard technique; nevertheless, measurements of repeatability and reproducibility can serve as a reasonable surrogate for accuracy.

Fig. 2.

Example of a manually segmented porcine heart model: Models of the left ventricle (LV), left atrium (LA), right atrium and ventricle RAV), and aorta (Ao) obtained via manual segmentation of a mid-diastole pre-operative MR image of a porcine subject.

If an atlas-based approach is used to generate the models, such as the techniques employed by Lorenzo-Valdéz et al. [17] or Wierzbicki et al. [18], the accuracy of the generated models can be evaluated against that of the gold-standard models generated via manual segmentation of the same images. Using similar techniques on human MRI data, subject-specific 3D cardiac models (Fig. 3) were generated by fitting an a priori heart model [18,19] to a mid-diastolic (MD) subject MR image [20,21]. The segmented anatomical structures were shown to be accurate within 5.0 ± 1.0 mm, 4.7 ± 0.9 mm, and 5.3 ± 1.3 mm for the left ventricle, left atrium, and the right atrium and ventricle, respectively compared to the gold-standard structures manually segmented under expert assistance. An improved atlas-based segmentation technique has recently been described by Zhu et al. [22], who proposed the development of a dynamic subject-specific model that simultaneously handles temporal dynamics and inter-subject variability with respect to cardiac shape or deformation.

Fig. 3.

Example of the registration-based segmentation approach used to generate subject-specific heart models: a) A priori average heart model of a human heart constructed from a small population of normal subjects, consisting of the left ventricle, left atrium and aorta, right atrium and ventricle, and mitral valve annulus; b) Clinical quality MR image of a new subject’s heart; c) New subject-specific heart model (consisting of the same components) instantiated by fitting the average model to the new subject’s cardiac image using non-rigid registration.

In addition, the non-rigid registration-based cardiac motion extraction technique used to animate the static models throughout the cardiac cycle was also validated using epicardial fiducial markers during both ex vivo porcine heart studies [18] and in vivo live porcine experiments [23], and it was demonstrated to provide an overall RMS target registration error of less than 3 mm [23].

For application in valvular interventions, similar modeling techniques were employed to predict the location of the mitral valve annulus; this approach led to a 2.8 mm and 3.4 mm accuracy in depicting the annulus location in diastole and systole, respectively, compared to its gold-standard location extracted from real-time 3D US images [21].

Models of the patient’s cardiac anatomy can also been generated from pre-operative CT data, as demonstrated in the context of the prototype system for advanced visualization for image guidance of left atrial ablation therapy. Using a semi-automated approach for generating models of the left atrium and pulmonary veins available within Analyze [24], seeded region growing was employed to extract the four chambers of the heart and surrounding structures. The segmented structures were then separated from the rest of the blood pool using an algorithm that searches for thin connections between user defined points in the volumetric data or on a surface rendering [25]. This process is iteratively repeated until a “clean” segmentation of the left atrium and pulmonary veins is obtained. A truth model of each subject’s left atrial anatomy was constructed from repeated expert manual tracings of each image volume; the first segmented dataset was used as reference to which the subsequent segmented datasets were registered, leading to a single gold-standard left atrial model.

The accuracy of the semi-automated segmentation approach was assessed against the truth model using a series of metrics, including the percent difference between the segmented models, the Dice overlap [26], and the distance between the boundaries of the segmented models. Overall, the semi-automatic approach was demonstrated to be repeatable within and between raters, and accurate when compared to the truth model. Mean accuracy for the semi-automated approach according to the percent difference from the gold-standard model was 3.1%, and the mean distance between the boundaries of the segmented left atria ranged from 0.3 to 3.9 mm, with an average of 2.5 mm across all raters and subjects [25]. A final segmented model from a patient dataset is shown is Fig. 4.

Fig. 4.

Anterior (left panel) and posterior (right panel) view of a patient-specific model of left atrial and pulmonary vein anatomy generated using a semi-automatic technique available within the Analyze medical image analysis software package.

3.3 Surgical Tracking Accuracy

Surgical tracking is essential for intra-operative guidance, as it provides precise knowledge about the position and orientation of the delivery instruments with respect to each other and to the anatomy at all times during the intervention. Optical tracking technology is frequently employed in image-guided therapy and generally provides a tracking accuracy on the order of 0.5 mm [27]. However, for procedures performed inside the human body, where no direct line-of-sight between the sensors mounted on the probe and the transmitting device can be achieved, magnetic tracking technologies are employed [28–31].

The model-enhanced US guidance system employs the NDI Aurora^™ magnetic tracking system. Both the surgical instruments (rigid tools or flexible catheters) and 2D TEE transducer are tracked using rigidly attached 6 degree-of-freedom (DOF) magnetic sensors. In addition to tracking the surgical instruments and the US transducer, for the in vitro accuracy studies conducted using a dynamic heart phantom, surgical targets implanted into the phantom were also tracked using 5 DOF magnetic sensors. The coordinates reported by the tracking system were treated as the ground truth locations of the surgical targets against which the overall targeting accuracy of the system was evaluated and described in section 3.5.

When instruments are tracked during image-guided surgery, their positions are generally reported with respect to world coordinates. In the context of the model-enhanced US environment, it is more important to ensure that the instruments are tracked accurately with respect to the ultrasound image, since it is the ultrasound image that ultimately determines the position of the target. Moreover, since tracked US images are used to identify the surgical target and tool locations during positioning, it is critical to assess how well the geometry of a virtual representation of a surgical tool or a surface-rendered anatomical model aligns with its geometry in the US image. According to a previous study [32], the accuracy in identifying features in the tracked US image was on the order of 2.2 ± 0.8 mm for the adult TEE probe and 2.4 ± 0.6 mm and the pediatric TEE transducer. From a qualitative perspective, the virtual objects appeared well-aligned with their cross-sectional reflection in the acquired US images, and by transposing these findings into a clinical context, the tracked US images were suitable to depict the intra-operative instruments and targets with sufficient accuracy to enable on-target tool positioning.

The prototype system for advanced visualization in cardiac ablation therapy also employs magnetic tracking technology. The therapy delivery catheter is tracked using magnetic tracking technology built into the Carto platform, providing real-time position and orientation of the ablation catheter with respect to a reference catheter. For these interventions, intra-operative ultrasound images are often acquired with an intracardiac probe. While the calibration procedures are similar to those employed when tracking the TEE transducer [32], the attachment or integration of a magnetic sensor with an intra-cardiac US catheter transducer may pose additional challenges and/or benefits and is currently under investigation.

Under the inherent limitations of the system (i.e., minimial presence of ferro-magnetic materials near the field generator, tracking the instruments within the optimal tracking volume, and quasi-static conditions - smooth motion of the tracked tools), the typical performance of the NDI Aurora 5 degree-of-freedom (DOF) sensors ranges from 0.9 mm to 1.4 mm, while the 6 DOF sensors are typically accurate within 0.9–1.6 mm over the entire tracking volume. In terms of its accuracy, the Carto system is comparable to the NDI Aurora system, featuring errors on the order of 1 mm provided the measurements are acquired in the “sweet spot” of the tracking volume. However, it is worthwhile noting that these quoted measurements were recorded under ideal conditions, which are very difficult to achieve in most laboratory or OR conditions. In spite of the remarkable accuracy in ideal environments, magnetic tracking accuracy can range between 1.5–2.5 mm depending on the environment in which the system is used [33].

Despite their transparent benefits in IGI, magnetic tracking systems do impose certain limitations when used in the OR. Fluoroscopy is commonly used in the guidance of many minimally invasive cardiac procedures and the presence of metal in the C-arm may interfere with the ability of the magnetic tracking system to track accurately. Moreover, it is difficult to position the magnetic field generator in a location that optimizes tracking accuracy without occluding the fluoroscopy image acquisition.

In these situations a possible alternative is the use of image-based tracking. A 2D/3D registration algorithm can be used to localize an object of known geometry in 3D space based on its 2D projection visible within the fluoroscopy image. The known geometry to be tracked can be either an attachment added externally to the tool, or the inherent geometry of the tool characterized by CT. The tracking accuracy is determined primarily by the geometry of the tracked object, but can also be affected by the presence of noise and partial occlusions (usually by other surgical tools) in the acquired fluoroscopic images. This technique has been employed to track the location of a TEE probe to allow registration of pre-operative CT to intra-operative ultrasound and fluoroscopy images. Tests on tracking accuracy have demonstrated that a tracking attachment with embedded sphere markers on the TEE probe (Fig. 6 a)) can be tracked with an accuracy on the order of 0.5 mm and 0.5 degrees [34]. This tracking is combined with US calibration using a Z-string phantom [35] to determine the 3D coordinates of points in the US image. Accuracy experiments using phantom (shown in Fig. 6 b)) and porcine models have demonstrated that targets seen by the US beam can be localized to within 3 mm [36].

Fig. 6.

a) Custom-built cover with implanted X-ray-visible beads designed to attach to the TEE probe to enable image-based tracking of the US transducer using fluoroscopy; b) Visualization example of a tracked US image of a cardiac phantom, achieved using fluoroscopy-based tracking of the US probe.

3.4 Registration Accuracy

Registration is critical in IGI, as it enables the integration of various co-relevant data streams, including pre- and intra-operative imaging data, the patient, and surgical tools, into a common framework. Due to their computational inefficiency, some registration algorithms may not be suitable for use in time-critical interventional applications in the OR. Instead, fast and OR-friendly registration techniques are employed.

In the effort to register the “intra-operative space” to the “pre-operative image space” during procedure guidance within the model-enhanced US platform, a clinically suitable method to augment tracked 2D intra-operative TEE images with pre-operative models was proposed; a feature-based registration that relies on the reconstructed mitral and aortic valve annuli [10,37] from the pre- and intra-operative datasets was used. The accuracy of this registration technique was assessed in a previous study involving 3D MR and US data of a healthy subject. For each cardiac surface (LV, LA, RA/RV), the root-mean-squared (RMS) distance error between the surface mapped using the feature-based registration method versus that mapped using a gold-standard iterative closest point (ICP)-based MR to US registration method previously explored and validated (Fig. 7 a, b, c)) [10] was determined. The accuracy achieved in aligning the pre-operative models with the intra-operative anatomy was on the order of 5.2 mm, 4.1 mm and 7.5 mm for the regions of the left ventricle (LV), left atrium (LA), and right atrium and ventricle (RAV), respectively, located within 10 mm from the mitral and aortic valve annuli. Furthermore, an accuracy of approximately 6–8 mm was maintained for the chambers of interest across regions located within 20–30 mm away from the mitral and aortic valves.

Fig. 7.

Distribution of the feature-based registration error displayed across the surface-rendered models of the a) left ventricle; b) left atrium and aorta, and c) right atrium and ventricle. Note the optimal registration accuracy in the region of interest (i.e. the valvular region). Visual display of the target registration error along the LAD vessel following lung deflation (d) and thoracic insufflation (e), where the gold-standard LAD is shown in white and the predicted LAD is shown in blue.

Similarly, Ma et al. [11] proposed a feature-based registration technique that relies on the alignment of the left ventricular surface and centerline of the descending aorta to fuse pre- and intra-operative data using a weighted iterative closest point (ICP) approach. While the features driving the registration are different, this technique provides comparable anatomical alignment (4–5 mm) of the pre- and intra-operative data.

Although clinically favourable, the alignment obtained using these techniques may not be sufficient to enable procedure guidance solely based on pre-operative models. To address this issue, refined guidance must be provided via intra-operative, real-time imaging, using either X-ray fluoroscopy or US. Within the framework of the model-enhanced US guidance platform [14], following tool-to-target navigation under model-assisted guidance, the on-target positioning of the surgical instrument is performed under real-time TEE imaging.

To address the RA-CABG planning application, a similar feature-based registration was employed to update the pre-operatively identified location of the target vessel to reflect the peri-operative movement of the heart. Since the LAD can only be seen clearly in the pre-operative CT image, its peri-/intra-operative location has to be inferred based on the available peri-operative data. The LAD begins at the left coronary ostia and runs toward the apex, while the mitral and aortic valves are located on either side of the starting point of the LAD. Its location was predicted via a registration algorithm that involves four features — left coronary ostia, mitral valve annulus, aortic valve annulus and left ventricular apex — all of which were easily identifiable in both modalities and sufficiently close to the target vessel to provide improved accuracy in the region of interest. The rigid-body registration driven by the above features was employed to map the pre-operative dataset to the peri-operative datasets (following left lung deflation and thoracic insufflation), predicting the LAD location at each subsequent stage [13]. According to preliminary phantom studies, this technique yielded a root-mean-squared (RMS) target registration error (TRE) of 3.5 mm at the target vessel, which is within the clinical requirements for the intended application (Fig. 7).

As part of the development of better support to facilitate the guidance of transcatheter aortic valve implantation, a pre-operative CT model of the aortic root region was registered to its corresponding intra-operative biplane TEE images. This approach was aimed at providing enhanced visualization by combining pre-operative CT with intra-operative TEE and fluroscopy of the aortic root to guide the transcatheter valve implantation. The registration technique used an ICP method to align the segmented aortic root surface from CT to manually selected outlines drawn on the short and long axis TEE views. An initial alignment was provided by two points defining the central axis selected on each image. Preliminary studies on human patient data have demonstrated a RMS TRE error of 5.0 mm [38] (Fig. 8).

Fig. 8.

Registration of aortic root model extracted from a pre-operative CT dataset to intra-operative bi-plane TEE images of the same structure.

For catheter-driven image-guided ablation therapy procedures conducted via the prototype system for advanced visualization, the goal is to register a pre-operative model of the patient’s left atrium to the intra-operative guidance environment, such that the continuously tracked catheter tip can be correctly visualized against the “interventional map” [8]. First, an estimate of the transformation between the “patient-space” and pre-operative “image-space” is determined. The registration is initialized by aligning landmark pairs and refined with a surface-based technique, where points on the endocardial left atrial surface are sampled with the tracked catheter and used for alignment with the pre-operative model. The sum of the squared distance between all sampled surface points and the pre-operative left atrial surface is used as the cost function to be minimized using the downhill simplex optimization algorithm. This approach has provided consistently adequate registration for a variety of noise levels and number of surface points in a series of simulation studies [39], leading to a 2.1 mm root-mean squared (RMS) point-cloud to surface distance error. The determined transformation is then applied to the tracked locations to map “intra-operative patient space” to “image space” and create an integrated visualization of the catheter tip in reference to the pre-operative anatomical models (Fig. 9).

Fig. 9.

Screen capture of the “intra-operative environment” mapped into the “pre-operative model space” using the estimated “subject-to-image space” transformation. In the display, note the tip of the tracked catheter, the surgical target labeled onto the pre-operative model, as well as the pre-operatively labelled and intra-operatively registered landmark pair points.

In an effort to assess registration accuracy in vivo, several canine experiments were conducted where metal clips were placed throughout the left atrium and pulmonary viens. A contrast-enhanced CT scan was acquired post clip implantation with the clips in place to serve as a gold standard. Landmark pairs and endocardial surface points were used to compute the “subject-to-image” registration transformation resulting in a mean procedural accuracy of 5.8 mm [40]. It was observed that the incorporation of the continuously tracked catheter points into the optimization algorithm decreased the overall registration error by approximately 1 mm and increased the robustness of the registration technique in terms of its sensitivity to the selected surface points.

Many challenges remain in assessing registration accuracy. With human data it is frequently difficult to select targets that can be reliably identified on all modalities. External targets can be implanted in animal models; however, the performance of geometry-dependent algorithms in animal models may not correlate well with performance on human images. Furthermore, embedded targets are generally manufactured from metal or a hard plastic. Artifacts caused by the targets make them difficult to localize accurately on TEE images, likely causing an over-estimation in TRE.

3.5 Targeting Accuracy

The targeting accuracy associated with an IGI platform provides a measure of the end-effector performance of the system: how well can a user guide a tracked instrument to a particular target identified pre-operatively and mapped to the intra-operative space, under the inherent limitations imposed by the tracking system, registration, and intra-operative visualization and image display. Such measurements require detailed experimental protocols where all variables are properly controlled, and while an in vivo evaluation is highly desired, these assessments are most often performed in phantoms.

To assess the targeting accuracy under model-enhanced US-assisted guidance while overcoming the difficulties of an in vivo intracardiac accuracy study, but still maintaining its clinical relevance, a series of experiments were modeled in the context of blinded, intracardiac interventions. A beating heart phantom was used to simulate endocardial procedures, where the sites “to be treated” represented by magnetically tracked Teflon spheres implanted in the endocardial wall of the phantom were reached transluminally, by means of a steerable catheter [41]. The targets were identified and marked on a pre-operative cardiac model which was then integrated into the intra-operative visualization environment. The “surgical task” consisted of guiding a tracked surgical instrument (i.e. a steerable catheter) to specific targets. The targeting accuracy achieved under model-enhanced US guidance was compared to that achieved under two other guidance modalities: endoscopic guidance and US image guidance. For the purpose of this study, the endoscopic guidance constituted a positive control modality that resembled guidance under direct vision. The latter guidance approach — US imaging — represented a typical modality employed clinically for cardiac interventional monitoring and guidance. In addition, to better replicate the clinical challenges associated with inaccuracies introduced during the model-to-patient registration [10, 11,42], the efficacy of model-enhanced US guidance in presence of misregistrations between the physical and virtual phantom model was also evaluated.

The tests demonstrated that model-enhanced US guidance led to more accurate targeting than US image guidance alone across all three simulated procedures, resulting in an overall targeting error comparable to the baseline accuracy measurements achieved under endoscopic guidance on the order of 1–2 mm. On the other hand, targeting error achieved under US image guidance ranged from as little as 4–5 mm to over 15 mm, where the large errors were associated with the poor navigation capabilities available to the user through the 2D imaging modality [41]. The performance evaluation under model-to-subject misregistration represented a key component of this work. While model-to patient misalignments are commonly encountered in cardiac interventions, the experiments confirmed that the real-time imaging component of the surgical guidance platform provided sufficient information to identify the correct intra-operative target location following model-assisted navigation, and compensate for the positioning error (Fig. 10), ultimately enabling consistent targeting within 1–1.5 mm [43].

Fig. 10.

In vitro evaluation of the targeting accuracy under both optimal registration, and in the presence of simulated model-to-phantom misalignments. Note the superior targeting accuracy on the order of ~ 1 mm achieved using real-time US imaging to complement the model-guided navigation.

A natural extension of this work was to initiate its translation into the clinic and illustrate its advantages over traditional real-time US image guidance, as well as provide a potential solution toward reducing the use of X-ray fluoroscopy for intracardiac catheter navigation. To date, a preliminary in vivo study comparing catheter navigation to clinically relevant sites in the right atrium of a porcine subject under model-enhanced US-assisted guidance vs. real-time US guidance was conducted. Results have reported targeting errors of less than 5 mm under the hybrid guidance environment — an improvement over US image guidance alone, which led to targeting errors as high as 30 mm.

Similar assessments were conducted to assess the guidance accuracy of the prototype system for advanced visualization in image-guided ablation therapy, using both in vitro realistic silicon rubber phantoms [44] of the left atrial anatomy, as well as in preliminary in vivo animal studies. Starting with a contrast-enhanced CT scan of a patient’s heart, the left atrial blood pool was segmented semi-automatically, resulting in a surface model shell which was then physically built using a 3D printer, and immersed in silicon rubber. After curing, the model was crushed and removed, leaving behind a hollow patient-specific left atrium phantom [45]. Surgical targets were labeled using fiducial markers implanted on the endocaridal surface of the left atrial phantom, and their correct targeting was monitored using real-time intracardiac US imaging.

An extension of this work was also demonstrated in vivo in canine models. Surgical targets inside the left atrium were labeled using metallic surgical clips visible under US, X-ray and CT imaging, delivered transluminally via a catheter. Following pre-to-intra-operative registration, the clinician guided the therapy catheter to the prescribed targets, leaving behind a burn mark in the tissue. Post-procedure assessment of the left atrium revealed optimal targeting, evaluated as the distance between the ablation burn mark and the surgical clip location [46].

4 Monitoring, Improving and Providing Accuracy Feedback to the Surgeon

As discussed, the errors associated with the various components of an IGI platform can be on the order of several mm. Such errors potentially to affect user decisions in a clinically significant way. To further improve guidance accuracy, intra-operative real-time feedback can be presented along with uncertainty distributions. Uncertainty information can be used in several ways depending on the surgical task:

• To allow the surgeon to modify tool positioning to reduce the size of the uncertainty distribution if the estimated error is too large, or if the directional error pattern is suboptimal;

• To provide guidance on the size of margins - for example, a larger patch may need to be used in cases with high uncertainty when attempting to patch an atrial/ventricular septal defect;

• To provide a suggested search area in cases where multiple attempts may be required to hit the intended target - in catheter ablation procedures, for example;

• To provide the opportunity for the clinician to request additional secondary imaging if the uncertainty is high - for example in guided percutaneous valve delivery the surgeon may request additional angiography.

As previously discussed, a typical IGI system incorporates several components, each of them exhibiting errors that may ultimately contribute to the overall uncertainty of the system. One challenge, for example, may be encountered when estimating modeling accuracy, mainly because there typically is no gold-standard patient-specific model against which a generated model can be assessed for a particular clinical case. While a feasible approach has yet to be identified to determine the impact of the uncertainties associated with the different IGI processes within the global system, a recent study by Simpson et al. [47] describes a method to propagate the uncertainty at the tip of the tracked surgical instrument from the local coordinate system to the pre-operative image space, based on the covariances of each of the involved transformations. Moreover, based on the TRE models developed by Fitzpatrick et al. [48] and more recently by Wiles et al. [49], the target registration error associated with the rigid body registration and tracked surgical instruments can be estimated and presented graphically to the surgeon using a 95% confidence ellipsoid. Similar approaches can be adopted to determine the uncertainty associated with the patient registration, estimate it at the surgical target of interest, and update the uncertainty information at the same rate as the patient registration.

While these approaches may provide uncertainty cues associated with the employed registration technique, they do not provide any measure of the absolute uncertainty with respect to the real world: is the information displayed in the guidance environment truly representing the real surgical field? Two challenges come into play in cardiac IGI: the complexity of the cardiac anatomy, and the complexity of the cardiac motion patterns. The former is very difficult to depict and reproduce in the constructed subject-specific models; the latter is difficult to integrate in the guidance environment given the performance of the registration algorithms currently employed. Hence, the main challenge is to know where to draw the fine line in terms of the trade-off between accuracy and real-time capabilities, such that it does not significantly affect clinical decision making. To address these challenges, real-time imaging is crucial, as is the need for non-rigid registration techniques, which allow for the modeling of the soft tissue in a way superior to that provided by the rigid-body approaches currently employed.

Furthermore, the incorporation of uncertainty information intra-operatively may present some cognitive challenges to the surgeon. Many cardiac procedures are performed very rapidly on a beating heart, and may involve multiple forms of image augmentation or multiple tracked tools. The addition of uncertainty information to the already over-crowded displays may result in cognitive overloading. Hence, further studies need to be conducted to investigate the need for such information, and if so, identify a simple and feasible approach to optimize such displays.

5 Summary and Future Challenges

In summary, clinical accuracy cannot be easily formulated or assessed, as it is procedure, patient, and surgeon dependent. Hence methods to better estimate clinical accuracy needs are currently being studied. The engineering limitations of the system also need to be studied by considering the error introduced by each of the integrated components, as well as estimating their compounded contribution to the uncertainty of the global system. Therefore, the accuracy considerations associated with image-guidance intervention systems need to be approached from two different perspectives: the clinical accuracy requirements need to be identified and then evaluated against the engineering limitations imposed by the system and its integrated components.

A more critical performance measure of an IGI system is whether it can improve clinical outcomes. Three important factors need to be addressed to evaluate the impact of these systems: 1) perception studies will help understand how to optimally fuse real images and computer generated data to ensure their correct registration in the mind of the user; 2) usability studies will help identify the direct benefits to the user provided by different environments, such as augmented and virtual reality environments, in comparison to other visualization approaches; and 3) the development of standardized protocols for image-guided intervention validation and accuracy assessment, similar to the ongoing efforts of the Retrospective Image Registration Evaluation (RIRE) Project initiated by Dr. J. Michael Fitzpatrick (Vanderbilt University, Nashville, TN, USA) will help ensure consistent evaluation of the developed image guidance platforms.

One may infer that such guidance environments will have a greater impact on challenging procedures than on routine ones. Moreover, it is also likely that they would have the greatest impact if they became standard OR tools for routine procedures, reducing OR time, decreasing patient trauma, and streamlining data management. Demonstrably effective systems are likely to be well-engineered, with accurate tracking, robust visualization and convincing displays that are seamlessly integrated within the standard OR, providing smooth data transfer and imposing minimal interference with the traditional clinical workflow.

To meet these goals, the system must be developed and evaluated in collaboration among engineers and clinicians to ultimately answer questions not just regarding accuracy, but rather overall value as a clinical tool. Without a robust and convincing implementation, potential end-users may develop a bias against the system and lose interest in further development and testing [50].

Fig. 5.

a) The Carto XP magnetic tracking system — note the field generator and a typical ablation catheter; b) the NDI Aurora magnetic tracking system and a typical 6DOF sensor that can be integrated with the instrument to be tracked.