Image Fusion During Vascular and Nonvascular Image-Guided Procedures

Abstract

Image fusion may be useful in any procedure where previous imaging such as positron emission tomography, magnetic resonance imaging, or contrast-enhanced computed tomography (CT) defines information that is referenced to the procedural imaging, to the needle or catheter, or to an ultrasound transducer. Fusion of prior and intraoperative imaging provides real-time feedback on tumor location or margin, metabolic activity, device location, or vessel location. Multimodality image fusion in interventional radiology was initially introduced for biopsies and ablations, especially for lesions only seen on arterial phase CT, magnetic resonance imaging, or positron emission tomography/CT but has more recently been applied to other vascular and nonvascular procedures.

Two different types of platforms are commonly used for image fusion and navigation: (1) electromagnetic tracking and (2) cone-beam CT. Both technologies would be reviewed as well as their strengths and weaknesses, indications, when to use one vs the other, tips and guidance to streamline use, and early evidence defining clinical benefits of these rapidly evolving, commercially available and emerging techniques.

Image registration and fusion are most often used in nuclear medicine with the combination of functional or metabolic imaging (positron emission tomography [PET]) with morphologic or anatomical imaging (computed tomography [CT]).^1,2 Detailed spatial or biological information provided by more advanced modalities such as magnetic resonance imaging (MRI) or PET may improve accuracy for therapeutic interventions, as seen in radiation oncology and interventional radiology (IR).³ In the absence of real-time fusion capabilities, information from preprocedural imaging modalities was integrated mentally by the operator using anatomical landmarks and estimation. This guesswork is traded for precision with the availability of fusion navigation technologies, and multimodality imaging integration has expanded to the interventional suite, initially used for needle-based procedures and more recently for vascular interventions.⁴

A variety of commercially available software solutions for image registration or fusion in the interventional suite are based upon the backbone of either ultrasound (US) (electromagnetic [EM] tracking) or fluoroscopy (cone-beam CT [CBCT] navigation) as real-time imaging. Alternatively, prior imaging can also be referenced to “step-and-shoot” interventional CT alone. Image fusion and registration is described as used in clinical cases.

Definitions

Terms like registration, fusion, and codisplay are often used interchangeably, however their scientific meanings slightly differ.^2,5 Image registration has rigid and deformable flavors, can be achieved by matching either features (eg, landmarks, vessels, and organ surfaces) or image intensity, and often requires several steps to spatially match 2 imaging sets. The first step is reformatting one of the image volumes (secondary) to match the other (primary) imaging volume.² The dynamic range of signal that can be displayed in 1 voxel defines a 3-dimensional (3D) imaging set. The voxels of the secondary imaging set are scaled accordingly to match the voxels of the primary imaging set. The second step consists of transforming the secondary reformatted images to ideally spatially match them with the primary imaging set.² In rigid transformation, the distance between voxels is rotated or translated but remains constant within an image.² With elastic (or deformable or warped) transformation, the distance between 2 voxels can be modified or “stretched” in certain subsections of the image. Registration and fusion can be automated, semiautomated, or manual, with various degrees of user interaction with the process.² However, by definition, elastic registration warps and deforms the original data, and may introduce wrong assumptions, so one must be careful to verify the apparent accuracy of the registration for the organ or area of interest when using elastic methods. Generally speaking, automatic algorithms are often organ dependent and modality dependent. There are no universal algorithms that work on everything. They may fail for various reasons such as image noise and low resolution. Evaluating the accuracy of registration is the final step and can be performed visually or in an automated fashion by examining position of fiducial patches or anatomical landmarks, as well as the edge of the organ of interest. Image fusion (or codisplay) consists of superimposing the registered images to view them as one, (or side by side but linked), however each imaging set may be assigned a different color look-up table.^2,6,7

Indications for the Procedure

Image fusion may be considered in any case where preprocedural imaging such as contrast-enhanced (CE) CT, MRI, PET/CT, or dual-phase CBCT provides information not available with conventional image-guidance modalities, that is, fluoroscopy, US, or CT.^6,8–10 With the device and the target displayed in the same image in real time, image fusion may improve device positioning and accuracy of treatment if a lesion is only visible on MR, CE-CT, or PET/CT or demonstrates heterogeneous PET uptake.^8,9,11 Recent generation image fusion software may have the additional capabilities of tumor segmentation and complex ablation planning.^5,6,12–15 The tumor margin is often semimanually segmented with an edge-detection algorithm and may be propagated to adjacent slices. Planned treatment volumes are defined. The number of ablation probes required for complete coverage can be planned.^13,15 After ablation, the treatment margin can be depicted in 3D and the postablation scan is fused to the tumor-defining scan to ensure adequate tumor coverage.^5,14 With the advent of CBCT navigation, image fusion can also be performed for vascular cases.^6,16–18 Indeed, many physicians are using image fusion during aortic stent graft deployment, uterine artery embolization, and chemoembolization, as well as TIPS, abscess drainage, and pain procedures in addition to other complex procedures.^16,18–20

Equipment Required

The equipment required depends upon the technique used to achieve image fusion. There are 2 basic approaches or platforms for image fusion commonly used in IR, each with different software and hardware requirements. A workstation communicating with the CT, US, or CBCT is often where the images are imported, coregistered, and displayed.

Image Fusion

Certain software can perform image fusion using rigid or elastic image registration without navigation. This technique does not require any additional equipment except for the software.^2,21–24 This offline static registration may be helpful for diagnosis, training, or preprocedural treatment planning, as a 1-time reference without real-time updating, but fusion software alone needs to be linked to the procedure to take full advantage of the real-time feedback.^7,22

EM Tracking

An EM tracking workstation, a field generator, fiducial patches, and “smart devices” (equipped with sensors) are required for EM tracking.²⁵ For real-time navigation, a tracked US is optional. Some tracking vendors combine an US or CT with their workstation,¹³ whereas others integrate to varying degrees the tracking software directly into the conventional US system, with some vendors having sensor coils manufactured directly into the US transducers. Some use sensor coils that are disposable and integrated inside needle tips, whereas others use a nondisposable stylet that resides inside the inner needle shaft. EM tracking can also be used as an independent system with CT (with or without US).

The field generator creates very weak differential magnetic fields, typically within a 50 cm³ work volume, however, it may vary depending on the manufacturer.⁶ Sensor coils within a magnetic field induce an electrical signal that is proportional to the strength (and thus the direction) of the magnetic field, as per Faraday’s law. This varying electrical signal enables the tracking workstation to define the sensor coil’s position. In a very simplistic manner, the field generator is similar to a satellite, the tracked device is the car with the GPS receiver and the preprocedure or intraprocedure imaging is the road map. Several devices equipped with sensor coils in the magnetic field can be tracked simultaneously.^6,10,26 Fiducial patches are equipped with sensor coils and visible on EM tracking. They are used to register procedural imaging in the virtual magnetic space to enable navigation.²⁵ If imaging can be obtained on the day of the procedure with fiducial patches, then semiautomated registration of that imaging data set can be performed directly to the virtual space. If that is not possible, previous imaging can be imported into the workstation and (manually) registered to the procedural imaging CT, and therefore to the virtual magnetic space.^6,13,25

CBCT

Flat-detector C-arms can be programmed to perform CBCT. With appropriate software added on, CBCT can be used for navigation and multimodality image fusion. No additional hardware or disposables are required.^27,28

CBCT consists of 3D CT-like data set generated by the rotation of a flat-detector C-arm around the patient. The work volume ranges from 23–46 cm³ and the acquisition time varies from 5–15 seconds, depending on the vendor. Using a Cartesian coordinate system, the CBCT is inherently registered to fluoroscopy.^27,28 Previous imaging can be manually overlaid on the procedural CBCT and used for navigation with real-time updating of this “virtual plan” with each change in needle or catheter location.⁸ Depending upon the vendor and software, real-time updating of the 2D or 3D registration also is possible with each change in magnification and table motion.⁸

Procedural Steps

Image Fusion Diagnostic Imaging Software: Conventional Workstation Approach

A procedural scan is obtained and manually fused with previous imaging using diagnostic fusion software. The operator can determine the target on procedural scan, but it is defined by the preprocedural imaging.^22,24 As opposed to other fusion IR solutions, this standard retrospective software method may not offer any real-time feedback, only a 1-time reference to preprocedural imaging.⁹ This “conventional software approach” can be time consuming; however, it does not require any additional hardware or disposables and can be performed on a conventional CT^6,7 console. Preliminary reports on rigid and nonrigid image fusion software during procedures are encouraging.²⁴ However nonrigid image fusion can be very time consuming, taking up to 20 minutes for the first registration and 10 minutes for each additional registration as every repeat scan during the procedure has to be registered separately.^7,24 These times vary dramatically with the operating system, processing power, GPU, graphics cards, and other solutions such as computing clusters.

EM Tracking

Once it is determined that a case would be performed using EM navigation, previous imaging should be transferred to the EM workstation. ^9,29,30 In certain cases, the “prior” imaging data set can be obtained immediately prior to or at the beginning of the procedure. In any case, the fiducial patches should be placed near the area of interest prior to image acquisition used for registration into the virtual space.²⁹ The operator should place the fiducial patches keeping in mind that their position should be in the EM generator’s field of view (within the work volume) without hindering a potential US or needle access window.⁶ It is noteworthy that the procedure does not have to be performed in the same physical location as the imaging as long as the fiducial patches are not moved relative to the target. For instance, a PET/CT can be obtained with fiducial patches that are dynamic references; the patient is transferred to another room where the procedure is performed. A large area should be prepped and draped, including the EM generator.^10,13,26 Procedural imaging is imported into the EM workstation and registered to the magnetic space automatically (with semiautomatic detection of the fiducial patches on procedural imaging).²⁵ If 3D imaging was acquired prior to the procedure day, it can be manually registered to the procedural CT using rigid registration and anatomical landmarks. The system provides the operator with a registration error normally by calculating the root mean square of the difference between the position of the fiducial patches on imaging and in the EM field. The registration error signifies how well the images match and ideally should be less than 2 or 3 mm.³¹

Once the registration is completed, the target and entry point can be chosen. The device is inserted using EM tracking navigation (±US fusion). Once the target is reached, confirmation imaging is obtained. Registration is performed once at the beginning of the procedure, and needle locations can be factored into an iterative treatment plan by integrating intraprocedural imaging. For composite ablations or multiple sampling in biopsies, the operator can choose several targets (Fig. 1) and even update the targets depending upon the location of earlier needles.10,26,30 The EM workstation can correct or patient motion without repeating registration when a dynamic reference patch (or fiducial patch) is used and the patient motion is not too complex. Certain EM systems also have respiratory gating.⁶

Figure 1.

Ablation planning case: (A) depicts coronal, axial, and sagittal views of a contrast-enhanced CT. The tumor and safety margin have been segmented (orange circles). The ablations zones required for complete coverage are also seen (blue circles). The target position for the ablation probe is depicted as the red cross. The virtual probe is seen as the purple line. As the operator advances the tracked probe, the “virtual” probe position adjusts in the software. (B) After the first ablation is complete, the ablation planning software updates the treated areas (as per manufacturer specifications) shown as the purple circle. If applicable, it also adjusts the position of subsequent probes seen as the red and purple crosses. The imaging can also be fused with ultrasound for real-time imaging guidance as seen on the left bottom screen. (Color version of figure is available online.)

EM tracking may be helpful in many clinical scenarios and sometimes even enables procedures otherwise deemed not possible or difficult to be performed.¹⁰ Indeed, a phantom study demonstrated that fewer passes (1.1 vs 3.6) and lesser time (9 minutes vs 14 minutes) were required to reach small targets in the lung when using EM tracking vs conventional CT.³² EM tracking provided crucial information for the completion of certain procedures in about half of the cases.¹⁰ Use of US and MRI fusion with EM tracking has been documented to roughly double the cancer detection rate for prostate biopsy.^33–36 However, metal interactions can interfere with EM tracking accuracy.³¹ Moreover, ‘smart’ versions of certain devices are not always available. Indeed tracked introducers or microwave and cryoablation probes are not available with all systems. In composite ablations, the ablation needle may be introduced in tandem, parallel to a tracked needle, or coaxially after removal of an inner tracked stylet. However, either method is not ideal for repositioning of needles, which may require insertion of 2 needles, and is not a guarantee of accuracy as the treatment needle is not the actual treatment needle, which may deviate. Alternatively, a tracked button-guiding device for the cluster cool tip with a custom tracked inner retractable needle in between the 3 cluster needles has been used.

US is often used with EM tracking, but US is of limited value in lung parenchymal interventions. If a pneumothorax develops, displacing the lesion, the operator might be unaware unless conventional imaging is repeated or the patient becomes symptomatic.⁶ Applications of EM tracking in vascular cases have been limited to due equipment availability. Integration of sensor coils in wires or catheters has been done, but modifies properties such as trackability and torquability.³¹

CBCT Navigation

Image fusion and navigation have recently become available using CBCT, which has several advantages. Firstly, flat-detector C-arms are commonly capable of CBCT. As equipment is upgraded and the technology evolves, flat detectors and CBCT navigation will be available to an increasing fraction of interventional radiologists.^6,8,27 Secondly, CBCT navigation requires additional software but not disposables or additional hardware.⁶ A compatible workstation or table upgrade could be required however (check with vendors). Thirdly, CBCT navigation enables operators to perform procedures in the interventional suite instead of a predominantly diagnostic CT. In facilities unequipped with dedicated interventional CT, this may help streamline workflow.³⁷ Finally, compared with fusion methods based on EM tracking, CBCT navigation and fusion may be used for nonvascular and vascular procedures.^19,28

Currently, regardless of the vendor, image fusion with CBCT navigation is based on rigid registration. Streamlining the process with automated registration and organ edge detection as well as elastic registration is work in progress for CBCT and EM tracking methods.

CBCT imaging has low signal-to-noise ratio. Patient setup and technologists’ training are key to reduce artifacts and improve image quality. The patient’s arms, radiopaque tubing, and EKG leads should be out of the field of view. For a hepatic procedure, the patient’s arm should be positioned safely over the thorax, and the radiopaque tubing should not run along the abdomen. Operators should keep in mind that the CBCT field of view is smaller than the fluoroscopic field of view. As technicians are setting up to acquire a CBCT image, the region of interest should be in the center of the fluoroscopic image otherwise key portions would not be included in the CBCT image. For example, if the dome of the liver is at the edge of the fluoroscopic image, it would not be seen on CBCT images. Including the skin entry point (in cases of percutaneous procedures) is a key pitfall that may require repeat imaging.

Previous imaging is imported from picture archiving and communication system into the workstation prior to the procedure. Once a CBCT image is acquired, the operator (or adept technologist) manually performs image fusion of preprocedural imaging and CBCT (this is 3D to 3D registration), however the CBCT is automatically registered to fluoroscopy (this is 3D to 2D registration). As additional imaging is acquired, the image fusion is automatically updated on the new CBCT and does not need to be repeated unless there is significant patient motion.^8,38

CBCT Image Fusion for Nonvascular Procedures

In general, the operator should match the organ of interest on both imaging sets, with attention to the area of interest or the organ edge nearest to the area of interest. Aligning adjacent structures or bones is less relevant, and often is a distraction and should be ignored. During procedures, patient positioning is variable whereas diagnostic imaging is acquired in supine position, often in deep inspiration (CT) or shallow breathing (PET/CT). The differences in patient positioning and breathing result in complex organ distortion that cannot be corrected with rigid registration. Concentrating on the organ in question maximizes the accuracy of the registration process. Moreover, ablation probes and needles can themselves result in organ shifts. Case series demonstrate that image fusion using CBCT navigation for biopsies and ablations is feasible and safe (Fig. 2).^8,12,28,37 Publications have shown increased sensitivity and specificity of CBCT-guided renal biopsies with image fusion as opposed to historical conventional CT guidance.³⁹ Moreover, other groups have simulated absorbed radiation dose using CBCT vs conventional CT dose during phantom lung interventions and demonstrated that CBCT had less radiation than conventional CT.⁴⁰

Figure 2.

Workflow of image fusion for CBCT. Patient with von Hippel-Lindau disease, multiple lesions and a new enhancing renal lesion are seen on MRI (A) the previous MRI is imported into the workstation and the lesion is segmented. (B-C) the MR is registered to the procedural CBCT. (D) The number of probes and their trajectory is planned by the operator, taking advantages of the information available on CBCT and preprocedural MRI. In this case, 2 lesions are seen (blue and green circles). The 3D reconstruction of the CBCT is displayed. Two probes were needed to ensure complete coverage of the inferior lesion. The data can be examined in the axial, coronal, and sagittal planes as well. (E) The operator advances the ablation following the virtual planned path displayed on live fluoroscopy. Both needles are planned on the same CBCT; however during navigation 1 virtual path is displayed at a time. The segmented tumor is seen. (F) Once the target is reached, a CBCT image is obtained to confirm needle positioning. Registration with preprocedural imaging is automatically updated. (G-H) After cryoablation, the iceball is segmented. The post-CBCT is registered to the pre-CBCTand the ablation is examined to ensure complete coverage. (Color version of figure is available online.)

CBCT Image Fusion for Vascular Procedures

Technical aspects of image fusion vary depending on the procedure. For transarterial chemoembolization, one or several enhanced CBCTs are obtained and fused with fluoroscopy.^18,41 CBCT images can be obtained in different phases (arterial, venous, and delayed). Dual-phase CBCT, an arterial and venous phase CBCT (acquired with a double rotation of C-arm around the patient), is available on certain equipment.¹⁹ Tumor segmentation and feeding vessels are identified based on the CBCT image in a semiautomated or manual fashion. A virtual path is mapped from the current catheter position to the desired feeding vessel.^18,19 As the CBCT image is automatically registered with fluoroscopy based on Cartesian coordinate system, the virtual path is displayed on the fluoroscopic image like a dynamic 3D road map. Depending on the vendor, a simple path or a maximum intensity projection reconstruction of the vessel can be overlaid on the fluoroscopic image.¹⁹ For diagnostic ability, some groups have examined the sensitivity, specificity, and accuracy of CE-CBCT to detect lesions, as opposed to digital subtracted angiography, CE-CT, and MR. In a series of 28 patients with 33 small hepatocellular carcinomas, all seen on CE-CT, CE-CBCT detected 93.9% of the lesions whereas digital subtracted angiography detected none.¹⁸ In those cases, CE-CBCT enabled more selective or more confident transarterial chemoembolization of these lesions.¹⁸ More recently, a prospective randomized study of 30 patients with hepatocellular carcinoma revealed no difference in lesion detection by 3 independent blinded reviewers when comparing dual-phase CBCT and multidetector CT.⁴² CE-CBCT detected 93.9% of lesions seen on CE-MRI.⁴³ Moreover, CBCT image fusion, tumor segmentation, and automated vessel detection were significantly more sensitive in determining tumor arterial blood supply during transarterial chemoembolization, with less interobserver variability (Fig. 3).¹⁹

Figure 3.

CBCT image workflow for transarterial chemoembolization; dual-phase CBCT, with arterial and venous phase acquired from the C-arm spinning back and forth or twice around the patient. (A) The lesions are segmented from the venous phase. (B) The arterial supply to the segmented lesions is extracted from the arterial CBCT. This process may be semiautomated or manual. A virtual path is mapped from the catheter position to the desired vessels. (C and D) The vascular map and virtual path are overlaid on the live fluoroscopy. The image may be adjusted to account for magnification, or table or C-arm movement. (Color version of figure is available online.)

For visceral artery stenting (Fig. 4), stent graft deployment, uterine artery embolization,⁴⁴ or TIPS, a slightly different workflow is used. Preprocedural CT angiography (CTA) or MR angiography (MRA) (CT or MR venography) is imported onto a dedicated 3D work-station.¹⁶ To complete the registration with live fluoroscopy, a low-dose CBCT without injection of iodinated contrast is performed after the patient is anesthetized, before sterile preparation.³⁸

Figure 4.

Renal artery stenting CBCT image fusion workflow. (A) Catheterization of the left renal artery using the volume-rendering MRA overlay. The tip of the catheter is in the ostium of the stenosed renal artery. (B) Placement of the stent under VR MRA overlay to cover the entire lesion. No contrast was used till this part of the procedure. (C) To confirm stent position in relation to stenosis prior to final deployment, 3 cc of contrast was injected. (Color version of figure is available online.)

While the patient is being prepped, the operator can manually fuse the preacquired MRA or CTA with the intraoperative CBCT using landmarks such as pelvic bone structures and calcifications on the aortic wall or target vessels or both. The previous CTA or MRA volume is brought into the same coordinate space as the live fluoroscopy, thus allowing fluoroscopic navigation using preprocedure CTA or MRA. As such, stent graft positioning and deployment and target vessel catheterization can be performed with little or no contrast (Fig. 5).³⁸ Publications comparing historical cohorts with CBCT image fusion for stent graft deployment have demonstrated significant reduction of contrast¹⁶ and one case report describes thoracic stent graft deployment without any contrast.³⁸ CBCT image fusion also enabled treatment of type II endoleaks and other complex interventions.^45,46 Prospective randomized studies are lacking but underway.

Figure 5.

Endovascular stent graft deployment with navigation. (A) Coronal view of contrast-enhanced CT depicting a large infrarenal aortic aneurysm with a short neck. (B) Multiplanar volume rendering of the contrast-enhanced CT. (C) Axial and coronal views of the contrast-enhanced CT demonstrating the infrarenal aortic aneurysm. (D) Axial and coronal views of the nonenhanced cone-beam CT demonstrating the infrarenal aortic aneurysm. These images match C. (E) Axial and coronal views of the fused cone-beam CT and contrast-enhanced CT overlaid on top of each other. The CBCT has a bluish tint. (F) Once registration and fusion are complete, the contrast-enhanced CT can be displayed on top of live fluoroscopy image, as shown. Markers (arrows) are placed over the renal arteries. The image shows the stent partially deployed. (G) Fluoroscopy image with CT-MPR overlay during stent deployment. MPR, multiplanar reconstruction. (Color version of figure is available online.)

Technical Challenges

The accuracy of the image fusion relies entirely on the robustness of the registration. For example, rigid registration of a PET/CT to CBCT for a PET-guided biopsy comprises the errors of registering the diagnostic PET and CT in addition to the error of registering the CBCT to the diagnostic CT.^4,23,47 Accurate image fusion between real-time and preprocedural images can be extremely difficult because of organ motion and deformation during the procedure. Patient positioning, extrinsic compression by pneumothorax, hematoma, or the device can modify the shape of the organ and interfere with registration. PET Dicom may introduce transient hurdles for data transfer as well. To compensate for such complex deformation, intraprocedural feedback and intricate computation are required to match the images, which is hard to accomplish during the procedure.

Conclusions

Image fusion allows combination of multimodality, preprocedural, and real-time imaging data sets to improve target delineation, device localization, and treatment planning and execution to enable certain procedures or potentially improve outcomes. Several platforms are available using EM tracking or CBCT navigation. Image fusion can be useful in a wide variety of cases, ranging from biopsies and ablations, to stent graft deployment. Image fusion can improve targeting for biopsies and tumor coverage during ablations, and reduce contrast and radiation dose during embolization or stent deployment procedures.

No prospective randomized study has compared the different navigation technologies but reported accuracies appear similar to each other, and may be superior to conventional image guidance.

Navigation and image fusion technologies are becoming more ubiquitous. As technologies improve, image fusion will become more streamlined and accurate. Operators will be able to utilize all the information that has long been available for diagnostic purposes, with fewer throughput hurdles. In addition, the critical importance of fusion guidance will become more apparent with the increasing role of IR in biopsy and tumor characterization as requisite for personalized therapies. Fusion guidance is becoming easier to use and more widely available, with better defined benefits. Fusion guidance can be like turning your headlights on while driving at night down a dark and windy mountain road.