When change happens: computer assistance and image guidance for minimally invasive therapy

Abstract

Computer-assisted interventions are medical procedures that rely on image guidance and computer-based systems to provide visualisation and navigation information to the clinician, when direct vision of the sites or targets to be treated is not available, during minimally invasive procedures. Recent advances in medical image acquisition and processing, accompanied by technological breakthroughs in image fusion, visualisation and display have accelerated the adoption of minimally invasive approaches for a variety of medical procedures. This Letter is intended to serve as a brief overview of available image guidance and computer-assisted technology in the context of popular minimally invasive applications, while outlining some of the limitations and challenges in the transition from laboratory to clinical care.

1. Introduction

Several decades ago, most of the conditions that required treatment via surgical intervention involved ‘cutting and slicing’ under highly invasive circumstances, allowing the surgeon to gain full access – both visual and physical – to the organs and sites to be treated. For example, the abdominal procedures involved a large incision through which the internal organ(s) were accessed and viewed, cardiac procedures were conducted via full sternotomy, cardiac arrest and extra-corporeal blood circulation and neuro-surgical procedures involved ample opening of the skull for access to the brain.

Using an abstract, possibly oversimplified view, surgery can be described as a three step process: access to the target, treatment and closure. Physicians realised that the large incisions and accompanying damage during the access stage were a significant source of complications and morbidity, sparking the clinical quest for minimally invasive therapies. Two notable changes in medicine occurred in the second half of the last century: the introduction of endoscopic imaging [1], which considerably reduced the incision size used for access; and, more recently, the establishment of interventional radiology as a clinical (surgical) sub-specialty [2], enabling treatment via catheters and/or percutaneous approaches under X-ray fluoroscopy or ultrasound (US) image guidance. Both developments reflect a shift from using direct visual feedback to relying on feedback from medical imaging.

In the current medical practice, X-rays are no longer simply used just to acquire chest radiographs, but rather to generate three-dimensional (3D) high-resolution images of internal anatomy in just seconds, using computed tomography (CT), reconstruct 3D anatomical representations in the interventional suite using cone-beam CT, or guide catheters and other devices through the vasculature under continuous two-dimensional (2D) or 3D acquisition via fluoroscopic imaging. Other medical imaging modalities have undergone significant improvement and expansion of their use. For example US imaging, typically used in 2D mode to view unborn babies, now provides 3D and four-dimensional (3D + time) imaging of dynamic structures and has become standard of care for monitoring and guiding several complex cardiac interventions, some of which rely solely on this modality for intra-cardiac visualisation and navigation. Lastly, one of the most fascinating discoveries of the late 20th century, was the ability to see inside the body by manipulating a series of magnetic fields which, following some heavy-duty complex mathematics, physics and engineering principles, resulted in high-quality images of the internal anatomy with exquisite soft tissue contrast; this modality is widely known as magnetic resonance imaging (MRI).

As a result, surgeons can use diagnostics imaging scans to plan the optimal therapy for each individual patient, rely on the video acquired via miniature cameras to visualise internal organs through small incisions, and employ real-time US, fluoroscopy or MRI to guide catheters and needles during percutaneous interventions. Such approaches have helped minimise the collateral damage associated with traditional surgery and therapy by employing medical imaging to provide an alternate visualisation to the direct vision, typically achieved at the expense of larger incisions.

Although medical imaging has enabled a variety of minimally invasive procedures, this path has not been free of several bumps along the way. The primary challenge arises due to the fact that the procedure outcome depends upon the physician's ability to mentally recreate the underlying ‘surgical scene’ based on the intra-operative images. This task is not trivial, given that the intra-operative images feature lower quality and smaller field of view compared with the pre-operative images. The goal of image guidance systems is to provide an accurate guidance to the target while avoiding critical anatomical structures. Most often this task is achieved by mapping information obtained pre-operatively – diagnostic images and/or models derived from them – to the intra-operative setting [3–5].

2. Computer-assisted intervention workflow

The typical computer-assisted intervention workflow (Fig. 1) includes several steps, each of which serves a well-identified scope: high-quality pre-operative images and anatomical models provide the bigger picture of the internal anatomy that helps the surgeon navigate from the point of access to the target to be treated, therefore serving as a road map. Surgical tools are typically instrumented with spatial localisation (i.e. tracking) sensors that provide information about the position and orientation of the surgical tools. Provided the patient anatomy is registered to the pre-operative image/model road map (typically using the surgical tracking system), the surgical tool representations can be visualised in the same coordinate system as the road map, much like a driver using a GPS navigation system to obtain a real-time positioning information along their route. However, most pre-operative images and models provide limited faith intra-operatively given their inherent pre-operative nature, rendering real-time intra-operative imaging modalities, such as real-time US, fluoroscopy or cone-beam CT, critical for accurate and precise target identification and on-target instrument positioning. Following the co-registration and fusion of all pre- and intra-operative images and tracking information, the surgeon uses the virtual environment for guidance according to the navigation-positioning paradigm: tool-to-target navigation using the pre-operative information augmented with the virtual tool representation, followed by on-target instrument positioning under real-time image guidance complemented by real-time instrument tracking.

Figure 1.

Components of computer-assisted intervention workflow for surgical interventions: pre-operative imaging, surgical instrument localisation, data integration, visualisation and information display, surgical navigation and intra-operative verification and evaluation

The multi-modality guidance and navigation information can be displayed to the surgeon via traditional 2D display screens available in the interventional suites, or overlaid onto the patient, either directly on the skin or via video augmentation (i.e. augmented reality or augmented virtuality, depending on the extent of real or virtual information present), via tracked head-mounted (stereoscopic) displays or recently developed and commercially available 3D displays [6]. During therapy delivery, the multi-modality surgical navigation images are typically verified against the direct intra-operative patient and the pre-operative images, as well as the intra-operative images registered to the patient. This step is critical in verifying and evaluating the achieved targeting accuracy of the navigation platform.

Based on the above workflow description, the following were identified as key technologies involved in the development of image guidance systems [4]:

1. Medical imaging;

2. Data visualisation;

3. Segmentation;

4. Registration;

5. Tracking and localisation systems; and

6. Software engineering.

3. Clinical applications

Two clinical disciplines are considered early adopters of image guidance systems: neurosurgery and orthopaedics [4]. The common feature shared by these two disciplines lies within the assumption that the anatomy is rigid or, in the case of neurosurgery, quasi-rigid, allowing guidance systems to readily map pre-operative information into the intra-operative setting. Following the success of the these early adopters, image guidance and navigation systems have been implemented for the abdominal procedures. These procedures can be effected laparoscopically in an efficient manner, while providing the surgeon with the visual information otherwise unavailable. As an example, imagine having access to a volumetric representation of a liver tumour either depicted using laparoscopic US imaging or extracted from pre-operative images and registered to the intra-operative patient, and superimposed onto the video feed of an endoscopic camera, showing the location and size of the tumour beneath the liver surface. Spine deformity correction procedures involving pedicle screw implantation can be planned virtually using patient-specific CT data; the result consists of detailed ‘recipes’ of the correct implants selected to instrument each vertebral level, accompanied by trajectory information that the surgeon can follow during navigation after performing the plan-to-patient registration. Similarly, the image-to-patient registration performed using the stereotactic frames in most neuro-surgical procedures can now be performed using anatomical fiducials, therefore enabling the navigation of needles into the deep brain for therapy delivery. Lastly, despite the challenge imposed by the beating heart, a blood filled environment that rendered cardiac procedures among the last surgical interventions to adopt minimally invasive techniques, these procedures have experienced significant reduction of invasiveness, gradually progressing from full sternotomy access, to mini-thoracotomies (i.e. small incisions between the ribs), to several port placements for robot-assisted intra-thoracic access and all the way to the least invasive approach: percutaneous access for navigation of catheters through the vasculature and into the heart for ablation therapy and percutaneous coronary interventions.

4. Limitations and challenges

Facilitating or impeding clinical translational? When introducing new technology into the OR, a systematic manner needs to be followed to address several concerns: identifying the real clinical need for the proposed technology; identifying the associated challenges that may impede successful implementation of the proposed technology; identifying the overall benefit on procedure outcome; and lastly, evaluating the cost effectiveness of the new technology.

Equipment and hardware: Several pieces of equipment and additional personnel are typically required for successful image-guided procedure workflows, all of which must be fitted within the constrained OR working environment and, in spite of their benefits with respect to intra-operative guidance and navigation, their compatibility with standard OR equipment is critical.

For example, if surgical tracking is required, interventional tools must be instrumented with the appropriate tracking sensors [7]. Nevertheless, most commercially available surgical instruments do not comply with this requirement, therefore raising the need to involve the medical device manufacturer in the tool design process, such that the resulting hybrid surgical instruments are compatible with the image guidance platform.

Calibration and communication: A seamless synchronisation of all signals, images and other data is also paramount, ensuring that all components of the image guidance environment (i.e. pre- and intra-operative images and virtual representations of the tracked instruments and so on) are integrated into a common coordinate system and also accurately registered to the patient.

Temporal synchronisation between the real and virtual/augmented intra-operative environments and the patient must also be achieved and maintained at all times during the procedure. While an ideal 20–30 frames per second update time is desirable, image acquisition, tracking, registration and visualisation all take time, resulting in inherent latency in the information being displayed [6, 8].

New technology footprint: The overall goal of image guidance environments is to facilitate the interventions by enhancing the direct view of the patient with additional information – either real, image-extracted or computer-generated – leading to increased safety and efficiency, while also boosting up the surgeon's confidence by decreasing risk. Unfortunately, the perceived ‘cost against benefit’ aspect is still a major barrier to new technology and, while the necessary infrastructure and team support could be leveraged, few centres around the world may be able to venture in projects of such scale, therefore limiting the wide spread of the proposed technology. Lastly, from a clinical perspective, minimal progress can be achieved without the collaborative support from clinical partners ready and willing to embrace and evaluate the proposed technology, demonstrating its clinical utility [9].

What does the surgeon expect? Addressing a real clinical need: The success of an intervention is dependent on precise knowledge of normal and pathological anatomy and physiology of the patient throughout the course of the procedure. Such knowledge must be defined and established at the start of the intervention and repeatedly reassessed during the procedure by taking into account all changes induced by both the workflow and therapy delivery. As an example, a robust visualisation paradigm will provide the necessary information during a tumour resection to limit collateral damage to adjacent structures, therefore invalidating the use of pre-operative images or models once the exposure has begun, and ‘zooming in’ more into the intra-operative visualisation of the structures adjacent to the target to be treated.

Will the system be cost effective? A decade ago this question would not have appeared in an academic article. Unfortunately, the introduction of novel technologies into the clinic is in part responsible for the increase in health care costs [10]. Considering current cost levels, the adoption of new technologies will also depend on their cost effectiveness, a constraint that only posed second tier concern to most developers of such systems. Studies addressing the cost effectiveness of image-guided navigation systems are few, and two that we are aware of were conducted in the context of orthopaedics. The first study evaluated the cost effectiveness of total knee arthroplasty navigation and did not reach clear conclusions [11], mainly because of the cost variability associated with the use of different navigation systems. The second study compared the cost effectiveness of two navigation systems for spinal surgery, and identified one of them to be more effective than the other [12].

5. What does the future hold?

Computer-assisted intervention environments enable clinicians to see beyond the surface of the patient and explore anatomy and surgical instruments otherwise hidden from direct vision by seamlessly blending diagnostic, planning and guidance information into a single visualisation environment displayed at the interventional site. For such a technology to succeed, it must be employed for the appropriate applications. Hence, while some may argue that some simple, routine procedures should continue to be performed under direct vision, as they may not benefit greatly from the use of mixed reality environments, image guidance technology would, on the other hand, make an impact in the outcome of more delicate and challenging procedures, allowing the surgeon to appreciate the real added value of the image-guided navigation environment [13].

One key technology that has not received sufficient attention in this domain is human–computer interaction (HCI). Often, the design of image guidance systems results in highly complex infrastructure for the clinical setting and the information provided overwhelms or confuses the user. In the extreme case described in [14], the interaction between the surgeon and support staff took 7 min to perform a desired selection that only requires a single mouse click.

Additional issues included the potential for information overload. In many cases, the systems display all the available information and it is up to the user to judge what is relevant and what is not. Although directly addressing HCI issues in image guidance is the exception and not the rule, some researchers have addressed these challenges for specific medical applications including the selection of visualisation strategy for percutaneous needle procedures in interventional radiology [15] and context awareness display of information to guide laparoscopic surgery [16].

The ideal system best positioned to make a significant impact on clinical outcome should provide accurate tracking and registration, robust, high-fidelity visualisation and information display, and, most importantly, fit within the traditional interventional suites with minimal interference with current workflows and typical equipment, providing smooth data transfer. An example of successful integration of several systems, signals and data sources into a clinical environment is the advanced multi-modality image-guided operating (AMIGO) suite (Fig. 2) as part of the clinical arm of the National Centre for Image Guided Therapy at the Brigham and Women's Hospital and Harvard Medical School [17].

Figure 2.

Quick glance into operating room of future — AMIGO suite at Brigham and Womens Hospital

Courtesy of Kirby G. Vosburgh, PhD – Harvard University and Brigham and Women's Hospital, Boston, MA, USA

AMIGO is a state-of-the-art facility that allows clinicians to perform interventions using multi-modality image guidance and navigation. The 5700 ft² three-room procedural suite comprises real-time imaging such as X-ray fluoroscopy and US integrated with high-fidelity tomographic imaging systems such as MRI and cone-beam CT, complemented by the latest functional imaging technologies such as positron emission tomography/CT to facilitate molecular image-guided therapy [18]. Lastly, the interventional suite also features intra-operative navigation devices such as needles, catheters and endoscopes within the body, and combines imaging and guidance with the latest techniques for identification and localisation of tumours or other abnormal anatomy and pathophysiology not available via direct view of the patient.

Finally, the broader scope of image guidance has seen technologies that aim to go beyond the minimally invasive, providing treatment in a non-invasive manner. One such technology, which has made inroads into clinical care, is based on the use of US imaging to deliver therapy, either in the form of thermal therapy or as a means to perform controlled drug delivery [19]. Although this technology has many potential benefits, it currently relies on interventional MRI to guide the therapy and is thus still a costly solution with limited widespread adoption.