Navigation and visualisation with HoloLens in endovascular aortic repair

Abstract

Introduction

Endovascular aortic repair (EVAR) is a minimal-invasive technique that prevents life-threatening rupture in patients with aortic pathologies by implantation of an endoluminal stent graft. During the endovascular procedure, device navigation is currently performed by fluoroscopy in combination with digital subtraction angiography. This study presents the current iterative process of biomedical engineering within the disruptive interdisciplinary project Nav EVAR, which includes advanced navigation, image techniques and augmented reality with the aim of reducing side effects (namely radiation exposure and contrast agent administration) and optimising visualisation during EVAR procedures. This article describes the current prototype developed in this project and the experiments conducted to evaluate it.

Methods

The current approach of the Nav EVAR project is guiding EVAR interventions in real-time with an electromagnetic tracking system after attaching a sensor on the catheter tip and displaying this information on Microsoft HoloLens glasses. This augmented reality technology enables the visualisation of virtual objects superimposed on the real environment. These virtual objects include three-dimensional (3D) objects (namely 3D models of the skin and vascular structures) and two-dimensional (2D) objects [namely orthogonal views of computed tomography (CT) angiograms, 2D images of 3D vascular models, and 2D images of a new virtual angioscopy whose appearance of the vessel wall follows that shown in ex vivo and in vivo angioscopies]. Specific external markers were designed to be used as landmarks in the registration process to map the tracking data and radiological data into a common space. In addition, the use of real-time 3D ultrasound (US) is also under evaluation in the Nav EVAR project for guiding endovascular tools and updating navigation with intraoperative imaging. US volumes are streamed from the US system to HoloLens and visualised at a certain distance from the probe by tracking augmented reality markers. A human model torso that includes a 3D printed patient-specific aortic model was built to provide a realistic test environment for evaluation of technical components in the Nav EVAR project. The solutions presented in this study were tested by using an US training model and the aortic-aneurysm phantom.

Results

During the navigation of the catheter tip in the US training model, the 3D models of the phantom surface and vessels were visualised on HoloLens. In addition, a virtual angioscopy was also built from a CT scan of the aortic-aneurysm phantom. The external markers designed for this study were visible in the CT scan and the electromagnetically tracked pointer fitted in each marker hole. US volumes of the US training model were sent from the US system to HoloLens in order to display them, showing a latency of 259±86 ms (mean±standard deviation).

Conclusion

The Nav EVAR project tackles the problem of radiation exposure and contrast agent administration during EVAR interventions by using a multidisciplinary approach to guide the endovascular tools. Its current state presents several limitations such as the rigid alignment between preoperative data and the simulated patient. Nevertheless, the techniques shown in this study in combination with fibre Bragg gratings and optical coherence tomography are a promising approach to overcome the problems of EVAR interventions.

Abbreviations: 2D, two-dimensional; 3D, three-dimensional; CBCT, cone-beam computed tomography; CT, computed tomography; DoF, degrees of freedom; DSA, digital subtraction angiography; EVAR, endovascular aortic repair; FAST, focussed assessment with sonography for trauma; FBG, fibre Bragg grating; IVUS, intravascular ultrasound; OCT, optical coherence tomography; OST, optical see-through; PLA, polylactic acid; TCP/IP, transmission control protocol/internet protocol; US, ultrasound

Introduction

Aortic aneurysm is defined as an enlargement of the aorta greater than 1.5 times the normal size, and is a relatively common and potentially lethal disease. The prevalence in Europe is estimated to be up to 5% and most are located at the abdominal segment of the aorta (abdominal aortic aneurysm). Stress (defined as force per unit area) on the aneurysm wall beyond the wall strength produces rupture [1], which causes life-threatening bleeding with mortality rates greater than 80% [2]. Therefore, the treatment is ideally performed before the event of rupture and currently elective aneurysm repair is generally recommended in diameters of 5.5 cm and over, or progressive aneurysm growth [3]. Endovascular aortic repair (EVAR) is a minimal-invasive technique that excludes the risk of rupture by implantation of an endoluminal (covered) stent graft into the aortic wall using introducer devices from the femoral or brachial arteries. EVAR shows lower perioperative morbidity and mortality, fast recovery [3], and is the standard procedure, if feasible. During the endovascular procedure, device navigation is currently performed by visualisation of the vessel anatomy using fluoroscopy in combination with digital subtraction angiography (DSA), applying radiation exposure and contrast agent. The application of contrast agents for endovascular navigation is a significant risk factor for kidney injury during these procedures. In this context, the proceduralist has to be aware that a significant proportion of patients with abdominal aneurysm bears an underlying renal impairment previous to surgery and the administration of contrast agent may cause deterioration of the renal function [4]. Acute kidney injury is a relevant post-operative complication following EVAR and is associated with increased mortality [5]. Next to the issue of contrast drugs, patients, as well as medical staff, are exposed to significant radiation doses during endovascular procedures [6]. Especially DSA runs of the abdomen and pelvis, used intraoperatively to visualise the vessel anatomy for adequate navigation and stent deployment, contribute to a significant radiation exposure during EVAR procedures. The average examination time with exposure to radiation in an EVAR procedure has been documented as being 22.6 min [7]. Providing care to patients with complex aneurysms using the chimney technique, or fenestrated and branched stent grafts has the negative side effect of increased radiation exposure [8], [9]. Deterministic effects of ionising radiation cause skin erythema (2 Gy) or hair loss (3 Gy) of the patient, which arise by exceeding a certain dose [10]. A previous study estimated deterministic radiation effects to patients in 29% of EVAR procedures [11]. Stochastic effects of ionising radiation due to repetitive exposure of the medical staff may cause cancer or cataracts, even years or decades after the exposure [12].

Another limitation of acquiring two-dimensional (2D) fluoroscopic images to obtain the position of endovascular tools such as catheters, guide wires, sheaths and stent grafts related to the patient´s anatomy is the lack of depth information. Navigation systems overcome this problem by superimposing the current position of endovascular tools in the operating room (information obtained with electromagnetic tracking systems) on preoperative computed tomography (CT) angiograms and anatomical three-dimensional (3D) models. A sensor (specifically, coils) on the tip of the endovascular device measures the magnetic field generated by a transmitter and the electromagnetic tracking system estimates the sensor position and orientation based on the theoretical knowledge of the transmitted magnetic field. This technology does not require line-of-sight as in the case of optical tracking systems but is potentially affected by ferromagnetic objects and other electronic devices, which distort the reference magnetic field [13]. An average error of 4.2 mm was obtained when tracking a guide wire in nonrigid scenarios (animal study) [14], but the accuracy is not uniform throughout the limited working volume of the electromagnetic tracking system.

Ultrasound (US) imaging is a common screening technique for abdominal aortic aneurysms and is used for guiding the femoral access in percutaneous EVAR procedures to reduce access-related complications [15], [16]. This relatively low-cost and portable technology requires the physician to identify which section of the patient’s anatomy is currently displayed and then to mentally reconstruct the 3D anatomy from 2D images [17]. In addition, out-of-plane motion owing to patient respiration or transducer movements changes the current slice of the patient’s anatomy. There is a growing interest in using real-time 3D US to deal with the aforesaid problems [18]. Delivery of fenestrated stent grafts could be guided with real-time 3D US as the tips of guide wires can be tracked in 3D [19]. Its combination with preoperative CT data, which addresses the orientation problem of 3D US with an intuitive and understandable way of displaying these volumes [20], could be useful for navigation in EVAR procedures.

In navigation systems, preoperative 3D studies and anatomical 3D models are commonly displayed on standard 2D screens. HoloLens glasses (Microsoft, Redmond, WA, USA) are an optical see-through (OST) head-mounted computer that renders virtual 3D objects superimposed on the real environment using a stereoscopic display. This wireless augmented reality technology enables a more intuitive visualisation of 3D content than on standard 2D screens. This is carried out without losing information of the real world as in the case of virtual reality. Augmented reality glasses go beyond other augmented reality systems such as tablets or mobile phones. The user does not need to hold the device or to look down at the display as the virtual objects are displayed in their visual field. There are several OST augmented reality systems on the market apart from HoloLens glasses [21]. Nevertheless, a recent study highlighted HoloLens as being more suitable for surgical interventions than two other representative, commercially available OST devices [namely Moverio BT-200 (Epson, Shinjuku, Tokyo, Japan) and R-7 (ODG, San Francisco, CA, USA)] regarding frame rate, contrast perception and task load [22]. This technology may be applied when guiding EVAR procedures.

Our study forms part of the ongoing research project Nav EVAR, which focusses on guiding EVAR procedures by means of an interdisciplinary approach to reduce its current disadvantages, namely radiation exposure (medical staff and patients) and contrast agent administration (patients). Navigation, electromagnetic tracking systems, intraoperative US imaging, fibre Bragg gratings (FBGs) and optical coherence tomography (OCT) are the technologies under evaluation in this project. Augmented reality, specifically HoloLens, is used to visualise data from these systems. In addition, the Nav EVAR project also includes rapid prototyping of patient-specific phantoms for generating realistic test environments, avoiding animal experiments according to the German guidelines and European legislation on the protection of animals used for scientific purposes (European Union Directive 2010/63/EU). A general overview of this research project (namely motivation, objectives and involved technologies) is available on this video https://youtu.be/t5Bhvb9kf44.

The aim of this article is to describe the technologies involved in the current prototype developed in the Nav EVAR project to guide EVAR procedures and the experiments conducted to evaluate this approach. This study presents the current iterative process of biomedical engineering within the disruptive interdisciplinary project Nav EVAR that includes advanced navigation, imaging techniques and augmented reality with the aim of reducing side effects and optimising visualisation during EVAR procedures.

Materials and methods

Aortic aneurysm phantom

Advanced rapid prototyping and model manufacturing techniques were also introduced at this stage of the Nav EVAR project to provide a realistic test environment for integration, examination and evaluation of the technical components. A human model torso was built in cooperation with HumanX GmbH and the Fraunhofer Research Institution for Marine Biotechnology and Cell Technology. This torso comprises a patient-specific aortic model including outgoing vessels, the spine from the thoracic vertebra IV until the coccyx and the pelvis for anatomical orientation, and a hosepipe system to enable circulation of fluids. The patient-specific aortic model and bones were produced from a preoperative CT scan of a patient who was treated for aortoiliac aneurysm disease. The declaration of consent was completed and anonymisation of patient data was also implemented. The segmentation of the high-resolution CT scan (1-mm slice thickness) and conversion into STL-data was performed semi-automatically using 3D Slicer (open-source software) [23]. The patient-specific aortic model, spine and pelvis were produced using rapid prototyping [materials: FLX9070-DM (TangoPlus FLX930 and VeroClear RGD810; Stratasys, Eden Prairie, MN, USA) for the thoracic part of the aortic model, silicone for the abdominal part of the aortic model and polylactic acid (PLA) for the bones]. The bones were painted with lime varnish to improve their contrast in CT scans. The patient-specific aortic model is not rigid and is also exchangeable for other 3D-printed aortoiliac pathologies (models based on patient data to be collected during the Nav EVAR project) to provide a flexible modular system for further research.

Augmented reality

The specifications of Microsoft HoloLens glasses include an inertial measurement unit (namely an accelerometer, a gyroscope and a magnetometer) to track the user’s movements, and a time-of-flight depth-camera for finger tracking and depth sensing (also referred as spatial mapping) of enclosed environments. This spatial information is related to the user by means of four environment understanding cameras [24]. Spatial mapping enables the simulation of physical effects that make virtual objects look more realistic and the placement of virtual 3D objects or 2D panels at specific locations. HoloLens also includes a 2.4-megapixel video camera and four microphones for recording or streaming from the user’s point of view, which facilitates documentation of surgical procedures, communication with experts and training [25]. In HoloLens, virtual objects are selected with a user’s gaze while interaction is built on hand gestures and voice commands [26]. This hands-free manipulation makes HoloLens suitable for surgical environments [27]. Surgeons can visualise the patient’s data in front of their eyes maintaining sterile conditions, without turning their heads to look at other screens. In addition, this approach does not require a member of the surgical team to control the workstation [25]. HoloLens also inherently allows sharing virtual objects across multiple augmented reality devices, making it possible for the team in the operating room who are wearing HoloLens to see the same scene.

In the Nav EVAR project, 3D models (or triangle meshes) of the phantom surface and vascular structures are the virtual 3D objects superimposed on the phantom in the real world with HoloLens [28]. These 3D models are reconstructed after contouring them on a CT scan (surface rendering technique). The HoloLens wearer can also visualise virtual 2D objects on 2D panels such as the orthogonal views of CT scans, 2D images of 3D vascular models and 2D images of virtual angioscopies also built from CT scans. The point of view of the catheter, which was incorporated in the initial version of the Nav EVAR prototype [28], was replaced with a virtual angioscopy implemented with Open Inventor library [29]. The appearance of the vessel wall follows that shown in ex vivo and in vivo angioscopies [30], [31].

Navigation

The initial approach of the Nav EVAR project is guiding these procedures in real-time with an electromagnetic tracking system after attaching a sensor on the catheter tip and displaying this information on HoloLens. The proof-of-concept of this approach was evaluated in a phantom study with the Focused Assessment with Sonography for Trauma (FAST) Ultrasound Training Model (Blue Phantom, Sarasota, FL, USA), where vascular surgeons, radiologists and thoracic surgeons highlighted the potential of the prototype [28].

Tracking data is mapped to both the CT scan and the virtual objects obtained from these radiological data by means of a landmark-based rigid registration algorithm whose inputs are the 3D coordinates of at least three landmarks on the phantom distributed evenly around the area of interest in both coordinates systems (namely the electromagnetic tracking system and the radiological data). The position of each landmark in the former coordinate system is recorded with the tip of an electromagnetically tracked pointer. This process is carried out with voice commands on HoloLens rather than touching any keyboard or mouse. In addition, another registration is needed to align the renderings of the virtual 3D objects on HoloLens with the phantom in the real world. This process is done after pointing at the virtual landmarks displayed on HoloLens with the tracked pointer. Navigation is then carried out by electromagnetic tracking of the catheter tip inside the phantom. HoloLens displays the current position of the catheter tip on the 3D model of the vascular structure and on the orthogonal views of the CT scan, and the current view of the virtual angioscopy.

Specific external markers were designed to be used as landmarks in the registration process and with the aim of improving both their localisation in CT scans and when acquiring their position with the electromagnetically tracked pointer before navigation (Figure 1). Before acquiring the CT scan, the location of each external marker is previously marked (specifically, a point) on the phantom with a marker pen for later repositioning. Each external marker is placed on the phantom surface by matching the point marker and the hole of the external marker. Then the external marker can be removed after the CT acquisition. Before navigation, each external marker is placed again on the phantom following the point markers. The hole of the external markers was specifically designed to insert the pointer tip (lower diameter 0.98 mm and upper diameter 1.98 mm) without any movement to each other when recording the 3D coordinates of each external marker to reduce inaccuracies during this process.

Figure 1:

Design of the external marker.

(A) Lateral view. (B) Top view.

Ultrasound imaging

The use of real-time 3D US is also under evaluation in the Nav EVAR project for guiding endovascular tools. Several studies streamed 2D US images to HoloLens [32], [33], [34]. In our research project, US volumes (raw data) are streamed from a real-time 3D cardiovascular US system (Vivid 7 Dimension, GE Healthcare, Chicago, IL, USA), using a 3V matrix array probe, to a client computer using an in-house modification [35] and via Ethernet. Then, these volumes are transmitted to HoloLens with an open-source multi-language/platform remote procedure call (gRPC [36]) and via WiFi. Both steps use a client-server architecture via transmission control protocol/Internet protocol (TCP/IP). Streaming US volumes to HoloLens enables visualisation of both US data and probe relative to the patient. This information is superimposed on the HoloLens user’s field of view at a certain distance from the US probe. This process was done after attaching an augmented reality marker to the US probe and including HoloLensARToolKit [37] in the Nav EVAR prototype.

Evaluation

The solutions presented in this study were tested by using an US training model (FAST Ultrasound Training Model, Blue Phantom, Sarasota, FL, USA) and the aortic-aneurysm phantom. The former is a realistic model that simulates human tissues in the thorax, upper quadrant and abdomen regarding the deformation of soft-tissue (for instance, the skin) and the acoustic characteristics for US imaging.

CT studies were acquired with a Biograph40 scanner (Siemens, Munich, Bavaria, Germany) in the case of the FAST Ultrasound Training Model, and a Siemens SOMATOM Definition AS+ scanner in the case of the external markers and the aortic-aneurysm phantom. These studies were used to check whether the external markers were visible in CT scans, to create a virtual angioscopy from the aortic-aneurysm phantom (model built from different materials), and to check the navigation workflow and its visualisation with HoloLens using the FAST Ultrasound Training Model and a tracked catheter. The 3D models of the surface and the vessels (specifically, six tubes) of the FAST Ultrasound Training Model were obtained from the CT scan of this phantom. The registration process to set up a common coordinate system before navigation was carried out with four anatomical landmarks on the phantom surface (namely both mammillae, the belly button and the joint between both legs). During navigation, the tracked catheter was inserted in one the tubes that simulated the vascular structure and moved along it.

US volumes (imaging depth of 15 cm) of the FAST Ultrasound Training Model were also acquired with the Vivid 7 Dimension system at a rate of 13.8 Hz and visualised with HoloLens. The latency (specifically, the time interval between sending a volume from the US system and its visualisation on HoloLens) was calculated over 75 s after synchronising both the US system and the HoloLens with Network Time Protocol.

Results

Figure 2 shows the external marker designed for this study and the tip of the electromagnetically tracked pointer. The hole of the external marker enabled the viewing of the point marker and the pointer tip fitted in that hole. In addition, the external marker was also visible in the CT scan.

Figure 2:

External marker and pointer tip used for mapping the tracking and radiological data.

(A) External marker not aligned with the point marker. In this case, the point was drawn on the surgical tape. (B) External marker aligned with the point marker. (C) External marker and pointer tip. (D) Pointer tip inserted in the external marker. (E) Bottom view of the insertion. (F) CT slice with the lateral view of the external marker. (G) CT slice with the bottom view of the external marker. Voxel size 0.5×0.5×0.6 mm.

Figure 3 illustrates the virtual angioscopy built from the CT scan of the aortic-aneurysm phantom, apart from the 3D model of the segmented vessel system with the aneurysm and the orthogonal views of the CT scan. This visualisation replaces the point of view of the catheter included in the initial version of the Nav EVAR prototype shown in Figure 4.

Figure 3:

2D panel built from the radiological data.

(A) 2D image of the 3D vascular model. (B) View (2D image) of the virtual angioscopy. (C) Axial view of the CT scan of the aortic-aneurysm phantom. (D) Sagittal view of that CT scan. (E) Coronal view of that CT scan. The red point and the yellow line represent the position of the catheter tip and a virtual path of the catheter along the vessel centre line respectively.

Figure 4:

Visualisation with HoloLens.

3D models of the phantom surface (blue wireframe) and vessels (yellow) superimposed on the FAST Ultrasound Training Model. 2D panels above the phantom show the point of view of the catheter (left, initial version of the Nav EVAR prototype) and an axial slice of the CT scan of the phantom (right). This photo was taken from the HoloLens user’s point of view after connecting to the web server Windows Device Portal on the HoloLens [38].

The catheter tip was navigated in the FAST Ultrasound Training Model. Figures 4 and 5 illustrate the visualisation of that navigation with HoloLens. During the navigation, the rendering of the 3D model corresponding to the phantom surface was not perfectly aligned with the phantom in the real world (Figure 4), neither was the catheter tip with the tube (Figure 5).

Figure 5:

Navigation with HoloLens.

3D model of the vessels (yellow) superimposed on the FAST Ultrasound Training Model. The red crosses, the red sphere and the blue ring correspond to the virtual anatomical markers used in the registration process, the tip of the catheter and the user’s gaze respectively. This photo was taken with a second HoloLens, specifically from that HoloLens user’s point of view.

US volumes of the FAST Ultrasound Training Model were acquired with the Vivid 7 Dimension system and the 3V transducer and subsequently visualised with HoloLens. This data was superimposed on the HoloLens user’s field of view at a certain distance from the US probe, more precisely from the augmented reality marker attached to the US probe (Figure 6). The latency measured was 259±86 ms (mean±standard deviation).

Figure 6:

Visualisation of US volumes with HoloLens.

The volume rendering of the US data shows the kidney of the FAST Ultrasound Training Model.

Discussion

Microsoft HoloLens glasses were the augmented reality technology selected to visualise data during the navigation in EVAR procedures. This OST device allows a more intuitive visualisation of 3D content such as the position of the catheter tip along the 3D model of the aorta. In addition, 2D panels with information such as the virtual angioscopy or the orthogonal views of the CT angiogram can be placed at specific locations defined by the user and that are more convenient than those fixed positions of standard 2D screens. The virtual angioscopy showed better appearance than that presented in our previous study [39] and that included in the initial prototype [28]. However, HoloLens presents some drawbacks such as its weight (579 g), memory (limited to 900 MB for applications), battery life (2–3 h), binocular visual field [approximately 30°×17.5° (horizontal×vertical) compared to 120°×135° in humans (region of binocular overlap)], and potential discomfort (for instance, dizziness, headache, eye strain or dry eyes) [21], [40], [41], [42], [43], [44]. New versions of this experimental device or other augmented reality systems may overcome these disadvantages in the future. Nevertheless, HoloLens’ weight can be distributed around the head by adjusting its headband, users can get used to its limited field of view, compensating it by head movements, and the visualisation of virtual objects can be adjusted after an automatic calibration of the interpupillary distance.

The mapping between the tracking data and the radiological data used a landmark-based rigid registration algorithm. This study presents new external markers that can be identified in the CT scan (specifically, the centre of the bottom portion of the cylinder). Before navigation, the hole included in each marker enables its repositioning on the point marker drawn on the phantom surface and its localisation with the tracked pointer, avoiding that its tip slides along the phantom surface. The external markers will include a transparent adhesive layer to attach the marker to the phantom surface. Disadvantages of external marking might be a patient’s compliance. In particular, complex aortic anatomies require custom made stent grafts, such as fenestrated or branched grafts, whose design is based on preoperative CT angiograms and with production delays up to 2 months. The point marker drawn on the patient’s skin before the CT angiogram acquisition should last until the endovascular intervention to allow the placement of the external marker.

During the navigation, the position of the catheter tip showed some error as it was not perfectly aligned with the tube. The use of the anatomical landmarks on the phantom that are not accurately identified in the CT images and just before navigation may cause this misalignment. Further research will include the use of the aortic-aneurysm phantom, the designed external markers and the quantitative assessment of the target registration error by means of acquiring CT or cone-beam CT (CBCT) scans of the whole setting (phantom with external markers and tracked catheter) during the navigation. In addition, the alignment based on external markers on the patient’s skin may not be sufficiently accurate in obese patients owing to skin movement. This matching could be improved when including intraoperative imaging, such as real-time 3D US, and a deformable registration between preoperative and intraoperative imaging modalities [45] to update the navigation with the patient’s current anatomy. Preoperative CT angiograms may provide the overview and orientation of the target, but not resemble the actual scenario in the operating room owing to patient position or tissue deformation/movement, while intraoperative US volumes offer a limited but updated picture during the treatment [46]. A robotised US system may address the problem of continuous acquisition of US volumes during a treatment without manual positioning of the US probe [47]. On the other hand, mapping between the renderings of the 3D models (namely the phantom surface and vascular structures) on HoloLens and the real world may be improved by including surface data. This approach will be tested after attaching a high-quality depth camera to HoloLens in order to improve the rough surfaces obtained from the spatial mapping of HoloLens [28], [48].

In this study, tracking of endovascular tools was based on attaching an electromagnetic sensor on the catheter tip. However, these surgical devices are flexible to facilitate their delivery through tortuous vessels. A better understanding of the position of the instrument related to the patient’s anatomy can be obtained when tracking several points of the tool rather than only its tip. In addition, the visualisation of the catheter shape could avoid damaging the vessel wall [49]. In [50], two miniaturised electromagnetic sensors with five degrees of freedom (DoF), namely three translations and two rotations, were attached inside a 5-F catheter, one at the catheter tip and another sensor a few centimetres below, in order to estimate the tip position, the orientation of the catheter axis at both sensor locations and the catheter curvature. The number of sensors was increased in [51], where the authors embedded seven electromagnetic sensors in a 7-F radiofrequency ablation catheter sheath [the sensor at the tip has 6 DoF (three translations and three rotations)] to reconstruct its shape for a distance of 70.5 cm inside a 2D silicone aortic phantom. Results showed an average error of 3.0 mm when estimating the catheter shape at three sensor positions (catheter tip, middle and catheter distal end) using the electromagnetic tracking data. A lower average error (2.1 mm) was obtained when the simulation of the mechanical characteristics of the catheter was included in the estimation of its position. An alternative for shape sensing is based on FBG technology. An FBG sensor consists of a periodic structure (grating) in an optical fibre core that reflects a narrow band of wavelengths of the incident light. Strain or temperature changes the grating period which in turn shifts the spectrum of the reflected light [52]. A common configuration for shape reconstruction consists in placing three optical fibres parallel to the longitudinal axis of the needle/catheter with several sets of FBG sensors at specific locations along the optical fibres. Each set is composed of three FBG sensors (one in each optical fibre) aligned following a triangle (cross section). This configuration enables temperature compensation and the determination of twist angles [53]. The shape is obtained after interpolating the magnitude and direction of the curvature calculated from strains measured at each set of FBG sensors. The accuracy is affected by factors such as the sensor configuration (number and placement of FBG sensors) and the interpolation method [54]. The error increases when estimating long shapes and tools that have low stiffness [53]. In [55], the authors estimated the needle shape with a maximum error of 0.74 mm when inserting the needle 115 mm into a soft-tissue phantom (gelatine) using four sets of FBG sensors (triangular configuration) separated by 30 mm each (first set of FBG sensors at 18 mm from the needle tip). A limitation of the FBG technology is that shapes are obtained in the needle/catheter coordinate system (local coordinate system) [56]. Therefore, no information about the tool position is available, only its deflection. Further research in the Nav EVAR project will focus on the use of FBG sensors for shape reconstruction [39] combined with electromagnetic tracking to increase accuracy [53] and provide the position in a global coordinate system.

US volumes of the FAST Ultrasound Training Model were superimposed on the HoloLens user’s field of view at a certain distance from the US probe by using an augmented reality marker attached to the US probe. However, the latency should be reduced to at least 100 ms in order to display real-time 3D US volumes on HoloLens. In addition, the implemented method of opacity-based volume rendering in HoloLens and the interaction with US volumes were limited. The opacity transfer function should be replaced with an approach that enhances structures of interest better than that implemented in commercial systems and that is more suitable for real-time 3D US data, such as that presented in [57]. Furthermore, the user interface in HoloLens should be intuitive and also include tools such as moving, rotating and cropping the volume rendering, and, if necessary, adjusting parameters of the opacity transfer function. This interaction would be built on hand gestures and voice commands.

Obesity, bowel gas and vessel calcification are factors that may hinder the use of real-time 3D US to guide EVAR procedures due to the limited penetration depth of US and the acoustic shadowing caused by interfaces like tissue/air or tissue/bone. Intravascular US (IVUS) enables the visualisation of vessels from the inside out to address those potential obstacles of real-time 3D US. Cross-sectional images of the vessel are acquired by means of a catheter with a miniaturised US probe mounted on its tip. This image modality enables the assessment of a vessel’s lumen and its wall, and also shows atherosclerotic plaque [58]. Stent grafts can be identified in these images, as well as the shadow caused by guide wires [58], [59]. In [60], an IVUS probe tracked with an electromagnetic sensor was proposed to guide the installation of stent grafts. 3D models of a silicone descending thoracic aorta were reconstructed based on this data with a cross-section radius average error of 0.9 mm. An alternative catheter-based imaging technique is OCT, which is based on near-infrared light emission. This technology provides less tissue penetration and scan diameter than IVUS, but offers a higher resolution and frame rate with a smaller catheter size [61]. Acquiring intraoperative OCT images for guiding EVAR procedures will also be assessed in the Nav EVAR project.

The Nav EVAR project tackles the problem of radiation exposure and contrast agent administration during EVAR interventions by using a multidisciplinary approach that includes navigation, electromagnetic tracking systems, intraoperative US imaging, FBGs and OCT to guide the endovascular tools (namely catheters, guide wires, sheaths and stent grafts). Augmented reality technology, specifically HoloLens, was included to enable a more intuitive visualisation of the navigation data than on standard screens. This article reviewed the current state of this project and its further research, describing the potential and limitations of each technology. The combination of these techniques is a promising approach to overcome the problems of EVAR procedures.

Supplementary Material

The article (iss-2018-2001) offers reviewer assessments as supplementary material.