Abstract
Minimally invasive neuroendoscopic surgery provides an alternative to open craniotomy for many skull base lesions. These techniques provides a great benefit to the patient through shorter ICU stays, decreased post-operative pain and quicker return to baseline function. However, density of critical neurovascular structures at the skull base makes planning for these procedures highly complex. Furthermore, additional surgical portals are often used to improve visualization and instrument access, which adds to the complexity of pre-operative planning. Surgical approach planning is currently limited and typically involves review of 2D axial, coronal, and sagittal CT and MRI images. In addition, skull base surgeons manually change the visualization effect to review all possible approaches to the target lesion and achieve an optimal surgical plan. This cumbersome process relies heavily on surgeon experience and it does not allow for 3D visualization. In this paper, we describe a rapid pre-operative planning system for skull base surgery using the following two novel concepts: importance-based highlight and mobile portal. With this innovation, critical areas in the 3D CT model are highlighted based on segmentation results. Mobile portals allow surgeons to review multiple potential entry portals in real-time with improved visualization of critical structures located inside the pathway. To achieve this we used the following methods: (1) novel bone-only atlases were manually generated, (2) orbits and the center of the skull serve as features to quickly pre-align the patient’s scan with the atlas, (3) deformable registration technique was used for fine alignment, (4) surgical importance was assigned to each voxel according to a surgical dictionary, and (5) pre-defined transfer function was applied to the processed data to highlight important structures. The proposed idea was fully implemented as independent planning software and additional data are used for verification and validation. The experimental results show: (1) the proposed methods provided greatly improved planning efficiency while optimal surgical plans were successfully achieved, (2) the proposed methods successfully highlighted important structures and facilitated planning, (3) the proposed methods require shorter processing time than classical segmentation algorithms, and (4) these methods can be used to improve surgical safety for surgical robots.
Keywords: Mobile Portal, Importance Visualization, CT Segmentation, Skull Base Surgery Planning
1. INTRODUCTION
Minimally invasive neuroendoscopic surgery is an alternative to open craniotomy to access skull base lesions [1]. Successful skull base surgery demands precise pre-operative planning due to the high density of critical neurovascular structures. Moreover, current endoscopic technology allows a limited field of view during the procedure, which makes the surgery more challenging.
During the planning for this type of surgery, surgeons attempt to minimize the disturbance to critical structures during the procedure, without sacrificing access to the pathology. In addition, skull base procedures often use more than one entry portal to reach the target, supporting the need for more detailed preoperative planning to select and examine possible portals [2]. With recently developed surgical techniques, which utilize novel portals other than the traditional trans-nasal portal, searching for the optimal plan becomes more complex, time-consuming, and error-prone [3].
Prior work has addressed the planning challenges for single [4, 5] and multi-portal approaches for specific skull base and target areas [6] from a clinical perspective. We propose to utilize image processing and visualization techniques to improve pre-operative planning for skull base surgery.
We introduced a novel concept of mobile portals that will allow surgeons to quickly review all potential pathways in a 3D model. This avoids the need for slice by slice 2D visualization of conventional computed tomography (CT) and magnetic resonance imaging (MRI) viewers. We propose a novel visualization of CT data based on the relative importance of structures, often depending on their distance to vital structures. The skull base bones are segmented into different contours with emphasis on critical structures, which in this region include the internal carotid artery, cavernous sinus, and cranial nerves.
In this paper, we describe a skull base surgery rapid pre-operative planning system, which uses novel importance-based visualization and the ability to move the portal to explore various approaches. The atlas based segmentation was performed to segment the skull base bones into different contours. A dictionary was created where the bone structures were assigned a relative value according to their importance or their distance to the vital structures. The 3D visualization was done by applying a pre-defined transfer function to highlight the critical anatomy. Surgeons can then manually specify the target, and the entry portal. Through manipulating the portal location, surgeons can quickly review 3D pathways in addition to viewing the conventional 2D image slices. With the importance-based visualization, surgeons can see the critical structures inside the pathways and more efficiently determine the optimal approach.
The rest of the paper is organized in the following way. Section II describes the proposed system and mean components, such as (1) the atlas based bone registration, (2) importance-based visualization method, and (3) mobile portal for rapid planning. In section III, experimental results for planning tasks are presented. Lastly, we discuss the proposed methods and our future directions.
2. METHOD
2.1. System Overview
Figure 1 shows the workflow of the proposed system which has 4 main components. In preprocessing the features and bone structures are extracted from CT data. In the registration stage the results from previous stage are used to pre-align the data with the atlas for rapid registration and segmentation of bone structures. In the volume rendering stage bone structures are visualized based on their importance level (previously assigned). In the mobile portal stage surgeons can explore different pathways by manipulating the portal in 3D.
Figure 1.
Workflow of rapid skull base surgery planning
2.2. CT Bone Atlas and Importance Dictionary
2.2.1. Manual segmentation of skull base bone
The atlas was created by manually segmenting the bone structure through collaboration with Head and Neck surgeons at the University of Washington.
2.2.2. Anatomy based importance dictionary definition
In medical imaging physicians often are interested in certain spatial regions where the data should be emphasized. We could utilize this prior knowledge to highlight these areas in the final volume rendering. In skull base surgeries bone anatomy is important because of its stable position. For this reason, surgical navigation systems are based on bone landmarks.
Bone segments and voxels are assigned a relative value based on proximity to critical structures (e.g. carotid artery and optic nerve). Figure 2 highlights the bone structures near the carotid canal and cavernous sinus.
Figure 2.
Relative Value Assignments
2.3. Atlas and Feature Based Rapid Registration and Segmentation
Image registration is a time consuming process. Registration efficiency was increased by focusing on bone windows and utilizing common features pre-registered from anatomical atlas.
2.3.1. Bone Extraction
Simple intensity global threshold was used for bone extraction, which improves efficiency and requires manipulation of only one parameter [7].
2.3.2. Feature Extraction
The goal of this step is to pre-align the test and atlas volumes to streamline registration process. For automatic alignment, the software finds the orientation of the patient’s skull and matches it with orientation of the atlas data set. The orbits and the center of skull are used for orientation. Edge Canny detection was used due its optimal performance with noise, localization, and the amount of the details that could be controlled [8]. The feature used for the identification of the orbits is the approximate circularity of the orbits and the Hough Transform (HT) was used to detect the circular objects [9]. To completely describe the orientation of the data set, the center of the image was used as the third feature. Thus, the two data sets were spatially aligned for rapid registration.
2.3.3. 3D Registration and Segmentation
Image registration is the process of finding the optimal transformation T: (x,y,z)->(x’,y’,z’) which maps any point in the test volume to a corresponding point in the reference volume while minimizing the cost function.
We used a combined transformation T which consists of global and local transformation. This step aims to correct for global differences in position, orientation, and size between an atlas and a subject volume. We used 9 degrees of freedom as the registration model, where the free parameters include three translational, three rotational, and three scaling factors. To find the optimal transformation parameters, we minimized the mean squared pixel-wise difference between the atlas and subject volumes using a gradient-descent optimization scheme. An additional transformation was required to model the local deformation of the skull because the bone volumes are different in both size and shape across patients and global transformation is not sufficient to capture this variation. Therefore, we used B-splines, which are a well-known approach for non-rigid registration of 3D medical volumes [10]. In this method, a mesh of control points with uniform spacing is defined, which act as parameters of the B-splines method. The degree of this non-rigid deformation is tuned by changing the resolution of the mesh of the control points. To achieve accurate and efficient registration, we used the same cost function, minimizing mean squared pixel-wise difference, and the Limited-memory BFGS optimizer. LBFGS is an optimization algorithm in the family of Quasi- Newton methods that approximates the Broyden Flecther Goldfarb Shanno (BFGS) algorithm using a limited amount of computer memory. Limited-memory makes this optimization technique suited for problems with large number of variables [11].
Once the registration problem is solved, in the case of the single atlas, the easiest and fastest way to assign a label to each input image voxel is to propagate the atlas labels to image space [12]. With this strategy, the segmentation process relies on a registration process that aims to estimate the anatomical differences between the atlas and the input image volumes. In the case of single atlas, we assume that the atlas is close to the subject’s anatomy. In future work, we plan to use the software and experts time to segment different data sets and construct probabilistic atlases. With a robust atlas, additional data set can be used to test the software.
2.4. Importance Based Visualization
The crucial step in volume rendering is to design the transfer function. This function determines which structures of the volume will be visible and how they will be rendered [13]. To better convey the information in CT volume data, we utilized the importance-based information to find the optimal window level, opacity, and color to highlight the influential structures. For example, bone structure adjacent to optic nerve would have a higher opacity and brighter color, while ethmoid bone structures would be more transparent with regular bone color. Since this transfer function is designed based on importance of different structures and segmentation, it is generalizable to all data sets.
2.5. Mobile Portal for Fast Surgery Planning
Surgeons identify the target in 3D visualization or any 2D view (coronal, axial and sagittal) and similarly select the portal entrance area, whether it is skull surface, transnasal or transorbital approaches. Surgeons visualize the defined pathways by using endoscopic view, or any conventional 2D slices of CT or MRI. Surgeons manipulate the portal and adjust the geometry in an iterative process until they are satisfied with the location and dimensions. With this method surgeons can quickly examine different options and choose the optimal surgical approach (or combination of approaches).
3. RESULTS
3.1. Atlas
The atlas contains 20 different bone contours inside the skull base area which was segmented manually. Figure 3 shows one the axial slice of the atlas.
Figure 3.
Manual segmentation of the skull
3.2. Registration and Segmentation Result
For assessing the algorithm, we warped the atlas volume and used it as our patient data set, where we have the ground truth to evaluate the segmentation result. We used a landmark deformation method to deform the atlas and utilized this as our test volume. Figure 5 shows the axial visualization of a slice in our atlas data set (5.A), the deformed visualization of the same slice (5.B), the difference between the original image and the warped image (5.C), visualization of a slice from the 3D registration result (5.D), and its difference with the original image (5.E). Then the bone structures were segmented into different contours and assigned a weight value according to the dictionary.
Figure 5.
Visualizing slice from 3D registration, left to right: A. fixed Image (atlas), B. moving Image (warped atlas), C. registration result, D. rescaled difference of moving image and fixed image, E. difference of registered image and fixed image
3.3. Visualization Effect
Direct volume rendering was used to create the 3D model. To visualize the bone segments, we used pixel wise rendering vs mesh generation of segments. This visualization technique helps surgeons to have access to the raw pixel information in the case of mis-registration. We used shades of red to color segmented bones according to their importance. Bone structures which belong to contours with high importance level were red and relatively less important structures were white (Figure 6).
Figure 6.
Bone visualization of a slice from 3D volume based on the importance
3.4. Mobile Portal for Fast Planning
Using the 3D model, or any 2D views, surgeons determined the target and defined the pathway to the target. Next surgeons manipulated the virtual pathway to examine various approaches to the target. Figure 7, shows screen shots of manipulating the surgical pathway.
Figure 7.
Manually Drag Portals to Fast Review Pathway
Surgeons can examine each pathway using 2D or endoscopic view. With emphasized critical structures they can easily find the optimal pathway. Figure 8 shows the selection of the target (yellow sphere) and two different pathways (labeled 1 and 2) to the same target.
Figure 8.
Importance based visualization for pathway review
Comparing these two pathways, it is shown that pathway 1 contains less critical structures than pathway 2 and in general the amount of bone removal would be less for pathway 1. Surgeons can use this visualization or switch to the raw CT scans to examine structures inside the selected pathway.
3.5. Numerical Evaluation and Software
We have developed software with mobile portal and importance based visualization which helps surgeons make a rapid selection and examine the path to the target in the skull base surgery. Using this software, surgeons can select a pathway for each instrument they plan to use during the surgery. This software helps to plan the surgery by improved visualization and by providing information about the angle between pathways and length of the pathway.
4. DISCUSSION
We have developed software for rapid planning of skull base surgery by a novel CT visualization and 3D pathway visualization to the target lesion. The new visualization is based on emphasizing the relative importance of skull base bone structures. In this software, after specifying the target surgeons can manipulate the by moving the portal to examine different approaches. We used atlas-based registration to segment bones into different structures. Using the prior knowledge we assigned a relative importance level to each bone segments. To improve the visualization, different transfer functions were generated according to the importance assigned to bone segments. Surgeons can thus rapidly visualize all vital structures inside the pathway.
The preliminary results show that the proposed software improves the surgical planning in the following ways: (1) the ability to rapidly explore different pathways to the pre-specified target, (2) the ability to visualize the structures inside the pathway according to their relative importance level, (3) the ability to manipulate the portals and change the geometry of each pathway with minimal interaction, and (4) the 3D visualization for efficient planning.
Currently, the software is only used with different warped atlas volume as the patient data. However, further work is needed to assess the software in following aspects: (1) construction of a rich collection of atlases representing different patients, (2) determine the registration rate for additional data set, (3) defining a more detailed dictionary by utilizing the prior knowledge, (4) evaluate the planning precision and assess the effect of the visualization, and (5) automatic pre-operative planning of the surgical pathway.
5. CONCLUSION
CT scans are widely used in skull base surgery due to their high discrimination of bony structures. However, CT scans have poor discrimination of soft tissues. In this paper, a 3D visualization method for rapid pre-operative planning of skull base surgery was proposed. Atlas and features are used to extract contours and different transfer functions are applied to each contour for volume rendering. The experimental results show that the software improves efficacy of planning skull base surgery.
Figure 4.
3D Bone visualization: atlas data set (left) and warped atlas data set as test data (right)
Figure 9.
Skull base surgery planning software GUI with work flow
ACKNOWLEDGMENTS
This work was supported by NIH grant 5R21EB016122–02 and Coulter grant “Computer Model for Advanced Preoperative Planning in Skull Base Surgery”.
REFERENCES
- [1].Bly RA, Su D, Lendvay TS, Friedman D, Hannaford B, Ferreira M, and Moe KS, “ Multiportal robotic access to the anterior cranial fossa a surgical and engineering feasibility study,” Otolaryngology-Head and Neck Surgery, 149(6), 940–946. (2013) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [2].Ciporen J,N, Moe K,S, Ramanathan D, Lopez S, Ledesma E, Rostomily R, Sekhar L,N, “Multiportal Endoscopic Approaches to the Central Skull Base: A Cadaveric Study,” World Neurosurgery, 73, (2010). [DOI] [PubMed] [Google Scholar]
- [3].Lim JH, Sardesai MG, Ferreira M, & Moe KS, “Transorbital Neuroendoscopic Management of Sinogenic Complications Involving the Frontal Sinus, Orbit, and Anterior Cranial Fossa,” Neurological Surgery, Skull Base, 73(6), (2012). [DOI] [PMC free article] [PubMed] [Google Scholar]
- [4].Bell B, Stieger C, Gerber N, Arnold A, Nauer C, Hamacher V, Kompis M, Nolte L, Caversaccio M, Weber S: A self-developed and constructed robot for minimally invasive cochlear implantation. Acta Otolaryngol. 132, 355–360 (2012) [DOI] [PubMed] [Google Scholar]
- [5].Wanna G, Balachandran R, Majdani O, Mitchell J, Labadie R: Percutaneous access to the petrous apex in vitro using customized micro-stereotactic frames based on image-guided surgical technology. Acta Otolaryngol. 130, 458–463 (2010) [DOI] [PMC free article] [PubMed] [Google Scholar]
- [6].Becker M, Hansen S, Wesarg S, Sakas G, [Clinical Image-Based Procedures], Springer International Publishing, 23–30 (2014). [Google Scholar]
- [7].Yao W. and Abolmaesumi P, “An estimation/correction algorithm for detecting bone edges in CT images,” Medical Imaging, IEEE Transactions, 24.(2005) [DOI] [PubMed] [Google Scholar]
- [8].Canny J. “A Computational Approach to Edge Detection,” Pattern Analysis and Machine Intelligence,” IEEE Transactions, 8. (1986) [PubMed] [Google Scholar]
- [9].Kaggelides K, Elliott PJ and Fisher RB, “Automatic location of the eyes in CT images of the head,” Proc. Int. Symp. on Computer Assisted Radiology, (1993). [Google Scholar]
- [10].Rubeckert D, Sonoda L,I, Hayes C, Hill D,L,G, Leach M,O, and Hawkes D,J “Non-rigid registration using free-form deformations: application to breast MR images,” Medical Imaging, IEEE Transactions, 18. (1999). [DOI] [PubMed] [Google Scholar]
- [11].Harsha B,S, Pournami P, Govindan V, “B-Spline non rigid image registration using L_BFGS optimizer,” IJCSIT, 5, (2014). [Google Scholar]
- [12].Kalinic H, “Atlas-based image segmentation: A Survey,” Department of Electronic Systems and Information Processing, Universiy of Zagreb. [Google Scholar]
- [13].Ruiz M, Bardera A, Boada I, and Viola I, “Automatic transfer functions based on informational divergence,” Visualization and Computer Graphics, IEEE Transactions, 17, (2011). [DOI] [PubMed] [Google Scholar]