Virtual Reality Cerebral Aneurysm Clipping Simulation With Real-time Haptic Feedback

Abstract

Background

With the decrease in the number of cerebral aneurysms treated surgically and the increase of complexity of those treated surgically, there is a need for simulation-based tools to teach future neurosurgeons the operative techniques of aneurysm clipping.

Objective

To develop and evaluate the usefulness of a new haptic-based virtual reality (VR) simulator in the training of neurosurgical residents.

Methods

A real-time sensory haptic feedback virtual reality aneurysm clipping simulator was developed using the Immersive Touch platform. A prototype middle cerebral artery aneurysm simulation was created from a computed tomography angiogram. Aneurysm and vessel volume deformation and haptic feedback are provided in a 3-D immersive VR environment. Intraoperative aneurysm rupture was also simulated. Seventeen neurosurgery residents from three residency programs tested the simulator and provided feedback on its usefulness and resemblance to real aneurysm clipping surgery.

Results

Residents felt that the simulation would be useful in preparing for real-life surgery. About two thirds of the residents felt that the 3-D immersive anatomical details provided a very close resemblance to real operative anatomy and accurate guidance for deciding surgical approaches. They believed the simulation is useful for preoperative surgical rehearsal and neurosurgical training. One third of the residents felt that the technology in its current form provided very realistic haptic feedback for aneurysm surgery.

Conclusion

Neurosurgical residents felt that the novel immersive VR simulator is helpful in their training especially since they do not get a chance to perform aneurysm clippings until very late in their residency programs.

INTRODUCTION

Reduced resident training hours, increasing complexity of some operations including open cerebral aneurysm clipping, greater emphasis on operating room efficiency, and concerns for patient safety have all driven the development of virtual reality (VR) neurosurgery simulation. The Accreditation Council for Graduate Medical Education (ACGME) has not yet required simulation-based training and assessment in neurosurgery as in some other fields (e.g., general surgery), but such assessment has been mentioned as a possible future milestone for residents in a recent ACGME-led workshop for Neurosurgery Program Directors.¹ The Congress of Neurological Surgeons Resident Training Program in Neurosurgical Simulation was created in 2010, and the program has produced effectiveness studies of both VR and manikin training.^{2, 3} The October 2013 Supplement to this journal focused on Simulation in Neurosurgery: Possibilities and Practicalities,⁴ including a detailed literature review.⁵ Theongoing discussion can therefore focus on particular simulators and procedures, including microsurgical clipping of cerebral aneurysms.

Brain aneurysms are associated with a relatively significant mortality and morbidity; their surgical management is technically demanding. Training in aneurysm clipping became more complicated with the drastic drop in the number of aneurysms being treated surgically: technically easier aneurysms and most unruptured aneurysms are being treated with endovascular means (stent-assisted coiling or flow divertingstents).⁶ Yet these difficult cases of clipping are not the ideal form in which novices would normally learn a new procedure, and regardless of the complexity of the particular aneurysm, it takes a lot of practice to learn to dissect and clipaneurysms in general.

Accordingly, researchers at the University of Illinois at Chicago and the University of Chicago have developed a prototype of an aneurysm clipping simulation designed to operate on the Immersive Touch® platform (Chicago, IL), referred to here as the Immersive Touch Aneurysm Clipping Simulator (ITACS). The project aims to provide the first realistic haptic virtual reality clipping of a patient-based cerebral aneurysm. The system is equipped with software and devices that provide users with stereoscopic visualization and force feedback simultaneously in real time. For the present study, a prototype middle cerebral artery aneurysm (MCA) clipping module has been tested by neurosurgical residents from three residency programs, the University of Illinois at Chicago, the University of Chicago, and Weill Cornell Medical College.

METHODS

The study group was comprised of 17 of residents who practiced clipping of a left MCA aneurysm on a patient-specific model for the Immersive Touch Aneurysm Clipping Simulator. The basic Immersive Touch platform, with its monitor-mirror system for hologram-like projection of 3-Danatomy, was introduced in this journal in 2007.⁷ The aneurysm clipping module was built based on existing haptic VR training modules for this workstation.⁸ These include ventriculostomy, which has been validated,^{7, 9-11} positively reviewed,¹² and independently tested,² bone drilling,⁸ spinal procedures such as pedicle screw placement,^{13, 14} vertebroplasty, and lumbar puncture, trigeminal rhizotomy,⁸ and other procedures.^{15, 16} These modules are currently used for training of neurosurgical residents around the world and are part of organized neurosurgery simulation training courses.^{2, 15} In all these cases, the inserted VR probe or cathetercollides with the virtual anatomy along a series of single points on a straight line to provide continuous force feedback. However, aneurysm clipping required additional engineering so that the blades of the virtual aneurysm clip can contact and deform the blood vessel walls at multiple points at the same time.

Aneurysm Clipping Module Creation

The ITACS module consists of a platform for 3-dimensional (3-D) image processing, visualization, and surgical planning. The user reaches in with both hands behind a half-silvered mirror to enter an interactive stereoscopic immersive environment containing the patient’s specific3-D imaging data as well as various tools. Well-matched haptics and graphics volumes are realized, and high resolution visualization and head and hand tracking are provided. Wearing the head tracking device allows the operator to have a fully immersive experience of the macroscopic parts of the aneurysm operation, such as burr hole drilling and craniotomy. Real-time tactile feedback is possible bimanually. Figures 1 and 5 of our 2012 article illustrate the Immersive Touch platform for open surgery.⁸

Figure 1.

A general view of the immersive virtual reality environment being used by a resident. The operator is using haptic devices in both right and left hand which mimic the operating room environment. The virtual image is projected on a screen in front of the resident.

Figure 5.

The result of the residents’ assessment, which was based on 5-point Likert scales where 5 represents the highest rank and 1 represents the lowest rank.

To construct the aneurysm clipping module, a computed tomography angiogram (CTA) was acquired of a patient with an MCA bifurcation aneurysm as part of a routine preoperative protocol for surgical clipping. CT was performed on a GE 64 slice scanner. Thin slice (1 mm) volumetric images were acquired and transferred as a Digital Imaging and Communications in Medicine(DICOM) format to the ITACS Platform. Using a combination of automatic and manual segmentation of DICOM imaging, a stereoscopic 3-Dmodel was reconstructed of the skin, skull, brain parenchyma, and blood vessels. This is followed by retopolgy of the generated 3-D polygonal mesh to represent the patient anatomy. The sylvian fissure was modeled to be partially open, displacing the frontal and temporal lobes, with the MCA proximal and distal branches within the fissure (as identified on CTA). This allows the operator to retract the brain if needed to be able to clip the MCA aneurysm. An artistic 3-D drawing of the minor brain vessels was used to generate a bump map texture applied to the virtual brain to increase the visual realism. The final model contains patient-specific information including the skin, bone, brain, and arterial network harboring the MCA aneurysm (Figures 1, 2 and 3).

Figure 2.

3-D stereoscopic immersive virtual reality model of the skull and head for middle cerebral artery (MCA) aneurysm clipping simulation, based on patient-specific CTA. A. The operator designs the craniotomy for the best surgical approach. B. View after the craniotomy is created and the dura opened. Frontal and temporal lobes are separate volumes that can be retracted independently. C. Virtual head model with skin flap raised and lifted anteriorly. The operator grasps the haptic device stylus, which appears in the VR space as an aneurysm clip holder. The aneurysm clip is suspended in space. D. The operator grasps the clip with the clip holder. This provides a test of 3-D depth perception.

Figure 3.

Virtual clipping of patient-specific MCA aneurysm. A. Translucent representation of brain parenchyma reveals the position of the proximal M1 and distal M2 branches within a partially opened sylvian fissure; clip is applied for reference purposes. B. Same view as A with opaque parenchyma shows the M1 to be hidden under the temporal lobe. C. Sufficient visualization of the M1, aneurysm neck, and M2 branches is achieved by greater separation of frontal and temporal lobes. This was the view presented to the neurosurgery residents. D. The operator uses the side of the aneurysm clip to retract aneurysm dome in real time, with sensory haptic feedback. E. The operator presses on the haptic stylus, representing the virtual clip applier, to open the aneurysm clip blades. As the clip is applied across the neck, it produces real deformation of the aneurysm. F. The aneurysm clip is released across the neck and the blades close, causing the aneurysm to deform and collapse. Proximal M1 and distal M2 branches appear to be patent.

The system allows the user to evaluate and appreciate the angio-architecture of the aneurysm and the surrounding anatomical structures, in addition to the spatial relationship of the aneurysm, parent vessel, and distal branches. Aneurysm clipping involves selecting a clip using a simulated clip applier. The simulator operator experiences real-time haptic feedback as the clip pushes against the aneurysm dome or neck or the adjacent parent vessel. As the clip is released, the aneurysm shows real-time deformation at the location of the clip, thus leaving an aneurysm remnant if the clip does not extend fully across the neck or compromising the parent MCA if the clip is applied partially or completely on the MCA. The sensory feedback reflects the material properties of the structure being manipulated (bone, brain parenchyma, and arteries).

Aneurysm Clipping Residents’ Trial

We evaluated the aneurysm module’s utility in a group of neurosurgical residents. Seventeen neurosurgery residents from three residency programs—University of Illinois at Chicago, University of Chicago, and Weill Cornell Medical College—practiced clipping the virtual reality MCA bifurcation aneurysm and responded to a questionnaire. This consisted of 24 questions that were scored using a combination of dichotomous-response items (Yes/No), 5-point Likert scales, and free-text responses. The primary goal of the survey was to assess the usefulness of the aneurysm clipping simulator as an education tool and determine to what degree it simulates the real surgery. Residents were allowed to orient themselves to the Immersive Touch modules and platform before evaluating the aneurysm clipping module.

The MCA aneurysm clipping operation begins with a pterional craniotomy (Figure 2 A, B). The trainee draws the outline on the surface of a haptically sensible skull and performs the craniotomy. The underlying software performs a bone volume removal operation to obliterate the bone. In another implementation the trainee cuts the outlines of the bone flap with a simulated craniotome.

The simulated surgical steps illustrated in Figures 3 and 4 are most easily observed in Video 1 (Supplemental Digital Content 1). Sensible force is felt and deformation is seen whenever the aneurysm clip or the suction tip touches a blood vessel or brain tissue. For the sake of this exercise, the dura is already open and the sylvian fissure is opened partially, demonstrating the MCA bifurcation aneurysm, the proximal M1 segment and the distal M2 branches within the depth of the sylvian fissure (Figure 3 C). The next task for the trainee is grasping the aneurysm clip (here modeled as straight 10 mm) with a virtually reconstructed aneurysm clip holder (Figure 2 C, D). The task requires a good 3-D spatial depth perception to position the clip holder in the same plane as the clip for grasping. A variety of VR clip designs are being produced for the simulator. With adequate visualization of the aneurysm neck at the junction of the M1 and M2 branches, clip placement can be performed (Figure 3 A-F). The trainee can feel force and see tissue deformation as the clip is positioned against the aneurysm neck. This is possible because of multiple collision detection points placed along the two shafts of the aneurysm clip. As the aneurysm clip shafts close, the aneurysm deforms in real time in response to sidewall collision with the clip, by a position-based dynamic polygonal mesh deformation algorithm. The force feedback from the collision between the clip and neck was designed to reflect that encountered in real-life surgery. Finally, the ITACS aneurysm module simulates aneurysm rupture if excessive manipulation and force is exerted at the dome of the aneurysm (Figure 4). Sudden blood flow from the dome is represented by a particle-based fluid simulation, and two-handed operation enables use of a suction tip to control the bleeding until the clip can be correctly placed. Future scoring of aneurysm clipping will include patency of the parent vessel, incomplete clipping of the aneurysm, excessive brain retraction, intra-procedural rupture, and proper selection of aneurysm clips.

Figure 4.

The virtual suction tip removes blood from simulated aneurysm rupture. The tool can also be used to retract the brain.

Data were entered into the IBM SPSS Statistical Package version 20.0.

RESULTS

Seventeen neurosurgery residents and fellows with an average postgraduate experience of 3.9 years participated in the evaluation survey (Table 1). On a pre-simulation questionnaire, thirteen residents reported that they had performed an average of 70 craniotomies; seven residents (seniors or fellows) had performed aneurysm clippings on actual patients. On the simulator no double images or difficulties perceiving three dimensions were recorded for any of the participants. Residents generally reported that the simulator represents real anatomy and enables visualization of the M1 and M2 arteries of this MCA bifurcation aneurysm. Six participants saw and felt the collision of the clip with the aneurysm neck and achieved closure. Nine participants had difficulty in grasping, opening, and closing an aneurysm clip in 3-D because of unfamiliarity with the depth perception needed to find the aneurysm clip located in the 3-D virtual space. More practice on the simulator may be especially helpful for them especially since many of them did not perform aneurysm clippings on actual patients.

TABLE.

Seventeen Residents Testing Aneurysm Simulator

Institution	N/(Average years of experience)
UIC	7 (4.5)
Weill Cornell	4 (5.5)
University of Chicago	6 (2.0)
Total	17(3.9)

Number of residents performing the aneurysm clipping simulation divided by institution and average years of experience.

Detailed responses concerning the residents’ use of the simulator are given in Figure 5. On a 5-point Likert scale, 47% of participants rank their knowledge of the aneurysm clipping procedure above level 4. About 64% of participants agreed that the ITACS is a useful education tool, whereas 12% disagreed. A total of 64% of participants believed that the ITACS can increase their understanding of aneurysm anatomy, whereas only 17% disagreed. About 59% of participants feel that on the whole, the aneurysm simulator will help them in preparing for aneurysm clipping surgery if they have time to rehearse the same procedure on a patient-specific model. Twelve percent agreed the haptic sensation produced by the simulator is identical to the one encountered in real surgery. A total of 71% of participants felt that the aneurysm simulation module would help define which approach should be used to access the aneurysm safely, whereas 23% disagreed. About 47% of participants agreed that the 3-D anatomy on the simulator represents the real anatomy, while 23% disagreed and 29% were neutral. On the Likert scale, 76% of participants rank their ability to operate the haptic stylus above 3, whereas24% ranked it below 2. So, it seems that the participants need more practice with the haptic stylus to gain more familiarity with this new technology. In sum, the highest rated feature of Immersive Touch aneurysm clipping is its usefulness as a practical education tool (Likert mean value 3.88 of 5.0).

DISCUSSION

An analysis of the features and ongoing development of the Immersive Touch simulation of aneurysm clipping must be set in the context of other methods of planning or training for this complex operation. Preoperative planning begins with a good understanding of the shape and orientation of the aneurysm and its parent vessels in three dimensions. Traditionally this has been recreated mentally by surgeons from 2-D digital subtraction angiography (DSA),but now commercial software packages perform volume rendering of data from 3-D DSA,3-D CTA, or 3-D magnetic resonance angiography, and there are various methods for stereoscopic visualization^17-19 in early virtual clipping simulation based on 3-D CTA was able to predict with good accuracy whether a proposed clip selection and placement would completely exclude the aneurysm as judged by postoperative imaging, but the study was limited to very specific aneurysm and clip geometries.²⁰ The now-discontinued Dextroscope, formerly available from Volume Interactions (Singapore), is the stereoscopic VR neurosurgery simulator most often discussed in the literature.^{19, 21, 22, 12, 23-27} Wong et al²⁸ loaded the Dextroscope with patient-specific 3-D CT data for the skull bone and angiography, then fused the data sets to demonstrate head positioning, craniotomy, trajectories to reach various aneurysms, and the angle of clip application. However, the “clipping” only involved placing the clip in the proper plane with no haptic interaction or visual deformation of the aneurysm.

Nakabayashi and Shimizu²⁹ developed a multilayer image segmentation and fusion method for patient-specific stereoscopic VR planning of aneurysm clipping at lower cost than the Dextroscope. They achieved VR model-creation times that were fast enough to plan emergency surgery for aneurysmal subarachnoid hemorrhage (e.g., arteries, 5 min; arteries, venous system, and skull, 15 min). Strategies selected in 3-D VR proved correct for 54 patients with no complications. The visualization was used for studying arterial structures and planning the approach, but clip selection is not separately discussed.

Image-based physical biomodels were the first technology to simulate patient-specific aneurysm clip application with tissue deformation and tactile feedback. An early computer graphics model for basilar artery aneurysm clipping included deformation, but was not patient-specific and lacked haptic feedback.³⁰ The physical models are created by a variety of additive fabrication and rapid prototyping technologies, including stereo lithography and 3-D printing.^31,32 The earliest biomodels by D’Urso et al³³ and Wurm et al³⁴ used 3-D CTA or MRA with stereo lithography to produce non-deformable solid replicas of aneurysms and vessels. These were found useful for selecting the best aneurysm clip, though not for simulating clipping itself. This changed with Kimura et al, who used stereo lithography to produce hollow aneurysm models of soft elastic silicone, so that the model aneurysm neck can be occluded with an actual clip and the patency of parent vessels checked with a vascular endoscope. Clips selected by these models proved to be the same ones used in the actual surgery with only minor adjustments (e.g., booster clips). A further effort by Wurm³⁵ used 3-D printing to produce similar soft aneurysm models, but went beyond Kimura by modeling the skull to simulate the approach. But even with Wurm’s 3-D printer, the models were time consuming and expensive to produce (1.5 weeks and 2000 Euro). Therefore, Mashiko et al³⁶ used the low-cost 3-D printer “UP Plus” (now superseded by UP Plus 2 at less than $2,000; Beijing Tier Time Technology Co., Ltd., pp3dp.com) to rapidly produce both solid and hollow aneurysm models, with good results.³⁷ Solid printed models made of a crylonitrile-butadiene-styrene(ABS) can be created from preoperative imaging in 2 hours or less, making them relevant for emergency surgery. The hollow elastic models are made by curing a liquid silicone layer applied to the solid models to represent the vessel walls, and then dissolving the ABS core, a process requiring 14-24 hours.

Currently there is no fully immersive haptics-based virtual reality aneurysm simulation besides the ITACS.³⁸ Recently the Selman Surgical Rehearsal Platform(SSRP)produced by Surgical Theater, LLC (Cleveland, OH), was introduced with particular reference to patient-specific aneurysm clipping.³⁹ As a stereoscopic 3-D visualization platform that represents an aneurysm free from immediately surrounding brain tissues, the SSRP may be classified with the Dextroscope and the system of Nakabayashi and Shimizu, but it goes beyond these by simulating aneurysm deformation during virtual reality clip application.

One clarification on the subject of the SSRP and haptic feedback may be useful. Bambakidis et al state that an ideal aneurysm clipping simulation should include haptic feedback from simulated instruments interacting with simulated tissues: “haptic feedback from instruments used in microsurgery, with accurate characterization of tissue responses to manipulation, is essential in simulator design.”³⁹ However, this haptic ideal is not realized in the SSRP. The aneurysm neck indeed dynamically deforms and closes as the virtual clip is placed around it and then released. Nevertheless, this is only a display of computer graphics (not haptics rendering), since the aneurysm cannot be sensibly touched by the operator’s instrument. Moreover, the graphics refresh rate is extremely slow as there is a visibly noticeable lag, roughly on the order of a second, between each incremental closing motion of the clip blades and the visual response of the aneurysm neck and sac (as may be seen in the authors’ supplemental video). By contrast, a response time of less than a millisecond is required for a virtual reality model to detect a collision between virtual surgical instruments and tissues and then output smooth and realistic force feedback. The SSRP does not appear to approach this level of performance.

The ITACS module takes a fundamental physics-based approach to the patient-specific modeling of brain tissue and vascular anatomy with real haptic feedback. This includes sensing the wall of the aneurysm, the surrounding brain tissue, the parent artery, the deformation of the aneurysm with release of the clip, the vessel pulsation, and the ability to mimic aneurysm rupture. All this gives the ITACS certain features not present in the SSRP as currently reported. Sensory feedback is the essence of the aneurysm clipping training and should be reproduced in simulation, since the complexity of aneurysm clipping training lies in these fine tactile details.

In our study, the residents found that the immersive 3-D visualization of the ITACS was very helpful in all the steps of virtual surgery. The residents could identify the important arteries, including the proximal (M1) and distal (M2) branches of the MCA bifurcation aneurysm. Based on the survey results, the senior residents rated this feature more highly than junior ones. The highest scoring survey question concerns the overall usefulness of the ITACS simulator, and it rated higher among senior residents. The participants gave high marks on the ability of the simulator to build on residents’ current understanding of aneurysm anatomy. The question whether a patient-specific aneurysm simulator would help them in preparing for the aneurysm clipping surgery if they have time to rehearse on a model of the same patient was also rated highly by residents, as was the simulator’s ability to help them plan the approach to get to the aneurysm. The participants rated the simulator to be quite engaging as well. With the exception of four residents the remaining participants felt that the simulator would help them plan their craniotomy tailored to each patient anatomy. When asked to what extent the 3-D anatomy on the simulator mimics real anatomy, the scores were moderate. This is another research area that needs attention. More realism in graphics and haptics with available computing power is an area of active research, and while we have come a long way, more needs to be done. The ability to operate the haptic stylus was rated a bit low because we used the default stylus setup, but our remedy for this is a custom pressure sensor to simulate gradual clip release and closure, as outlined below.

Residents raised an important concern about haptic interaction between the virtual aneurysm clip and aneurysm that has suggested a revised engineering approach. While most residents had difficulty feeling the collision of the clip with the aneurysm neck and achieving closure, others noted that the simulated aneurysm neck closed realistically, but that the magnitude of the force feedback from the contact of the clip with the neck is lower than in live surgery. As a remedy for both problems, the initial force of the open clip contacting the aneurysm neck can be increased by use of a more powerful haptic device, the Geomagic Touch X (formerly Sensable Phantom Desktop) instead of the Geomagic Touch (formerly Sensable Phantom Omni) used by the residents. The aneurysm model itself can be made more resistant to deformation during the early phase of clip contact and closure to satisfy the simulator user. Then as the clip is gradually released by the user reducing finger pressure on a custom sensor on the stylus, the stiffness of the virtual aneurysm can be automatically reduced to complete clip closure by satisfying the requirements of the haptics algorithm that powers the ITACS. To the best of our knowledge, no other group is seeking this kind of haptic realism in virtual aneurysm clip application.

In future developments with the ITACS, patient-specific immersive 3-Dstereoscopic virtual images can be studied and analyzed with respect to an aneurysm’s location, size, and orientation, as well as its relationship to the proximal vessels and other surrounding structures (anterior clinoid process, optic nerves, occipital condyle, etc.). From this, a surgical strategy can be planned with regard to optimal patient positioning, size and location of the craniotomy, retraction of brain parenchyma, avoidance of eloquent brain regions, and proper selection of the aneurysm clips. Detailed immersive 3-D pre surgical planning or simulated surgery with ITACS could theoretically improve the surgeon’s performance and thus patient care. While it is difficult to assign a statistically proven value to the benefit of this technology, we sought to assess its current usefulness in the training of neurosurgical residents as a guide to further developments.

The role of haptics in neurosurgery simulation, including Immersive Touch modules,^{22, 40, 41,} continues to be a topic of discussion. A limitation of this study is the use of questionnaire to access the response of the residents. We are planning on performing validation experiments with experienced neurosurgeons as well as planning a prospective study with patient-specific data, where residents can perform pre-operative surgical rehearsal; this will be correlated with intra-operative results, and resident learning curve will be assessed as well.

CONCLUSION

The ITACS module offers a training environment that closely approximates the hands-on experience that residents gain in the operating room in regard to designing a craniotomy, early recognition of aneurysm geometry and surgical projection, and selection and placement of aneurysm clips.

Supplementary Material

Supplemental Video. Supplemental Digital Content, Video 1.

Video that demonstrates the immersive virtual reality technique of clipping a middle cerebral artery aneurysm, 1.35 minutes, 147 MB.