Virtual Reality Assisted Cardiac Catheterization

Abstract

Cardiac catheterization is a delicate strategy often used during various heart procedures. However, the procedure carries a myriad of risks associated with it, including damage to the vessel or heart itself, blood clots, and arrhythmias. Many of these risks increase in probability as the length of the operation increases, creating a demand for a more accurate procedure while reducing the overall time required. To this end, we developed an adaptable virtual reality simulation and visualization method to provide essential information to the physician ahead of time with the goal of reducing potential risks, decreasing operation time, and improving the accuracy of cardiac catheterization procedures. We additionally conducted a phantom study to evaluate the impact of using our virtual reality system prior to a procedure.

1. INTRODUCTION

A conventional cardiac catheterization procedure consists of a physician accessing a larger artery, generally the femoral or iliac artery in the upper thigh area¹, and inserting a catheter. This catheter is then guided through the vessel into the heart. Due to the difficulties with the procedure, intra-operative imaging systems, such as X-ray fluoroscopy, are also often used², contributing ionizing radiation to both patients and physicians³. This is far from the only possible issue; however, additional challenges are avoiding unnecessary rupture or puncture of blood vessels, which can cause thrombosis⁴. Although these challenges result in risks for the patient, the procedure is still a widely used procedure for cardiac catheterization. Reasons for the procedure vary. Physicians perform cardiac catheterizations for motives such as locating blockages in blood vessels, performing heart biopsies or angioplasties, treating irregular heart rhythms, closing holes within the heart, or various other purposes⁵. Currently, the most common cardiac catheterization method is to access the femoral artery⁵, however, patient shift makes relying on pre-operative CT data inaccurate, resulting in additional intra-operative imaging. Using conventional techniques, and no additional guidance tools, 72% of femoral artery punctures required greater than one attempt⁶.

Recently, virtual reality (VR) has been explored to better train clinicians for various procedures including cataract, laparoscopic, and orthopedic surgeries^7–9. Using VR to better retain spatial information has been proven to be more effective than conventional methods (such as going through 2D images slice by slice) by quantitatively increasing spatial knowledge retention for physicians in training. When physicians in training were given a 30-minute VR cardiac experience they scored 23% higher on a multiple choice quiz regarding visual-spatial as well as cardiac anatomy when compared to their peers that used conventional, independent study methods^{10, 11}. VR-based training regimens provide an exciting new avenue for physicians in training to be able to perform various procedures in a low-stress environment, while providing exact repeatability and the ability to adapt each procedure to best suit the needs of each individual physician in training. These strengths, as well as others allow for VR training regimens to often outperform conventional methods^12–16.

Additionally, the usage of VR angiograms has been proven to aid in various challenges such as spatial and depth perception¹⁷. VR training simulators present a relatively low-cost, adaptable, and easily supervised method to provide preparation for various surgical procedures and will likely become the training standard as VR technology continues to improve and increase in availability. Several major medical centers have already implemented various augmented reality assisted training techniques¹⁸.Due to this, it is imperative to continuously push the boundaries of VR planning and training simulations to better understand their capabilities and weaknesses. To optimize operation time, we present a procedure for visualizing all important data in a fully virtual world, allowing for major decision making to take place before the operation even begins.

2. METHODS

2.1. Data Processing Workflow

Our method begins with obtaining either standard CT or CT-Angiogram DICOM images for the patient. These are then segmented in 3D Slicer¹⁹. The regions segmented include the aorta, femoral artery, and any additional vascular features desired. Additional segmentations include the skeleton of the patient as well as the catheter, and we have chosen to include a translucent segmentation of the skin of the patient to provide as much spatial information as possible to the physician. After satisfactory segmentations are extracted, we use the video game development software called Unity to create the virtual scene that will be visualized. However, since there is no preoperative imaging of the catheter, we propose that precise dimensions of the catheter be used to create an accurate copy within the virtual Unity world to make use of the catheter while planning. This model can be generated by importing a high-resolution CT of the catheter into 3D Slicer.

2.2. Three-dimensional (3D) Visualization and Virtual Hand Interaction

Once the segmentation labels are completed within 3D Slicer, we then export the files in the OBJ format into a Unity scene. Custom virtual hands were developed in Unity with the help of the Oculus Toolkit SDK to interact and move the various segmentation labels as desired by the physician. These hands are fully customizable in terms of transparency, size, shape and their ability to grip certain objects. Our VR headset of choice was the Oculus Rift S, (Facebook Inc. Menlo Park, CA, USA) due to the very high accuracy and precision of the head-mounted display and controllers²⁰, as well as its ability to connect to a computer through a wired connection. This allows for complex computations such as processing and visualizing highly textured 3D models to take place on a computer. This improves visualization of the segmentations within the Unity scene while eradicating any need to build the Unity project to another device. Additionally, allowing all computations to take place on a desktop allows for rapid Unity scene adjustments. Another benefit of a wired system is that the head-mounted display itself is much smaller and lighter as there is no need to integrate bulky and heavy visualization and processing chips¹⁷. Thereby, using this system, the physician can utilize the handheld controllers to move the virtual catheter within the virtual artery and visualize the procedure ahead of the real-world operation.

2.3. Implementation

The Oculus is a comfortable fit, even over long periods of use. We have also implemented some quality-of-life adjustments, such as the ability to move in the virtual world either using joysticks attached to the handheld controllers or by physically walking through the scene. This was done to better accommodate rooms with less space for movement. Interacting with each segmentation is fluid, instant, intuitive, and the entire process, (excluding the manual segmentation) can be easily completed within an hour. Moreover, the computation power needed is minimal: this project was completed using a Ryzen 5 1600 processor, in conjunction with an NVIDIA GTX 1060 6GB graphics card.

2.4. Phantom Evaluation

The effect of preparation using VR versus viewing the standard 2D anatomical planes was evaluated using four phantoms and two users. The four phantoms were constructed from a 3% agar solution, with a 7 mm vessel of 3% agar and iodine. The iodine was used a contrast for X-ray imaging. One user was experienced in viewing 2D anatomical images while the other was inexperienced. Each user would first receive two minutes to view the phantom using either the 2D plane views or the VR model (Table I). They would then insert two needles into the phantom, with the goal of embedding each needle into the vessel. Once both needles were inserted, a 3D X-ray computed tomography (CT) image of the phantom was acquired using a Ziehm Vision RFD 3D (Ziehm Imaging Inc., Orlando, FL) to estimate the needle insertion accuracy. The 3D volume was 512 × 512 × 512 voxels, with an isotropic spatial resolution of 0.3125 mm. Each user was allowed to take as long as they wished when placing the needle.

Table 1:

User preparation by phantom

Phantom Number	User A	User B
1	VR	2D Planes
2	2D Planes	VR
3	2D Planes	2D Planes
4	VR	VR

3. RESULTS

An example of a sample virtual world is shown below in Figure 4. We observe that adding more segmentations and information increases the available information for the physician, but also may serve to clutter the area.

Figure 4.

3D visualization without a surrounding skin (A) and with the skin (B).

3.1. Scene Creation

The Unity scene was created mainly through the manipulation of prefabricated assets available within the Oculus Integration SDK available through the Unity store. There are a few implementation requirements which need to be considered within the virtual world. There must be a virtual “floor” within the scene to ensure that the headset wearer does not continuously fall through the virtual space. Additionally, the scene requires a camera through which the Oculus headset will see, as well as a pair of virtual hands with which the physician can manipulate the virtual objects through the use of the handheld controllers (see Figure 5).

Figure 5.

Virtual hand interaction and control of the virtual world in two different situations; before moving a segmentation label (A) as well as after (B).

3.2. Phantom Experiments

Results from the phantom evaluations are shown in Figure 6. The quantitative data suggests that observing each user independently, the use of VR tended to reduce the distance of the needle tip to the edge of the target vessel, suggesting an improvement to the user’s spatial awareness after preparation with the VR environment. However, additional phantoms and users will be required to state any trends conclusively.

Figure 6.

Graphs depicting the average distance per phantom per user. The X-axis is the trials using either VR or standard 2D anatomical planes. The Y-axis is the distance of the needle to the edge of the artery.

Qualitative examples are shown in Figure 7. A red outline delineates the target vessel, while the needle tip is marked by a black arrow. In Figure 7A and 7B, the user successfully hit the vessel with the tip of the needle using both the 2D image and VR preparations, respectively. In Figure 7C and 7D, the user missed the target vessel by 10.7 mm and 6.0 mm, respectively.

Figure 7.

Phantom experiments and needle position evaluation using X-ray computed tomography. The vessel is outlined in red and the needle tip is denoted by the black arrow. A & B: User A after preparation with the 2D images and VR headset, respectively. C & D: User B after preparation with the 2D images and VR, respectively.

4. DISCUSSION

Full immersion in three dimensions can prove invaluable in planning procedures ahead of time. Currently, our visualization method uses CT data and can be extended to other images such as CT angiography (CIA), MRI, or MR angiography (MRA). Multimodality imaging data, such as CT and MRA, can be registered and fused to provide more complementary information (such as bone structure, vasculature, or muscle). The 3D visualization and virtual hand interaction can enhance the manipulation and navigation of various organs or tissues.

Moreover, although the phantom experiment findings remain preliminary, they do demonstrate an encouraging trend, namely that for each user, the use of VR decreased the distance from the needle tip to the artery edge. The large fluctuations in accuracy between phantoms can be explained by significant variations within the phantoms causing certain phantoms to be more difficult to accurately orient oneself than others.

While employing virtual reality measures to aid in medical procedures is not a new idea, there is still much discourse as to how useful VR can be in its current state due to its technical limitations²¹. While this is a valid area of discussion, we would argue that the possible benefits of VR integration can easily surpass the limitations in expanding its use for cardiac catheterization integration. Moreover, our VR system is rather popular with our users, both stating that they felt more confident and comfortable whilst using the VR system in comparison with the conventional 2D methods. They also stated that they believed it would take less time to understand the 3D structure of the patient while using the VR method versus the standard 2D method.

Additionally, the use of VR environments allows for high levels of personalization. While not delved into here, a VR system could allow for various relevant patient information to be displayed onto the headset within the virtual world, such as blood pressure, BPM (beats per minute), or various other relevant patient vitals. With the development of a graphical user interface (GUI) within Unity, it becomes possible to monitor patients’ vitals in the real world through the VR headset.

In order to build on our current research, we plan on additional projects arising from this work including, for example, virtual-reality gloves that may take the spot of the handheld controllers through the use of an optical tracking system. We are also attempting to streamline the procedure into use as an intraoperative as well as preoperative system with help from an intraoperative imaging device such as X-ray.

Limitations of the study include the low number of user tests available at the time and the simplistic nature of the phantoms, in addition to requiring a non-trivial understanding of 3D Slicer and Unity in order to properly set up the system. Future work will expand the number of phantoms used, as well as increases to the complexity of the task required.

5. CONCLUSION

The use of virtual reality is a promising step forward in the visualization, planning, and training required for a cardiac catheterization operation. Our visualization and virtual interaction system maintain promising avenues for further expansion. The potential gain from customizable 3D visualization in a virtual world merits further study.

Figure 1.

An example of the femoral artery segmentation (dark red) position in relation to the pelvis (blue).

Figure 2.

An example of a 3D Slicer segmentation environment. The three black boxes show the body from the three standard views (i.e., axial, sagittal, and coronal). The blue box shows the 3D visualization of the segmented tissues.

Figure 3.

A flowchart describing the main aspects of the virtual reality procedure.