Augmented reality and three-dimensional printing in percutaneous interventions on pulmonary arteries

Abstract

Background

Percutaneous pulmonary interventions require extensive and accurate navigation planning and guidance, especially in regard to the three-dimensional (3D) relationships between anatomical structures. In this study, we are demonstrating the feasibility of novel visualization techniques: 3D printing (3DP) and augmented reality (AR) in planning transcatheter pulmonary interventions.

Methods

Two patients were qualified for balloon pulmonary angioplasty (BPA) for treatment of chronic thromboembolic pulmonary hypertension (CTEPH) and stent implantation for pulmonary artery stenosis, respectively. Computed tomography images of both patients were processed with segmentation algorithms and subsequently submitted to 3D modelling software. Microsoft HoloLens^® AR headsets with dedicated CarnaLife Holo^® software were utilized to display surface and volume rendering of pulmonary vessels as holograms.

Results

Personalized life-sized models of the same structures were additionally 3D-printed for preoperative planning. Holograms were shown to physicians throughout the procedure and were used as a guidance and navigation tool. Operative team was able to manipulate the hologram and multiple users of the AR system could share the same image in real time. Clinicians expressed their satisfaction with the quality of imaging and potential clinical benefits.

Conclusions

This study reports the potential value of AR in pulmonary interventions, however, prospective trials need to be conducted to decide on whether novel 3D visualization techniques affect perioperative treatment and outcomes.

Introduction

Percutaneous cardiovascular interventions on pulmonary arteries are still associated with major technical challenges and require extensive planning and intraprocedural precise visualization methods. Pulmonary vascular tree is characterized with complex, three-dimensional (3D) anatomy that is difficult to visualize precisely with planar angiography. Chronic thromboembolic pulmonary hypertension (CTEPH) is related to presence of multiple post-thrombotic lesions disseminated in different branches of pulmonary arteries. Promising results of balloon pulmonary angioplasty (BPA) were reported in treatment of persistent or recurrent CTEPH (1,2). One of the most important requirements for effective treatment of CTEPH with BPA is an accomplishment of effective dilatations in the largest possible number of segmental and subsegmental pulmonary arteries (3). Ogo et al. reported successful use of cone beam computed tomography in supporting BPA procedures with 3D image reconstructions (4). However, currently available tools have limited availability and there is still a need for novel techniques that would represent patient’s anatomy especially accurately in regards to 3D relationships between structures.

In recent years, several new approaches emerged to provide 3D visualization of medical data, including 3D printing (3DP), augmented reality (AR) or virtual reality (VR). 3DP and AR are considered especially useful since they can guide physicians during the procedures. However, literature is very limited and consists of only few pilot and feasibility studies. AR systems have been researched in many disciplines including urology (5), neurosurgery (6), while in cardiology AR was found feasible i.e., in revascularization of coronary chronic total occlusion (7) and transcatheter aortic valve implantation (8,9). 3DP is much more explored with a wide range of applications in cardiology field (10), including preclinical studies of printing pulmonary artery models (11). Having said that, no feasibility studies evaluating use of neither 3DP nor AR were performed in clinical setting with patients undergoing transcatheter pulmonary interventions. Thus, the aim of this study was to evaluate the usefulness of 3D printing and AR system in planning and guidance of transcatheter pulmonary interventions based on treatment of 2 cases.

Methods

Case 1

A 53-year-old male with persistent CTEPH despite previous treatment with pulmonary endarterectomy, chronic kidney disease in stage 3a, chronic obstructive pulmonary disease, hypothyroidism, and splenectomy performed years ago was referred for BPA. He was administered specific medical therapy with riociguat at the dose 2.5 mg three times a day (TID). He presented symptoms of NYHA functional class III on admission. N-terminal prohormone of brain natriuretic peptide (NT-proBNP) plasma level concentration was increased to 818 pg/mL. In the 6 minutes walk test (6MWT) he covered the distance of 318 m with severe dyspnea (5 points in Borg index) and desaturation from 93–90% (no oxygen supplementation). Right heart catheterization (RHC) performed in accordance to current standards (12) after pulmonary endarterectomy confirmed severe pulmonary hypertension (mPAP: 46 mmHg) with normal cardiac index (CI: 3.08 L/min/m²), normal capillary pulmonary wedge pressure (PCWP: 9 mmHg) and increased pulmonary vascular resistance (PVR: 6.42 Wood units). The patient suffered from chronic kidney disease and his glomerular filtration rate (GFR) was decreased to 42 mL/min/1.73 m². Angiography of both pulmonary arteries revealed residual clots in segmental and subsegmental branches.

The first BPA session was performed on eight segmental and subsegmental arteries in the right lobe. There were no complications during and after first procedure. According to poor renal function, the reduction of contrast medium was planned in the 2nd session. 3D-printed model of the patient’s left pulmonary artery was developed in order to better understand the anatomy of the lesions and check the feasibility of the procedure. During the second BPA session, an AR system was used as an additional tool. After the series of 6 BPA procedures, patient’s functional capacity improved to NYHA II class. Controlled RHC presented reduction of mPAP to 38 mmHg and PVR 4.38 Wood units. The improvement of renal function (eGFR >60 mL/min/1.73 m²) and decrease of NT-proBNP to 324 pg/mL was noted.

Case 2

A 67-year-old male with pulmonary precapillary hypertension associated with idiopathic mediastinal fibrosis and proximal ring-like stenosis of pulmonary arteries resulting from abnormal connective tissue consolidation in mediastinum, with coronary artery disease, paroxysmal atrial fibrillation was admitted for interventional treatment. The patient was administered specific medical therapy with sildenafil without any significant clinical and hemodynamical improvement. He presented symptoms of NYHA functional class III on admission and reported deterioration in health related quality of life. The NT-proBNP plasma level concentration was normal. He covered the distance of 363 m in 6MWT with very severe dyspnea (8 points in Borg index) and without any significant desaturation (from 98% to 97%). RHC performed at rest confirmed mPAP increase to 29 mmHg, decreased CI to 2.38 L/min/m², normal PCWP 14 mmHg and borderline PVR 3.01 Wood units. During the catheterization, a gradient of systolic PAP (from 53 to 38 mmHg) was noticed when crossing the ring-like stenosis in pulmonary lobar artery by Swan-Ganz catheter.

During the first session, a selective angiography confirmed proximal stenosis of left artery of lower lobe. An assessment with pressure wire (Verrata, Philips Volcano) confirmed significant gradient across the narrowed artery (P_dist: 19 mmHg; P_prox: 28 mmHg; P_dist/P_prox: 0.67). Intravascular ultrasound (IVUS) showed significant reduction (over 50%, area of 23 mm²) of the lumen of pulmonary artery of left lower lobe. Self-expanding stent CID (CID S.p.A, Alvimedica Group, Istanbul, Turkey) 12/40 mm was implanted in the occluded vessel. After implantation, a significant improvement in functional pressure assessment (P_dist: 29 mmHg, P_prox: 30 mmHg; P_dist/P_prox: 0,96) and increase in the area of the vessel to 72 mm² were confirmed in IVUS.

The second session was performed in the right lower lobe artery, where significant ring-like stenosis was confirmed in 3D rotational pulmonary angiography. AR visualization was used for navigation of stenotic lesion and crossing it by guidewire. Pressure wire confirmed significant gradient across the lesion (P_dist: 19 mmHg, P_prox: 28 mmHg; P_dist/P_prox: 0.67) and IVUS showed the >50% reduction in proximal right lower lobe pulmonary artery. Self-expanding stent EPIC 12/40 mm (Boston Scientific, Marlborough, MA, USA) was implanted and post-dilatation of Advance 35LP 12/18 mm (Cook Medical, Bloomington, IN, USA) balloon was performed. There were no complications during procedure and improvement of pressure gradient (P_dist: 25 mmHg, P_proxd 26 mmHg; P_dist/P_prox: 0.96) was confirmed. Control RHC performed after 2 invasive procedures with stent implantation confirmed decrease of mPAP to 25 mmHg.

Image acquisition and processing

All images were registered with Philips Brilliance Big Bore CT (Koninklijke Philips Electronics NV, Eindhoven, Netherlands) scanner with 120 kVp protocol and 1.00 mm slice thickness. Studies saved in Digital Imaging and Communications in Medicine (DICOM) format were imported into 3D Slicer (version 4.8.1) for further processing (13). Built-in semi-automatic algorithms were used for initial segmentation which results were subsequently manually refined and exported into 3D meshes. In Blender 3D modelling software (open-source; Blender Foundation, Amsterdam, Netherlands) pulmonary arteries were divided into parts including segmental and subsegmental branches feeding upper lobe, lingula/middle lobe and lower lobe, respectively (Figure 1).

Figure 1.

Image processing for surface rendering. (A) Raw CT scan; (B) CT with segmentation overlay mask (pink); (C) exported segmentation result: pulmonary artery with segmental branches divided into differently colored parts. CT, computed tomography.

Results

3D printing and AR

3D objects require preparation specifically for 3D printing, including making meshes watertight and ensuring that multipart printouts will fit each other. To achieve that, we used Boolean-based approach described elsewhere (14). Ultimaker 2+ (Ultimaker, Geldermalsen, Netherlands) 3D printer with polylactic acid (PLA) material and Form 2 (Formlabs, Somerville, MA, USA) with transparent resin, were used for printing all parts which were assembled together later (Figure 2).

Figure 2.

3DP models fabricated with white polylactic acid (A), flexible resin (B) and multi-part polylactic acid model (C). 3DP, three-dimensional printing.

3DP models were developed and delivered to the surgical team prior to the procedure. Clinicians reported that there was no difference in terms of quality or utility of 3DP models depending on used material and fabrication method. Also, AR system was provided a few hours before planned operations. Developers of the AR system were on site dedicated to assisting physicians in case of any issues and they conducted a short introduction before going into the surgical room.

CarnaLife Holo^® (MedApp, Kraków, Poland) software was used to visualize pulmonary arteries in holographic form, utilizing both volume and surface rendering (Figure 3). Operative team was able to manipulate the hologram (moving, zooming, rotating, cutting, changing transfer functions) with gestures and voice commands. Team performing the intervention had the AR headset on throughout the procedure and was able to refer to the hologram at any time. As reported, the hardware nor the method of visualization did not obstruct the vision. As for hardware, three Microsoft HoloLens^® head-mounted displays were used. The AR system enables users to visualize medical datasets loaded directly from DICOM files or connected PACS system and utilized software implements the ability to share the same hologram between multiple users, so that all members of the operative team could discuss the image at the same time (Figure 4) (Online video: https://www.youtube.com/watch?v=CxYsZGB-CCw).

Figure 3.

First person view of holograms displayed throughout the procedures. (A) surface rendering of pulmonary artery with selected branches highlighted in a different color (blue, red, green); (B) volume-rendering-based hologram of pulmonary artery and its branches.

Figure 4.

Operative team during one of the procedures. The AR software used in these cases enables multiple users to share the same image in real time. AR, augmented reality.

Discussion

We have conducted a feasibility study of 3D printing and AR for pulmonary interventions in a clinical setting with two real patient cases. 3D models in physical and holographic form were used during the process of preparing for surgery, and while performing the procedure. 3D-printed pulmonary artery was used primarily as a preoperative planning tool, while AR system primary use was for navigation and guidance. However, both were available for the whole process.

Head-mounted AR displays have been found to be safe and clinically useful in percutaneous procedures. Opolski et al. revealed that patients in group treated with AR assistance had lower contract exposure with no risks to procedural safety (7). Clinical cases published recently by Lodziński et al. (15) and Rymuza et al. (8) confirm accuracy of holograms for cardiovascular interventions. Regarding 3DP, there is strong evidence supporting its use in planning difficult transcatheter interventions. However, only a few describe specifically pulmonary procedures. El Sabbagh et al. showed that 3DP model was useful in preprocedural assessment of location of pulmonary obstruction and stent sizing (10). This is the first report of AR application in interventional treatment of non-congenital pulmonary artery branches stenosis caused by fibrotic lesions or thrombotic material. Potential role of AR and 3DP in improvement of BPA technique is related to better selection of target lesions, providing more complete revascularization, avoiding vessel injury caused by oversized balloons and decreasing the volume of contrast medium. In procedures requiring stent implantation, augmented 3D visualization is crucial in stent sizing and positioning to avoid unnecessary closure of side branches which are prominently developed in pulmonary circulation.

Our study has some limitations that have to be taken into consideration. Primarily, this is a feasibility study that can be treated as a proof of concept. Although opinions of all physicians engaged in the process were positive about its accuracy, usability and potential effectiveness in routine use, it is yet to be decided how the use of these technologies will affect perioperative parameters and outcomes. Prospective trials in the future will decide if novel 3D visualizations techniques are cost-effective.

There is also a debate when and where we should use 3DP instead of AR. In our study, operators had access to both technologies at all times and they felt it was more natural to use 3D printed models prior to the surgery for additional tactile experience and more straightforward manipulation, while during the procedure they almost exclusively used AR which allowed them to visualize structures with gestures and voice commands.

Thinking about further development of these visualization techniques, it seems that AR might offer more powerful toolset in several years if progress in hardware and other areas like artificial intelligence, advances simulations continues. When AR surpasses the point of being able to mimic tissues mechanical properties—which is major limitation of 3D printing—it will become able to simulate not only physical mechanisms like fluid simulation, but also perform whole mock procedures. Until then, 3DP seems a great addition when planning and measuring preoperatively, and is even more encouraging when used for education.

Conclusions

This technical note suggests that in cases with changed and difficult anatomy of pulmonary arteries, imaging with CT aided by 3D printed model and AR is helpful. The strategy with novel 3D visualization techniques has potential to reduce the dosage of contrast medium and minimize the risk of potential nephropathy.

Ethical Statement: All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. Informed consent was obtained from all individual participants included in the study.