Abstract
The safety and efficacy of minimally invasive surgery relies on visual information. We aimed to develop an integrated image navigation system (RoboSurgeon System) that combines head-mounted displays (HMDs) with multiple image modalities, and assessed its feasibility in 5 prostate cancer patients who underwent gasless single-port endoscopic radical prostatectomy. A robotically manipulated transrectal ultrasound (TRUS) system was used. In all cases, preoperative magnetic resonance (MR) images and intraoperative real-time images of an endoscope, TRUS, and HMD-mounted camera were integrated and displayed synchronously on each HMD in a four-split screen mode during the entire process. The TRUS helped identify the boundary with the adjacent structures endoscopically in reference to MR images. There were no negative incidents in intraoperative or postoperative courses. Integrated image navigation using HMDs as individualized monitors is feasible in the natural ergonomic position and may be beneficial to identify correct dissection planes. The efficacy of the RoboSurgeon System deserves further evaluation.
Keywords: head-mounted display, image navigation, minimally invasive surgery, prostate cancer, radical prostatectomy, transrectal ultrasound
Introduction
As a result of dramatic advances in surgical equipment, robot-assisted laparoscopic surgery using the da Vinci Surgical System (Intuitive Surgical, Sunnyvale, CA, USA) has become a popular form of minimally invasive radical prostatectomy (RP). However, procedures in laparoscopic surgery are highly dependent on visual information because of the lack of tactile feedback [1]. Therefore, intraoperative image guidance is essential to improving the efficacy and safety of RP. Recently, real-time transrectal ultrasound (TRUS) evaluation during laparoscopic RP and robot-assisted RP has been reported to be useful for the intraoperative identification of anatomical structure [2–5]. Additionally, TRUS offers real-time image registration in synchronization with preoperative magnetic resonance imaging (MRI) using an MRI/TRUS-fusion technique [6]. However, when multiple imaging modalities are simultaneously used in endoscopic surgery, multiple display monitors for each modality are required, and physicians have to direct and move their gaze to each. It would be ideal if each surgeon and assistant involved in the operation could constantly and simultaneously monitor all types of visual information in a natural ergonomic position. A head-mounted display (HMD) is an individualized imaging monitor that removes the need to look at stationary overhead or floor-standing monitors [7, 8]. Simultaneous and synchronous monitoring of preoperative images and intraoperative real-time images through a HMD would provide seamless and efficient surgical navigation. We have developed a novel, integrated image navigation system (RoboSurgeon System) that combines a three-dimensional organic electroluminescent HMD unit (HMS-3000MT; Sony Corporation, Tokyo, Japan) with various image modalities, including endoscope, TRUS, and MRI, through interface multiplexers.
Aim
We present the initial experience of integrated image navigation using the RoboSurgeon System and assess its feasibility in gasless single-port endoscopic RP, a minimally invasive surgery via a single port without CO2 gas insufflation [9, 10].
Material and methods
This was an institutional review board-approved study, with each patient providing informed consent. Between April 2013 and August 2013, 5 men with clinically localized prostate cancer underwent gasless single-port endoscopic RP using the RoboSurgeon System and a robotically manipulated TRUS system. All patients underwent preoperative MRI using a 1.5-Tesla imager (Achieva; Philips, Best, The Netherlands) [11] and intraoperative real-time TRUS.
HMS-3000MT or its prototype was used for the HMD units, which were connected to the endoscope system (Visera Elite and Endoeye Flex 3D; Olympus Medical Science, Tokyo, Japan), the ultrasound console (Prosound; Hitachi Aloka Medical, Tokyo, Japan), a video camera attached to the surgeon's HMD (HMD-mounted camera), and the electronic health record system (Fujitsu FIP, Tokyo, Japan) (Photo 1). MR images stored in the record system were displayed on the HMDs for preoperative reference. Visual information was split using the HMS-3000MT's built-in imaging splitter, and the composite image was output into multiplexers (VPM-H1, MEDIAEDGE Corporation, Hyogo, Japan). The images were integrated using a four-split screen technique on the multiplexer and were displayed on the HMDs.
Photo 1.
External appearance of gasless single-port endoscopic radical prostatectomy using the RoboSurgeon System. The surgeon and assistants wear head-mounted displays
At each step of the operation, the visibility of processing objects was examined for each image modality. Intraoperative ultrasonographic evaluation of the prostate and surrounding tissue was performed using the robotically manipulated real-time TRUS system as follows: with an anesthetized patient in the lithotomy position or the supine position with leg straddled, a biplanar TRUS probe attached to a floor-mounted stepping and stabilizing unit (SURE-POINT; Bard Medical, Covington, GA, USA) was placed in the rectum. The probe was moved and adjusted automatically using a brachytherapy planning system (Oncentra Prostate; Nucletron, Veenendaal, the Netherlands) operated by a dedicated assistant (Photo 2).
Photo 2.
Robotic transrectal ultrasound system. Inset (left, above) shows a transrectal ultrasound probe attached to a floor-mounted stepping and stabilizing unit
The techniques and procedures of gasless single-port endoscopic surgery have been presented previously [8–10]. Briefly, a single port in the lower abdomen is made that is estimated to narrowly permit extraction of the specimen, and is prepared by placement of an Alexis wound retractor (Applied Medical, Irvine, CA, USA). All devices, including the endoscope, are inserted through the single port. While maintaining the intrapelvic working space using PLES retractors (Inomedics, Tokyo, Japan), specimens are removed using hand-held surgical devices through the single port without direct hand contact.
Results
Patient characteristics are shown in Table I. In all cases, the RoboSurgeon System, including the robotic TRUS, could be set up within 10 min. Preoperative MR images and intraoperative real-time images of the endoscope, TRUS, and HMD-mounted camera were available for each participant wearing HMDs during the entire process. All of the images displayed in the four-split screen mode had sufficient resolution for clinical application in the RP procedure (Photo 3).
Table I.
Patient demographics with clinical and pathological characteristics
| Variable | Data |
|---|---|
| Patients, n | 5 |
| Age, median (range) [years] | 67 (59–70) |
| PSA, median (range) [ng/ml] | 12.1 (4.6–16.4) |
| Prostate volume, median (range) [ml] | 25.8 (24.3–42.8) |
| Clinical tumor stage, n: | |
| T1c | 2 |
| T2 | 3 |
| Biopsy Gleason score, n: | |
| ≤ 6 | 1 |
| 7 | 2 |
| ≥ 8 | 2 |
| Pathological tumor stage, n: | |
| T2 | 5 |
| T3 | 0 |
| Pathological Gleason score, n: | |
| ≤ 6 | 0 |
| 7 | 5 |
| ≥ 8 | 0 |
PSA – prostate-specific antigen
Photo 3.
Four-split screen on the head-mounted display. Multiple integrated images are displayed simultaneously and synchronously. Endoscopic view is monitored as the main image. Transrectal ultrasound image, head-mounted display-mounted camera view, and preoperative magnetic resonance image are monitored as sub-images
The surgeon usually chose the endoscopic view as the main image of the four-split screen on the HMD. He also used either the panoramic view from the HMD-mounted camera or direct vision by glancing downward. Meanwhile, assistants situationally selected a main screen image from the endoscopic view, the TRUS view, and others. The images from the surgeon's HMD-mounted camera helped assistants share the surgeon's view, and they educated medical students on the process of the operation. In our cases, images displayed on the four-split screen could be chosen or arranged independently by each surgeon or assistant using foot pedal controllers.
Robotic TRUS images of each step are shown in Photo 4. Positive lesions on preoperative MRI were identified on real-time biplanar TRUS through the HMD screen. TRUS images helped identify the shape of the prostatic apex, the boundary between the prostate and the rectum or bladder, and the location of seminal vesicles, with reference to preoperative MRI. Tips of surgical instruments were identified as hyperechoic signals. Doppler ultrasonography was also helpful in identifying the dorsal vein complex and the neurovascular bundles. In our gasless RP, it was unlikely that intervening air interfered with the visibility of TRUS images.
Photo 4.
Real-time images of transrectal ultrasonography. A – The dorsal vein complex and the prostatic apex (sagittal view). B – The dorsal vein complex and the prostatic apex (transverse view). C – Positional awareness of the neurovascular bundles to preserve erectile function. D – Identification of the plane between posterior bladder neck and seminal vesicles. A scissors tip is identified as a hyperechoic signal (arrow). E – Release of Denonvilliers’ fascia (arrowheads) from the rectum. F – Division site of the urethra as close to the prostatic apex as possible to avoid excising excessive amounts of the striated sphincter. A scissors tip is identified as a hyperechoic signal (arrow)
Apex – prostatic apex, DVC – dorsal vein complex, NVB – neurovascular bundle, Pro – prostate, Rec – rectum, SV – seminal vesicle, Ure – urethra
In all cases, the RP procedure was successfully performed without rectal injury and operating times were within usual limits. The TRUS probe in the rectum did not hamper the surgical procedures in the pelvis. There were no notable negative outcomes or incidents in the postoperative courses. Physicians wore HMDs throughout the surgery and experienced no HMD-related adverse effects or discomforts.
Discussion
This is the first study on integrated image navigation using a HMD in RP. Our concept is novel in terms of the application of a HMD as an individualized monitor that simultaneously displays various types of imagery. The surgeon and his assistants could independently arrange image arrays on their HMD screens, depending on their roles. Unlike the conventional endoscope-assisted RP, the RoboSurgeon System provides panoramic vision and multi-screened vision as well as magnified stereoscopic vision. Integrated image navigation using the above visions would help with the accurate identification of the dissection planes, suggesting the potential to improve surgical safety and oncological and functional outcomes, which are in need of further investigation. Another potential role of the RoboSurgeon System relates to its educational use for novice physicians and medical students. Image guidance using multi-screened vision would enhance the learning curve of endoscopic RP [3].
Technological gains accelerate the further refinement of minimally invasive surgery. Recently, the image quality of HMDs has greatly improved, and their size has been reduced to a portable format [7, 12, 13]. The HMD of HMS-3000MT weighs 490 γ and offers a resolution of 1,280 × 720 pixels, which is sufficient for most medical purposes [14, 15]. However, the RoboSurgeon System has room for improvement. In the current study, the surgery was performed in coordination with a robotic-TRUS operator. Remote-control devices would enable a surgeon himself to manipulate the robotic TRUS in the same way as image arrays on the HMD screen can be arranged using a foot pedal controller [4, 5].
Among the present case series, one patient underwent gasless single-port endoscopic RP using a robotic endoscope holder (Aerovision; Skynet, Tokyo, Japan) that was driven by pneumatic actuators and synchronously tracked the surgeon's head movement with the sensor on the HMD (Photo 5) [16]. This robotic holder eliminates the need for an assistant who holds the endoscope and might experience hand tremor. Thus, the RoboSurgeon System is a developing but promising surgical guide system. Compared to the da Vinci Surgical System, which requires space for a bulky console, expensive equipment, and running costs [17], our RoboSurgeon System is compact and easy to affordably introduce in various types of endoscopic surgery.
Photo 5.
A robotic endoscope holder synchronously tracked the surgeon's head movement with the sensor on the head-mounted display
This study has several limitations, including the small sample size and the lack of a control arm. Although further study is needed to prove the actual benefits of the RoboSurgeon System, our experience suggests the clinical application of this system as a surgical image navigation unit for minimally invasive RP.
Conclusions
We have developed a novel HMD-based image-integrating system and introduced it into RP. This compact system enabled individualized image navigation for each participant. The usefulness of this system deserves further evaluation in various types of minimally invasive endoscopic surgery.
References
- 1.Rassweiler J, Frede T, Seemann O, et al. Telesurgical laparoscopic radical prostatectomy. Initial experience. Eur Urol. 2001;40:75–83. doi: 10.1159/000049752. [DOI] [PubMed] [Google Scholar]
- 2.Ukimura O, Gill IS. Real-time transrectal ultrasound guidance during nerve sparing laparoscopic radical prostatectomy: pictorial essay. J Urol. 2006;175:1311–9. doi: 10.1016/S0022-5347(05)00661-0. [DOI] [PubMed] [Google Scholar]
- 3.van der Poel HG, de Blok W, Bex A, et al. Peroperative transrectal ultrasonography-guided bladder neck dissection eases the learning of robot-assisted laparoscopic prostatectomy. BJU Int. 2008;102:849–52. doi: 10.1111/j.1464-410X.2008.07830.x. [DOI] [PubMed] [Google Scholar]
- 4.Hung AJ, Abreu AL, Shoji S, et al. Robotic transrectal ultrasonography during robot-assisted radical prostatectomy. Eur Urol. 2012;62:341–8. doi: 10.1016/j.eururo.2012.04.032. [DOI] [PubMed] [Google Scholar]
- 5.Long JA, Lee BH, Guillotreau J, et al. Real-time robotic transrectal ultrasound navigation during robotic radical prostatectomy: initial clinical experience. Urology. 2012;80:608–13. doi: 10.1016/j.urology.2012.02.081. [DOI] [PubMed] [Google Scholar]
- 6.Siddiqui MM, Rais-Bahrami S, Truong H, et al. Magnetic resonance imaging/ultrasound-fusion biopsy significantly upgrades prostate cancer versus systematic 12-core transrectal ultrasound biopsy. Eur Urol. 2013;64:713–9. doi: 10.1016/j.eururo.2013.05.059. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 7.Prendergast CJ, Ryder BA, Abodeely A, et al. Surgical performance with head-mounted displays in laparoscopic surgery. J Laparoendosc Adv Surg Tech A. 2009;19(Suppl 1):S237–40. doi: 10.1089/lap.2008.0142.supp. [DOI] [PubMed] [Google Scholar]
- 8.Kihara K, Fujii Y, Masuda H, et al. New three-dimensional head-mounted display system, TMDU-S-3D system, for minimally invasive surgery application: procedures for gasless single-port radical nephrectomy. Int J Urol. 2012;19:886–9. doi: 10.1111/j.1442-2042.2012.03044.x. [DOI] [PubMed] [Google Scholar]
- 9.Kihara K, Kawakami S, Fujii Y, et al. Gasless single-port access endoscopic surgery in urology: minimum incision endoscopic surgery, MIES. Int J Urol. 2009;16:791–800. doi: 10.1111/j.1442-2042.2009.02366.x. [DOI] [PubMed] [Google Scholar]
- 10.Kihara K, Kobayashi T, Kawakami S, et al. Minimum incision endoscopic surgery (MIES) in Japanese urology: results of adrenalectomy, radical nephrectomy and radical prostatectomy. Aktuelle Urol. 2010;41(Suppl 1):S15–9. doi: 10.1055/s-0029-1224662. [DOI] [PubMed] [Google Scholar]
- 11.Matsuoka Y, Numao N, Saito K, et al. Combination of diffusion-weighted magnetic resonance imaging and extended prostate biopsy predicts lobes without significant cancer: application in patient selection for hemiablative focal therapy. Eur Urol. 2014;65:186–92. doi: 10.1016/j.eururo.2012.10.010. [DOI] [PubMed] [Google Scholar]
- 12.Bhayani SB, Andriole GL. Three-dimensional (3D) vision: does it improve laparoscopic skills? An assessment of a 3D head-mounted visualization system. Rev Urol. 2005;7:211–4. [PMC free article] [PubMed] [Google Scholar]
- 13.Wang Q, Cheng D, Wang Y, et al. Design, tolerance, and fabrication of an optical see-through head-mounted display with free-form surface elements. Appl Opt. 2013;52:C88–99. doi: 10.1364/AO.52.000C88. [DOI] [PubMed] [Google Scholar]
- 14.Yoshida S, Kihara K, Takeshita H, et al. Head-mounted display for personal integrated-image monitoring system: ureteral stent placement. Urol Int. 2014 doi: 10.1159/000356987. in press. [DOI] [PubMed] [Google Scholar]
- 15.Yoshida S, Kihara K, Takeshita H, Fujii Y. A head-mounted display-based personal integrated-image monitoring system for transurethral resection of the prostate. Videosurgery Miniinv. 2014 doi: 10.5114/wiitm.2014.43040. in press. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Tadano K, Kawashima K. Development of a master-slave system with force-sensing abilities using pneumatic actuators for laparoscopic surgery. Advanced Robotics. 2010;24:1763–83. [Google Scholar]
- 17.Murphy DG, Bjartell A, Ficarra V, et al. Downsides of robot-assisted laparoscopic radical prostatectomy: limitations and complications. Eur Urol. 2010;57:735–46. doi: 10.1016/j.eururo.2009.12.021. [DOI] [PubMed] [Google Scholar]