The future of robotic surgery in urology: from augmented reality to the advent of metaverse

Dear Editor,

In the last years, following the concept of precision surgery¹ different new technologies have been introduced to tailor surgical interventions.

Among them, the application of 3D virtual models (3DVMs) is one of the most attractive² and, thanks to the integration of such models with robotic platforms, augmented reality (AR) procedures driven by the superimposition of the 3DVMs can be performed. The creation of the 3DVMs is the first crucial step for this kind of image-guided surgery and a rigorous pathway should be followed aimed to obtain high definition models that strictly reproduce the surgical anatomy and that can have a real benefit during the surgical procedures³ (Table 1).

Table 1.

Clinical potential application of 3DVMs for prostate and kidney cancer surgery.

3DVMs applications for prostate cancer	3DVMs applications for kidney cancer
Prostate landmarks identification	Kidney landmarks identification
Bladder neck identification	Tumor identification (especially when endophytic)
Tumor location during nerve-sparing phase	Renal pedicle management
Selective biopsy guidance at the level of preserved neurovascular bundles at the end of extirpative phase	Management of intraparenchymal structures during demolitive and reconstructive phase

3DVMs, 3D virtual models.

Several experiences have already demonstrated the potentialities of 3DVMs guidance in uro-oncologic surgery.⁴ For prostate cancer, a selective modulation of the nerve-sparing approach can be carried out, reducing the positive surgical margins rate;⁵ likewise, for kidney cancer, the AR images can help during both the resection and suturing phases⁶ with promising, although under scrutiny, benefits on postoperative renal function.⁷ More recently, this technology was also applied during robotic kidney transplantation, solving the open issue of atheromatic plaques identification, expanding the indication of robotic approach in this field.⁸

Furthermore, telementoring and telesurgery could be possible by using 3DVMs assistance, thanks to the potentialities of the 5G connections. The first remote 3DVMs cognitive telementoring during partial nephrectomy was preliminarily shown during the 9th edition of Techno-Urology-Meeting (http://www.technourologymeeting.com)  from Italy to Spain and Belgium: a 3DVM was shown inside the robotic console with the Tile-Pro and was maneuvered in real time remotely by an expert following the different phases of the intervention.

In fact, the 3DVMs can be exploited in many different settings such as patients counseling, presurgical planning, and surgical training, displaying images on a 2-dimensional (2D) flat screen or in a 3-dimensional (3D) virtual environment by using head-mounted display [mixed reality (MR) or virtual reality (VR) setting].⁹ The latter, which allows to live the 3D experience integrated with reality (MR) or in a totally virtual environment (VR), represents the origin of the concept of metaverse, for which a clear definition is still lacking. Generally, it usually refers to a totally alternative virtual world or a union between a fictional virtual world and the real one. So, it is correct to say that the metaverse can either represent an ‘enhanced’ world (i.e. MR) or, conversely, a totally different virtual world.¹⁰ Hence, the metaverse allows an opportunity to build infrastructure, through which other platforms like artificial intelligence (AI) and blockchain could improve different medical outcomes.

Both doctors and patients can benefit from its potentialities starting from population health management (i.e. cancer screening); however, herein we try to focus on robotic surgery only and briefly give a snapshot of the two faces of the coin. Looking at the patients’ perspective, VR in the metaverse may allow to create avatars to interact with physicians, allowing consultations and personalized approaches, creating a new model of patient care. The patient’s avatar is called ‘digital twin’ and it represents the way by which the patient can ‘live’ in the metaverse setting. Preoperative counseling can also be carried out in this virtual environment, where 3DVMs of the affected organ can be displayed and the surgical intervention can be discussed and explained. Furthermore, also the preoperative anesthesiologic assessment of pulmonary or cardiologic abnormalities [i.e. obstructive sleep apnea (OSA) or other sleeping/breathing to detect lung capacity with spirometry] can benefit from the introduction of such technologies.

Moreover, for postoperative care, surgeons may visit patients in a virtual metaverse-clinic, collecting data using dedicated sensors and interacting using telemedicine devices and softwares developed for patients’ monitoring,¹¹ performing a fully virtual physical examination, including observation, auscultation, and vital sign collection. Finally, the creation of social platforms in the metaverse could be used to improve social advocacy for different events such as prostate-specific antigen (PSA) prostate cancer screening or to improve patients’ recruitment for clinical trials.

From a surgical point of view, the first application that we can emphasize is training, which will be carried out in a totally virtual and immersive environment. Students will be able to train intensively in a simulated setting, overcoming the barriers of elevated costs associated with training (e.g. cadaver labs). To do so, surgical simulation requires the employment of precise movement-tracking devices and software to reproduce both the human anatomy and intraoperative movements of the surgeon. Furthermore, the integration of AI in this environment can be used to help with movement prediction for training/intraoperative surgical steps.

Another interesting field of application is the preoperative case discussion through multidisciplinary team discussion (MDT) which can take place in a virtual room using digital avatars, where the required specialists (e.g. oncologist, radiologist, pathologist, expert surgeon) have the chance to share patient’s information (e.g. clinical data on cloud platforms, 3D models) overcoming physical distances, building, when it is necessary, a ‘super-expert’ MDT maximizing the patient’s care.¹² Finally, switching to an intraoperative setting, as reported above, 3DVMs can allow to perform telementoring and telesurgery with 3D Cognitive or AR guidance.

However, not all that glitters is gold (Table 2).¹³ The metaverse still needs to face some major issues such as data management, privacy, or cybersecurity, and these firm security parameters should be established first to the introduction of metaverse in medicine. We must acknowledge that this new setting may open the doors to new types of risks, mainly related to personal data stealing.

Table 2.

Advantages and disadvantages of metaverse application in robotic urology (adapted from Bhugaonkar et al.).¹¹

Advantages	Disadvantages
Customization and modularity	Expensive
Integration with different non-medical professionalities	Risk of dependency
Three-dimensional environment	Legal and safety issues
Controlled and safe environment	Loss of human empathy
Multidisciplinary interaction	Clinical and surgical benefit under scrutiny
Surgical training, planning or navigation with increasing safety of the patients	Still in an experimental stage

Surely, the lack of accessibility (i.e. lack of Internet connection) and disparities in care, especially if patients in certain countries cannot afford these newer technologies or do not have the bandwidth/data storage to participate, can potentially represent an obstacle in the diffusion of such technology and to further widen the gap between the different health care systems in wealthy and poor countries. Furthermore, another aspect that should be considered is the absence of physical relationships, which may potentially hinder the patient–physician communication and the transmission of empathy of the direct human contact.

In conclusion, even if the metaverse is still in an experimental stage, it may reserve great potentialities for both patients and robotic surgeons, improving the quality of the health care system in terms of intervention and treatment, guaranteeing standardized surgical training, and helping the surgeons to maximize the quality of their surgery.