Clinical applications of artificial intelligence in robotic urologic surgery

The integration of artificial intelligence (AI) into the realm of robotic urologic surgery represents a remarkable paradigm shift in the field of urology and surgical healthcare. AI, with its advanced data analysis and machine learning capabilities, has not only expedited the evolution of robotic surgical procedures but also significantly improved diagnostic accuracy and surgical outcomes.

In the synergy of AI and robotic urologic surgery, we witness a revolution that spans various facets of the field. Radiology, as a crucial component of preoperative planning and diagnosis, has benefited from AI-powered image analysis techniques, thereby enhancing the precision of surgical interventions. Furthermore, AI extends its influence directly into the operating room, where it plays an indispensable role in guiding surgeons during complex urologic procedures. This article delves into the clinical applications of AI in robotic urologic surgery, providing real-life examples of its utilization in preoperative decision making and intraoperative intelligent assistance.

The integration of AI has yielded substantial advancements in the early detection and characterization of urological diseases. A recent search of the literature looked at the focus of most AI research on urologic surgery and found that in kidney cancer, most of the research was on using advanced imaging analysis to predict tumor pathology preoperatively [1]. In prostate cancer, it focused more on predicting disease aggressivity and accordingly treatment counselling as well as functional and oncological outcome prediction algorithms [1]. In bladder cancer, the focus was more on attempting accurate preoperative staging to counsel on neoadjuvant chemotherapy [1]. It highlighted how machine learning was directly improving clinical decision making, without describing its integration in robotic surgery per se.

Accurate diagnosis of pathology preoperatively is a key component when allocating patients for robotic surgery which can be further enhanced using AI. When looking at different pathologies, specifically oncological cases, AI has improved imaging and risk stratifying techniques allowing better identification of surgery candidates. Kocak et al. [2] used machine learning methods and CT texture analysis to be able to differentiate between different subtypes of renal cell carcinoma (RCC). The algorithm was able to distinguish non-clear cell renal cell carcinoma (non-cc-RCC) from cc-RCC with external validation accuracy, sensitivity, and specificity of 84.6%, 69.2%, and 100.0%, respectively. In bladder cancer, multiple studies have investigated diagnosing cancerous lesions using artificial neural networks and convolutional neural networks (CNNs). Using cystoscopic images, Ikeda et al. [3] were able to build a CNN that was able to differentiate tumors from normal lesions with an area under the curve (AUC) of 0.98, sensitivity of 89.7%, and specificity of 94.0%. Multiparametric MRI of the prostate has become the preferred imaging method for detecting prostate cancer and radiomics, which extracts quantitative measures from qualitative features on imaging, has been widely implemented in this imaging modality. When comparing it to the Prostate Imaging Reporting and Data System classification, radiomics showed better performance in terms of sensitivity, specificity, and AUC (90% vs. 83%, 70% vs. 47%, and 0.85 vs. 0.73, respectively, all p<0.05) [4]. Khosravi et al. [5] looked at the MRI results of 400 patients retrospectively with biopsy-proven diagnosis of prostate cancer. They developed an algorithm using machine learning for the purpose of potentially reducing the number of unnecessary biopsies. The AI model achieved AUCs of 0.89 (95% confidence interval: 0.86–0.92) and 0.78 (95% confidence interval: 0.74–0.82) to classify cancer and benign lesions, high-risk and low-risk prostate cancer, respectively. The potential of such algorithms can be most appreciated for patients enrolled in an active surveillance protocol, allowing a noninvasive method of monitoring progression of disease.

Reading and diagnosing on pathology slides can sometimes be difficult and cumbersome. Other subjective factors related to specimen extraction can also create a suboptimal condition for diagnosis. AI has been shown to augment pathological diagnosis in a multitude of pathologies, including urological cancers. After analyzing 913 whole-slide images of bladder cancer patients, Zhang et al. [6] were able to develop a CNN algorithm that was capable of differentiating normal versus tumor tissue in patients with papillary urothelial carcinoma achieving true positive rate of 0.95 as well as obtaining similar results as the expert pathologists (AUC=0.97). Tabibu et al. [7] used a publicly available whole slide image library (The Cancer Genome Atlas) to train a CNN model to distinguish RCC from normal tissue. The trained model was able to distinguish clear cell and chromophobe RCCs from normal tissue with classification accuracy of 93.39% and 87.34%, respectively. The limiting factor in most of these acquisitions is the amount and variety of data available; as Tabibu et al. [7] mentioned, the model lost performance ability distinguishing rarer subtypes of renal cancer due to lack of data. In diagnosis of prostate cancer, there have been models that explored the ability to differentiate benign from malignant tissue based on cribriform pattern analysis of whole slide images, all with varying degrees of accuracy [8,9]. A unique distinction necessary in prostate cancer is also the ability to differentiate between high-grade and low-grade disease in order to determine which mode of management is the most suitable for the patient. Silva-Rodríguez et al. [10] used CNNs to train a model on over 6600 whole slide images of prostate biopsy cores, focusing on cribriform patterns. The model was able to identify patterns of Gleason score 4 in the specimen, pinpointing patients that are at risk for adverse features on final pathology and higher-risk disease.

Most robotic systems in clinical application now follow a master-slave method of operation. A scoping review of the literature conducted by Vasey et al. [11] identified the different applications of AI intraoperatively in robotic procedures. They concluded that most studies lacked outcomes of intraoperative use of AI in robotic surgery and were still in the initial phases of autonomy. This sheds light on the potential of enhancing robotic systems to incorporate AI as a form of increasing its level of autonomy, and becoming less of a tool and more of an assistant to the surgeon. Ma et al. [12] also found similar results in their review as well as the widespread use of AI in surgical simulation, training as well as automated segmentation and predictive analysis.

AI has been applied in surgical workflow recognition as well as augmented reality assistance during procedures. Nakawala et al. [13] used a combination of deep learning networks as well as knowledge representation and reasoning to aid in the identification of robot-assisted partial nephrectomy (RAPN) workflow. Using annotation and recognition of instruments and steps, the “Deep-Onto” network successfully identified 10 steps in RAPN with a prevalence-weighted macro-average (PWMA) recall of 0.83, PWMA precision of 0.74, PWMA F1 score of 0.76, and accuracy of 74.29% on 700 000 frames extracted from nine videos of RAPN. Aiding in surgical workflow would improve surgical planning and possible outcomes. Other investigations included predicting operative time, as well as intraoperative or postoperative events [14,15].

One of the more novel uses of AI is integrating these imaging techniques into the surgery itself. Reconstructing the images during robotic surgery provides immense support to the surgeon during surgery, especially in complex cases. Experiments have also been conducted using augmented reality to assist in visualization and surgical anatomy during robot-assisted surgery [16,17]. Canda et al. [18] reconstructed virtual reality (VR) models of prostates to aid in tumor navigation during robot-assisted radical prostatectomy (RARP) procedures. They utilized multiparametric MRI and ⁶⁸Ga-prostate-specific membrane antigen PET/CT to create three-dimensional reconstructions of the images, which were transferred to VR headsets and the da Vinci surgical robot via TilePro (Intuitive Surgical Inc., Sunnyvale, CA, USA).

Another study has shown potential for the augmented reality and AI use in various crucial steps of RARP, especially the neurovascular bundle dissection. Checcucci et al. [19] demonstrated similar feasibility when overlaying index lesion positive biopsy prostate cases onto the extirpative phase of RARP for maximal neurovascular bundle preservation. Despite only including 34 patients, they found it possible to detect 87% of the lesion locations in the neurovascular bundle of pT3 patients, and perform an additional resection after nerve preservation without significantly compromising the surgical margin rate. Despite technology still lacking today, the surgical community realized that the ideal AI system should be able to recognize a patient's individual anatomy while adapting continuously to the dynamic operative environment. It should draw upon all the aforementioned components of robotic surgery, including surgical steps, instrument recognition, and tissue differentiation, as well as utilizing the medical record in order for it to provide the optimal assistance to the surgeon [1].

A key clinical application of AI would be to make robotic surgery safer. One way to accomplish that is by ensuring that experienced surgeons are performing the surgery. Key performance metrics need to be established to objectively classify surgeons according to their experience and accordingly assess their surgical skills. Hung et al. [20] were able to collect automated performance metrics using the da Vinci recording device, analyze the gathered data using machine learning algorithms, and identify key performance metrics such as bimanual dexterity and frequent camera manipulation as indicators of experienced surgeons which in turn correlated with good outcomes. In a similar study, Ghodoussipour et al. [21] found that bimanual dexterity was also a key indicator of surgical expertise in RAPN and that its use during the tumor excision and renorrhaphy segments of the procedure was directly correlated to the complexity of the tumor. Another method to ensure the safety of the procedure would be to minimize adverse events. While attempting to gain expertise, robotic surgeons are more likely to experience difficult situations which could lead to unwanted surgical events. Using AI as a warning system intraoperatively, drawing attention to possible adverse events with unfavored movements of emergency halt systems could be of use. Checcucci et al. [22] were able to develop a model that could effectively predict intraoperative bleeding during RARP procedures with true positive rate of 98%. The artificial neural network built would scan the preceding 3 s recorded by the endoscope and predict a percentage value of likely bleeding in the next few seconds. However, the algorithm was tested on a limited number of procedures, which is the case for many studies that are assessing the efficacy of AI in improving patient safety in robotic surgery. Most studies also lack transparency in their datasets as well as more clinically useful results [23]. However, when the larger datasets become available, they do lay the groundwork for other more sophisticated measures to be built in the future.

However, a major obstacle in adopting AI into clinical practice, let alone high-risk environments like robotic urologic surgery, is the lack of compensability and trust between the entire medical community and the algorithms developed [24]. The primary reason for this situation stems from the limited transparency and human comprehension of these systems. Additionally, AI systems are susceptible to manipulation through human data input and malicious interactions. This underscores the critical importance of the concept of “Trustworthy AI” in fostering wholehearted human acceptance of AI. “Trustworthy AI” centers around two key elements: pinpointing factors that breed human distrust in AI systems and devising methods to enhance human trust [25]. Meanwhile, offering an augmented experience, as opposed to a complete human replacement, finds greater acceptance within the surgical community. The AI applications at the moment are predominantly predictive in nature, with generative AI gaining traction with much work being done on augmented reality and VR applications [16,17,19]. These modalities are what will allow precision medicine and personalized treatment plans to become more prevalent and enhance patient outcomes even further. Furthermore, with the introduction of telesurgery, we can see that AI is involved in the mitigation of data rates and latency to allow safe telecommunication between systems. It also paves the way for some automation in telesurgery. However, what is important to remember is that it should be integrated as part of a surgical ecosystem, rather than standalone systems, to allow for maximal efficiency in the surgical workflow.

In the realm of robotic urologic surgery, the integration of AI has ushered in a transformative era, enhancing the diagnostic accuracy, surgical outcomes, and overall patient care. From AI-driven advancements in radiology for precise preoperative planning to real-time guidance during intricate urologic procedures, the synergy of AI and robotics has revolutionized the field. The contribution of AI extends to pathology, offering improved diagnostics through advanced image analysis, particularly in kidney, prostate, and bladder cancers. Its roles in the perioperative setting, predicting surgical outcomes, and providing augmented reality assistance, underscore its versatility. The potential of AI in enhancing surgical autonomy, warning against adverse events, and establishing surgeon performance metrics is significant; however, acknowledging limitations, such as the current lack of trust and transparency in AI systems, remains crucial for their seamless integration into clinical practice in high-stakes environments like robotic urologic surgery.

Author contributions

Study concept and design: Shady Saikali, Runzhuo Ma, Vipul Patel, Andrew Hung.

Data acquisition: Shady Saikali, Runzhuo Ma.

Data analysis: Shady Saikali, Runzhuo Ma.

Drafting of manuscript: Shady Saikali, Runzhuo Ma.

Critical revision of the manuscript: Vipul Patel, Andrew Hung.

Conflicts of interest

The authors declare no conflict of interest.