Three-dimensional automatic artificial intelligence driven augmented-reality selective biopsy during nerve-sparing robot-assisted radical prostatectomy: A feasibility and accuracy study

Abstract

Objective

To evaluate the accuracy of our new three-dimensional (3D) automatic augmented reality (AAR) system guided by artificial intelligence in the identification of tumour's location at the level of the preserved neurovascular bundle (NVB) at the end of the extirpative phase of nerve-sparing robot-assisted radical prostatectomy.

Methods

In this prospective study, we enrolled patients with prostate cancer (clinical stages cT1c–3, cN0, and cM0) with a positive index lesion at target biopsy, suspicious for capsular contact or extracapsular extension at preoperative multiparametric magnetic resonance imaging. Patients underwent robot-assisted radical prostatectomy at San Luigi Gonzaga Hospital (Orbassano, Turin, Italy), from December 2020 to December 2021. At the end of extirpative phase, thanks to our new AAR artificial intelligence driven system, the virtual prostate 3D model allowed to identify the tumour's location at the level of the preserved NVB and to perform a selective excisional biopsy, sparing the remaining portion of the bundle. Perioperative and postoperative data were evaluated, especially focusing on the positive surgical margin (PSM) rates, potency, continence recovery, and biochemical recurrence.

Results

Thirty-four patients were enrolled. In 15 (44.1%) cases, the target lesion was in contact with the prostatic capsule at multiparametric magnetic resonance imaging (Wheeler grade L2) while in 19 (55.9%) cases extracapsular extension was detected (Wheeler grade L3). 3D AAR guided biopsies were negative in all pathological tumour stage 2 (pT2) patients while they revealed the presence of cancer in 14 cases in the pT3 cohort (14/16; 87.5%). PSM rates were 0% and 7.1% in the pathological stages pT2 and pT3 (<3 mm, Gleason score 3), respectively.

Conclusion

With the proposed 3D AAR system, it is possible to correctly identify the lesion's location on the NVB in 87.5% of pT3 patients and perform a 3D-guided tailored nerve-sparing even in locally advanced diseases, without compromising the oncological safety in terms of PSM rates.

1. Introduction

Nowadays, in uro-oncologic surgery we have entered in a technology driven era, in which the latest innovations are adopted aiming to personalize the surgical treatment for every single patient [1]. In order to minimize the impact on postoperative patients’ quality of life, functional and oncological outcomes are considered to be equally important.

Focusing on prostate cancer robotic surgery, the preservation of neurovascular bundles (NVBs) is associated to a higher postoperative potency recovery rate; however, during the procedure, delineating tumour's margins is not always simple and, in particular during the nerve-sparing (NS) phase of the intervention, this could lead to the possible occurrence of positive surgical margins (PSMs), which are often related to disease persistence or recurrence [2].

Therefore, the possibility to label and precisely detect tumour's location during surgery could give great benefits to patients, preventing the unnecessary removal of healthy tissue related to potency and continence preservation.

During the last decade, multiple innovative technologies and techniques such as near-infrared fluorescence, optical coherence tomography, and confocal laser endomicroscopy, have emerged, seeking to offer a visual guide to surgeons in the operation theatre [3].

Trying to give a contribution in this field, our group already published the promising results of an innovative augmented reality (AR) platform that allows to overlap intraoperatively in real-time hyper accuracy three-dimensional (HA3D™) models (Medics srl, Turin, Italy) specifically created for every single patient, starting from their preoperative multiparametric magnetic resonance imaging (mpMRI) images, over the real anatomy [4]. This system has proven to be accurate in the identification of tumour's extracapsular extension (ECE) and location, both in a static and a dynamic phase of the intervention.

Notwithstanding these impressive results, the main limit of this technology is represented by the fact that the entire overlap process is manually driven by an assistant placed near the surgical console.

In order to overcome this limitation, we developed an automatic augmented reality (AAR) system, guided by artificial intelligence (AI), that allows to project prostate's and tumour's virtual images at the level of the prostatic lodge and NVB preserved at the end of extirpative phase of the intervention, allowing to perform a 3D AAR guided biopsy, without the need of an assistant for the overlapping process.

2. Patients and methods

2.1. Study aim

The aim of this study was to evaluate the accuracy of our new 3D AAR system guided by AI in the identification of tumour's location at the level of the preserved NVB at the end of the extirpative phase during the NS phase of robot-assisted radical prostatectomy (RARP).

2.2. Study population and HA3D™ models reconstruction

This was a prospective study, enrolling patients with prostate cancer (clinical stages cT1c–3, cN0, and cM0) whose diagnosis was based on a positive target biopsy at the index lesion [5]. All of them underwent RARP at San Luigi Gonzaga Hospital (Orbassano, Turin, Italy), from December 2020 to December 2021. The study was conducted in accordance with good clinical practice guidelines, and informed consents were obtained from the patients. According to Italian law (Agenzia Italiana del Farmaco Guidelines for Observational Studies, March 20, 2008), no formal institutional review board or ethics committee approval was needed. Preoperative assessment included high-resolution (at least 3 mm slices) mpMRI with a 1.5–3 T scanner, according to ESUR guidelines and Prostate Imaging-Reporting and Data System (PIRADS) v.2 recommendations [6,7].

Only patients with an index lesion suspicious for capsular contact or ECE at mpMRI were enrolled in this study. The images in DICOM format were processed by a dedicated software, authorised for medical use by Medics Srl (www.medics3d.com), in order to perform HA3D™ reconstruction as we already described [8]. The final output was in STL format. On the basis of the availability of this technology at our institution, the patients were scheduled for 3D AAR RARP.

2.3. 3D AAR system development

Our new AAR system guided by AI is based on a two-step automatic system able to overlap the HA3D™ prostate model to endoscopic bidimensional images, which relies for each step on a different convolutional neural network (CNN) (Fig. 1). For the first one, a CNN able to recognize and then localize the catheter, introduced in the prostatic lodge at the end of the extirpative phase of the procedure, was set. From this CNN's output, two main features were collected: the rotation angle for z-axis direction and the anchor point of the catheter, its entry point in the operative field. y-axis was not considered since uncommonly required during prostatic surgery.

Figure 1.

Augmented reality system development strategy. AR, augmented reality; 3D, three-dimensional; RARP, robot-assisted radical prostatectomy; CNN, convolutional neural network; NN, neuronal network.

Then, to train the second CNN, an artificial dataset was created, containing images with a real anatomical background taken from recorded videos of RARP and a rendered model of a catheter with different rotations. For this purpose, five prerecorded videos from different RARP surgeries showing the entrance of the catheter into the operative field were chosen. From all these videos, a set of frames was selected. At the end of this procedure, the total number of frames was approximately 15 570 images. Each frame was manually tagged by two senior urologists, identifying the position of the catheter.

The final dataset included frames coming from the five different videos (A–E): 325 frames come from A, 90 from B, C, and D each, and 100 from E. Different combinations of segmentation architecture and base networks were tested at first. Then, the second CNN was trained leveraging this artificial dataset, to predict the rotation along the x-axis and testing it on endoscopic images, reaching a high rate of correct estimations. Coupling the results of both networks, the automatic overlay in real time of the 3D-virtual model over the real endoscopic view was obtained.

Briefly, the first CNN works to output catheter position and z-axis rotation by identifying the anchor point. The second CNN returns antero-posterior rotation on the x-axis. Their combined results allow the proper overlay of the model over the real anatomy in real time.

The transparency of the different parts of the model can be modulated, leaving projected on the in-vivo anatomy only the catheter and the tumour (Fig. 2).

Figure 2.

Three-dimensional automatic augmented reality processing. According to the convolutional neuronal network method pipeline, the image (A) was segmented (B) and after the shape detection (C), the 3D model was projected over the two-dimensional image (D). 3D, three-dimensional.

2.4. Surgical technique and 3D AAR selective biopsy

All the procedures were performed according to our total anatomical reconstruction technique [9,10]. For this specific study, an intrafascial NS procedure was performed at the side of index lesion, whilst contralaterally intra- or inter-fascial NS procedure were carried out according to clinical indication.

At the end of extirpative phase, after prostate removal from the surgical field, the virtual images of the prostate were projected automatically in the prostate bed as 3D AR images by using our AI software, and were visualised by the surgeon thanks to the Tile-Pro (Intuitive, Sunnyvale, CA, USA) inside the robotic console monitor. The virtual prostate 3D model allowed to identify the tumour’s location projected at the level of the NVB preserved. Therefore, a selective excisional biopsy was taken on the NVB as indicated by the 3D AAR images. With this manoeuvre, at the side of index lesion, the final result was a 3D-imaging tailored NS approach with a selective extrafascial section performed at the level of the lesion indicated by 3D models, whilst the remaining portion of NVB was preserved with an intrafascial approach (Fig. 3).

Figure 3.

Three-dimensional automatic AR biopsy workflow. (A) Operative field after bilateral full nerve-sparing procedure; (B and C) The catheter was inserted into the prostatic lodge and the 3D model was automatically overlapped; (D) The lesion projection was left into the Tile-Pro screen; (E) The biopsy was performed; (F) The final result was a 3D tailored nerve-sparing procedure. AI, artificial intelligence; AR, augmented reality; 3D, three-dimensional; NVB, neurovascular bundle.

2.5. Histopathological evaluation

Whole-mount histological sections resected from the radical prostatectomy specimens of selected patients were used as the reference standard. More specifically, prostate surface was marked with black ink and, subsequently, the gland was cut following a previously reported method [11,12] modified to get 3-mm thick sections; slices were obtained perpendicularly to the rear gland surface, with the same inclination of the axial T2W images. Conversely, each base and apex were sectioned longitudinally and bundle biopsy was totally included.

The presence of tumour was evaluated during the reduction phase, on the prostate surface and at the level of NVB's excisional biopsy, sent separately.

Then, after routine histologic process, 5-μm sections were taken from each thick slice and stained with haematoxylin and eosin.

All samples were then assessed for cancer foci by an experienced uro-pathology team (F.M. and E.B). The tumour volume of each node was calculated as previously described [11]. The pathologist also assessed the pathological Gleason score [13] for each focus and specified the presence or absence of prostatic capsular invasion and the amplitude of the extraprostatic extension following previously reported methods [14] in particular under the ink.

2.6. Data collection

Preoperative variables such as demographic, radiological, and pathological features were collected. Intraoperative variables were also gathered, with particular attention to the rate of full and partial NS surgical interventions (according to Pasadena Consensus classification [15]). The grade of NS procedure was considered as follows: full NS procedure for bilateral intrafascial; partial NS procedure for interfascial/intrafascial or interfascial/interfascial.

Finally, postoperative variables including catheterization time, hospitalization time, and complication rates (according to the Clavien-Dindo classification [16]), were recorded.

Pathological variables were analysed as described in section 2.5.

PSMs were evaluated at the level of prostate surface for the lobe contralateral to the index lesion, whilst for the lobe with the index lesion the PSMs were evaluated at the level of external margins of the excisional biopsy.

Potency recovery was assessed using the International Index of Erectile Function-5 (IIEF-5) questionnaire (score of ≥17 means “valid erection”) [17]; urinary continence recovery was defined if patients did not use any pad or in case of one single safety pad use per day. Urinary continence was evaluated at catheter removal and at 1 month, 3 months, 6 months, and 12 months after RARP. Oncological outcomes were determined at a minimum of 12 months, assessing the biochemical recurrence (BCR). BCR was defined as (i) any postoperative cancer treatment, or (ii) prostate-specific antigen level of >0.2 ng/mL with a single repeated measurement for confirmation.

2.7. Statistical analysis

Based on our study aim (please see section 2.1), considering an expected success proportion of 70%, a null proportion of 40%, and a power of 90%, 20 patients were needed. Considering a 20% of dropout (due to an erroneous rT3 staging), 24 patients allowed a power of 90%. The calculation was performed with PROC SEQDESIGN by SAS® statistic software (SAS Intuitive, Cary, NC, USA).

A descriptive analysis of the subjects was performed in order to assess a general description of the sample. Medians and interquartile ranges (IQRs) were used to describe continuous variables. Categorical variables were summarised by frequency tables.

The concordance rate for mpMRI and pathological findings was expressed by Cohen's kappa coefficient. Data were analysed by StatSoft v.10 (Dell, Texas, USA).

3. Results

3.1. Demographics and mpMRI findings

A total of 34 patients were enrolled in this study with a median age of 62 (IQR 60–69) years and a median prostatic volume of 40 (IQR 30.3–44.2) mL. Demographic and mpMRI variables are reported in Table 1. Median preoperative IIEF-5 score was 21 (IQR 12–24). In 15 (44.1%) cases, according to prostatic mpMRI, the target lesion was in contact with the prostatic capsule (Wheeler grade L2) whilst in 19 (55.9%) cases ECE was detected (Wheeler grade L3).

Table 1.

Preoperative variables (n=34).

Patient characteristic	Value
Age, year	62 (60–69)
BMI, kg/m2	24.9 (23.6–27.1)
ASA score	2 (2–3)
PSA, ng/mL	5.6 (4.1–9.2)
Positive DRE	8 (23.5)
IIEF-5 score	21 (12–24)
Prostate volumea, mL	40 (30.3–44.2)
Lesion volume, mm	9 (6–12)
PIRADS score
3	2 (5.9)
4	21 (61.8)
5	11 (32.4)
Radiological Wheeler
L2	15 (44.1)
L3	19 (55.9)
Gleason score	7 (7–8)

BMI, body mass index; ASA, American Society of Anaesthesiologists; DRE, digital rectal examination; IIEF-5, International Index of Erectile Function-5; PIRADS, Prostate Imaging and Reporting and Data System; PSA, prostate-specific antigen.

Note: values are presented as median (interquartile range) or n (%).

^aProstate volume was measured on multiparametric magnetic resonance imaging.

3.2. Intraoperative and perioperative findings

Mean operative time was 123 (standard deviation 31) min. Twenty-four (70.6%) patients underwent partial NS (i.e., bilateral interfascial or unilateral intrafascial) procedures according to the clinical stage and mpMRI findings. In 10 (29.4%) patients, a full NS (i.e., bilateral intrafascial) procedure was performed. Intraoperative data are summarised in Table 2. No major intraoperative complications occurred. Median hospital stay and catheterization time were 6 (IQR 5–6) days and 5 (IQR 4–7) days, respectively. No major postoperative complications (i.e., Clavien-Dindo Grades III–V) were observed in the study cohort.

Table 2.

Intra- and peri-operative findings (n=34).

Intra- and peri-operative variable	Value
Operative time, min	123 ± 31
Blood loss, mL	235 ± 50
NS procedure
Full NS	10 (29.4)
Partial NS	24 (70.6)
Intraoperative complications	0 (0)
Postoperative Clavien-Dindo Grades III–V complication	0 (0)
Catheterization time, day	5 (4–7)
Hospital stay, day	6 (5–6)
Urinary continence
After catheter removal	22 (64.7)
1 week after catheter removal	25 (73.5)
1 month after surgery	28 (82.4)
3 months after surgery	29 (85.3)
6 months after surgery	30 (88.2)
12 months after surgery	32 (94.1)
Potency rate
1 month after surgery	13 (38.2)
3 months after surgery	20 (58.8)
6 months after surgery	24 (70.6)
12 months after surgery	24 (70.6)
Biochemical recurrence at 12th month	0 (0)

NS, nerve-sparing.

Note: values are presented as mean±standard deviation, n (%), or median (interquartile range).

3.3. Pathological and 3D AAR guided biopsy findings

At final pathology, 18 and 16 patients were pathological tumour stages pT2 and pT3, respectively (Table 3). A good concordance between mpMRI and pathological findings was recorded (kappa=0.79).

Table 3.

Pathological data (n=34).

Pathological data	Value
Prostate volume, mL	34.3 (27.2–38.7)
Tumour volume, mL	2.1 (1.5–2.8)
Pathological tumour stage
pT2	18 (52.9)
pT3	16 (47.1)
Gleason score
2–6	0 (0)
7	30 (88.2)
8–10	4 (11.8)

Note: values are presented as n (%) or median (interquartile range).

However, 21.4% (3/14) of the patients with suspicious capsular contact at mpMRI revealed to be pT3, while in case of suspicious ECE at mpMRI, pT2 disease was found in seven (35.0%) patients.

3D AAR guided biopsy at the level of suspicious lesion projection was negative in all pT2 patients while it revealed the presence of cancer in 14 (14/16; 87.5%) cases in pT3 cohort (Table 4).

Table 4.

Three-dimensional automatic augmented reality biopsy findings (n=34).

cT stage	pT stage	Patient	NVB Bx+	PSM
cT2 (n=14)	• pT2	11 (78.6)	0 (0)	0 (0)
	- Apical lesion	4 (28.6)	0 (0)	0 (0)
	- Equatorial lesion	3 (21.4)	0 (0)	0 (0)
	- Basal lesion	4 (28.6)	0 (0)	0 (0)
	• pT3	3 (21.4)	3 (21.4)	1 (7.1)
	- Apical lesion	1 (7.1)	1 (7.1)	1 (7.1)
	- Equatorial lesion	2 (14.3)	2 (14.3)	0 (0)
	- Basal lesion	0 (0)	0 (0)	0 (0)
cT3 (n=20)	• pT2	7 (35.0)	0 (0)	0 (0)
	- Apical lesion	1 (5.0)	0 (0)	0 (0)
	- Equatorial lesion	6 (30.0)	0 (0)	0 (0)
	- Basal lesion	0 (0)	0 (0)	0 (0)
	• pT3	13 (65.0)	11 (55.0)	0 (0)
	- Apical lesion	4 (20.0)	3 (15.0)	0 (0)
	- Equatorial lesion	3 (15.0)	3 (15.0)	0 (0)
	- Basal lesion	6 (30.0)	5 (25.0)	0 (0)

NVB Bx⁺, positive neurovascular bundle biopsy; PSM, positive surgical margin; cT, clinical tumour stage; pT, pathological tumour stage.

Note: values are presented as n (%).

PSM rate in pT2 patients was 0%, whilst considering the extension of the resection with the 3D guided excisional biopsy, the PSM rate was 7.1% (1/14 patients) in the pT3 group. Furthermore, in the only case of PSM, the margin was <3 mm and a Gleason score of 3 was observed.

3.4. Follow-up findings

The 30-day urinary continence and sexual potency rates were 82.4% (28 patients) and 38.2% (13 patients), respectively. At 12 months of follow-up, 32 (94.1%) patients were continent and the erectile function recovery rate was 70.6% (24 patients).

After a minimum of 12 months of follow-up, prostate-specific antigen was undetectable in all cases and no patient experienced BCR, including the single case with PSM in the pT3 group. No patient underwent further oncological treatments.

4. Discussion

Herein we present the results of our experience with the first application of AI guidance for 3D models guided surgery. After the feasibility description in an IDEAL 1 stage study [18], with this work we expand the application of such technology, describing the IDEAL 2a phase.

Our findings are particularly encouraging for three different reasons: (1) the 3D AAR guided biopsy was able to correctly identify cancer presence at the level of the NVB in 87.5% of pT3 patients; (2) the execution of an excisional biopsy at this level allowed to record only 7.1% of PSMs at the index lesion’ side, even in locally advanced disease; (3) this 3D-imaging tailored NS approach allowed to preserve a huge amount of neurovascular fibres, with a good rate of potency recovery (58.8% at 3 months of follow-up).

These findings should be analysed under two points of view: the clinical and the technological one.

Concerning the first, the importance of PSMs reduction for oncological reasons (i.e., lower risk of metastasis development or need for further therapies) is already well demonstrated [19]. Following this direction, different technology-guided techniques have been introduced, such as NeuroSAFE [20] and confocal laser endomicroscopy [21], aiming to reduce the occurrence of this unwanted event. Our experience perfectly fits in this scenario: thanks to the correct identification of suspicious ECE using the 3D images overlapped in an AR manner, it is possible to remove the amount of cancer tissue accidently left on the NVB during the NS phase of the procedure, with a consequent reduction of PSMs. In fact, in current literature, PSM rates in locally advanced disease range from 17% to 65% [22], whilst in our series we found an overall PSM rate of 2.9%, with only 7.1% in pT3 disease, defining the presence of PSMs as the presence of cancer cells on external margins of the excisional biopsy at the side of the index lesion.

These promising findings pave the way to a new concept of 3D guided modulated NS approach: the correct identification of lesion's location and its removal with the selective biopsy allowed to safely preserve the NVB next to the lesion, even homolaterally to the index lesion. These findings are in line with our previously published experience [23] in which the use of 3D models brought to a reduction of PSM rate (25.0% vs. 35.1%, p=0.01), especially using AR (p=0.004 at multivariable linear regression model). Furthermore, in this past experience on 160 patients undergoing 3D-guided RARP, we found a higher rate of full NS in the 3D group (p=0.02) without compromising the oncological safety in terms of PSM rate (p=0.87, p=0.21, and p=0.03 in full, intermediate, and standard NS, respectively, favouring the 3D group) or BCR (p=0.07 at Kaplan-Meier plots at 12 months of follow-up). The optimization of the NS phase of the intervention still remains an hot topic for radical prostatectomy, especially in the current precision surgery era [24,25], and our approach seems to be a promising one, allowing to safely preserve the fibres also in patients with a locally advanced disease, reaching a good balance between oncological safety (PSM rate of 7.1%) and functional recovery (potency rate at 3 month of 58.8%).

Moving to the technological aspects, HA3D™ models for prostate cancer have already showed their accuracy in prostatic anatomy definition and tumour identification [4,26], even in an AR setting [27,28]. Herein we moved forward, with the development of an AR platform and introducing for the first time the application of AI for the automatic overlapping of the 3D models over the real anatomy.

The integration of two different CNNs allows to correctly orient the 3D models in the space axes with a subsequent accurate overlay over the real anatomy in real time. This new tool allows to overcome one of the main limitations of the previous AR platforms, with a total independency from the surgeon's assistant who manually overlapped the images. In fact, this system is able to provide the automatic overlapping only with the click of a single button on the keyboard of a PC placed next to the robotic console.

In current literature, the application of AI to robotic surgery is in its preliminary phase, also in urology [[29], [30], [31], [32], [33]]. As recently reported by Moglia et al. [34] in their systematic review, only 12 studies are focused on AI in urologic robotic surgery; however, they are focused on the building of prediction models, evaluation of surgeons’ experiences, and recognition and classification of gestures during the execution of specific steps of the interventions. Considering the intraoperative AI assistance, a preliminary experience was carried out regarding an autonomous camera positioning [30]; on the contrary, the application of AI to more complex tasks (e.g., suturing, knot-tying, and tissue dissection) is more difficult to achieve and, to the best of our knowledge, no previous experience on image-guided surgery based on AI tools exists.

The current study is not devoid of limitations. First, the clinical results should be weighted according to the project development stage (IDEAL 2a), and further studies with a higher level of evidence should be designed. We indeed recently started a randomised clinical trial, approved by our local ethical committee (practice 371/2022) to evaluate the real clinical benefits of such approach. Second, the implementation of AI dataset, based on patients’ data, still needs to face some major issues like data management, privacy, cybersecurity, and the lack of punctual legislation, which represent technical obstacles that still need to be overcome.

Notwithstanding these limitations, our findings are promising in the current era of tailored surgery, in which oncological and functional outcomes are equally important. In this scenario, researchers should pave the way for new technology applications in surgery such as 3D AR and AI, aiming to offer patients an even higher level of surgical performance.

5. Conclusion

Our 3D AAR system allows to correctly identify the lesion location on NVB in 87.5% of the pT3 patients. This brings to the possibility to execute a 3D-guided tailored NS procedure even in locally advanced diseases, without compromising the oncological safety in terms of PSM rate. Further studies with a randomised control trial design are needed to corroborate these preliminary oncological and functional outcomes.

Author contributions

Study concept and design: Francesco Porpiglia, Enrico Checcucci.

Data acquisition: Sabrina De Cillis, Federico Piramide, Gabriele Volpi, Alberto Piana, Cecilia Gatti, Federica Massa.

Data analysis: Federico Piramide, Ilaria Stura.

Drafting of manuscript: Enrico Checcucci, Alberto Piana, Gabriele Volpi, Pietro Piazzolla.

Critical revision of the manuscript: Gabriele Volpi, Enrico Checcucci, Daniele Amparore, Cristian Fiori, Enrico Bollito, Michele Di Dio.

Conflicts of interest

The authors declare no conflict of interest.