Validation of user-friendly models predicting extracapsular extension in prostate cancer patients

Abstract

Objective

There are many models to predict extracapsular extension (ECE) in patients with prostate cancer. We aimed to externally validate several models in a Japanese cohort.

Methods

We included patients treated with robotic-assisted radical prostatectomy for prostate cancer. The risk of ECE was calculated for each patient in several models (prostate side-specific and non-side-specific). Model performance was assessed by calculating the receiver operating curve and the area under the curve (AUC), calibration plots, and decision curve analyses.

Results

We identified ECE in 117 (32.9%) of the 356 prostate lobes included. Patients with ECE had a statistically significant higher prostate-specific antigen level, percentage of positive digital rectal examination, percentage of hypoechoic nodes, percentage of magnetic resonance imaging nodes or ECE suggestion, percentage of biopsy positive cores, International Society of Urological Pathology grade group, and percentage of core involvement. Among the side-specific models, the Soeterik, Patel, Sayyid, Martini, and Steuber models presented AUC of 0.81, 0.78, 0.77, 0.75, and 0.73, respectively. Among the non-side-specific models, the memorial Sloan Kettering Cancer Center web calculator, the Roach formula, the Partin tables of 2016, 2013, and 2007 presented AUC of 0.74, 0.72, 0.64, 0.61, and 0.60, respectively. However, the 95% confidence interval for most of these models overlapped. The side-specific models presented adequate calibration. In the decision curve analyses, most models showed net benefit, but it overlapped among them.

Conclusion

Models predicting ECE were externally validated in Japanese men. The side-specific models predicted better than the non-side-specific models. The Soeterik and Patel models were the most accurate performing models.

1. Introduction

Surgical treatment for prostate cancer aims to achieve the “pentafecta” that includes preservation of potency and continence, negative surgical margins, no surgical complications, and freedom from biochemical recurrence [1]. Performing a nerve-sparing radical prostatectomy (NSRP) is fundamental to conserve potency and continence [2]. However, NSRP increases the risk of a positive surgical margin which has prognosis implications and increases the associated cost [3,4]. Thus, there is a delicate balance between the amount of tissue resected and the risk of a positive surgical margin.

Extracapsular extension (ECE) is defined as the continuation of the tumor beyond the confines of the prostate, independently from seminal vesicle involvement [5]. ECE was found in about 20% of specimens in men with clinically localized prostate cancer [6]. It is an independent adverse prognostic factor and is associated with a worse prognosis [6,7]. Imaging technologies such as computed tomography scan and magnetic resonance imaging (MRI) may help in ECE detection in low-risk prostate cancer patients and may aid in decision-making in NSRP, but present a limited performance due to low sensitivity for detecting T3a stage [[8], [9], [10]]. Therefore, a correct prediction of whether there is ECE or not, is a cornerstone to proper management of the disease, determining patients’ eligibility and limiting the resection of tissue during NSRP [11,12].

However, the adequate way to choose candidates who benefit the most from NSRP has not been well elucidated. For this purpose, several tools (nomograms, linear simple formulas, and web calculators) that predict the probability of ECE have been developed and validated [[13], [14], [15], [16], [17], [18], [19], [20], [21]]. Some of these tools have been periodically updated incorporating new clinical data and insights regarding the procedure. Some of these models are available on a website and friendly to users, and patients can easily consult them.

Current American Urological Association guidelines recommend performing NSRP in patients with localized disease [22]. The European Association of Urology-European Society for Radiotherapy & Oncology-International Society of Geriatric Oncology guidelines for prostate cancer recommend not to preserve the neuro vascular bundle in patients with clinical stage higher than T2c, and with any biopsy International Society of Urological Pathology (ISUP) grade group higher than 3 and recommend to offer NSRP to patients with a low-risk extracapsular disease [7]. The National Comprehensive Cancer Network guidelines and the Japanese Urological Association guidelines do not stand a recommendation on the use of models to estimate the risk of ECE and select patients considered for NSRP [7,9,23].

However, most models for predicting ECE have not been externally validated at all or in Japanese. Given these facts, we aimed to externally validate models for predicting ECE in a Japanese cohort.

2. Patients and method

2.1. Patients and operation

The study has been approved by the institutional research ethics committee (approval number 2021–123, date 12/06/2021) and has been conducted in accordance with principles of the Helsinki Declaration. All patients gave their informed consents for participation in this research study.

We included a cohort of Japanese patients who underwent robot-assisted radical prostatectomy (RARP) between June 2009 and March 2021 at Kyushu University Hospital (Fukuoka, Japan) [24]. We included patients with diagnosis of carcinoma of the prostate that had complete clinical and histopathological results available and had a preoperative prostate MRI. We excluded patients with neoadjuvant treatment. Clinical and pathological data were collected prospectively in an electronic database. Individual patient data were used to calculate the risk of ECE in each nomogram. Tumor-node-metastases classification was used to assess clinical T-stage. ECE was defined as the continuation of the tumor beyond the confines of the prostate into the periprostatic tissue, independently from seminal vesicle involvement or bladder neck involvement [5]. RARP was performed by seven surgeons using the daVinci Surgical System (S, Si, and Xi, Intuitive Surgical, Sunnyvale, CA, USA). Neurovascular bundle preservation was performed combining anterograde and retrograde approaches and using a thermal technique. Patients were selected according to the operation date and the risk of ECE was determined by preoperative cancer risk, and preoperative patients’ potency and preference [25,26].

2.2. Models predicting ECE

In all, 10 models and the Roach formula were validated [27]. All of them were available online and friendly to users. The Patel model (available in https://prece.it/) and the Memorial Sloan Kettering Cancer Center (MSKCC) web calculator (https://www.mskcc.org/nomograms/prostate/pre_op) were calculated using the website calculator. We also included the logistic regression models that are available on https://www.evidencio.com/home. This online platform allows researchers to translate prediction models into online calculators for application and external validation. The estimated prognostic of ECE was calculated for each patient. The variables included were preoperative prostate-specific antigen (PSA) level, PSA density, clinical T-stage obtained by digital rectal examination, total Gleason score, the total number of cores, the number of positive and negative cores, and MRI or transrectal ultrasound findings, depending on each model. Side-specific models included the two prostate lobes, and the non-side-specific models included an overall calculation for each prostate gland.

2.3. Statistical analysis

Baseline characteristics were expressed as the median and interquartile range (IQR), or number and percentage. Comparisons between groups were performed using the Mann-Whitney, and Chi-square tests. Model performance was assessed by the area under the curve (AUC), calibration plots, and decision curve analyses. A logistic regression model was performed for each tool and the AUC was calculated to quantify model accuracy. AUC ranged between 0.5 and 1; AUC of 0.5 indicated no discrimination; 0.5<AUC<0.7 indicated poor discrimination; 0.7≤AUC<0.8 indicated acceptable discrimination; 0.8≤AUC<0.9 indicated excellent discrimination; 0.9≤AUC<1 indicated outstanding discrimination; and AUC of 1 indicated perfect discrimination [28]. Logistic regression statistics were used to evaluate model estimation by performing calibration plots that showed the agreement between predicted and observed ECE. Calibration was assessed by calculating the calibration slope and intercept. When the former was close to 1 and the latter was close to 0, it indicated a good calibration among the range of individuals. The predicted risks were divided into deciles and a line obtained by locally weighted scatterplot smoothing was superimposed. A well-calibrated model showed predictions that lay at or around the reference line in the calibration plot. Model performance was assessed by AUCs, calibration plots, and decision curve analyses. Decision curve analyses were performed to show the net benefit of the models as a function of the threshold probability. The net benefit was to correctly identify which patients had ECE or not. The unit of net benefit was the ratio of true positives to the target population [29]. All statistical analyses were performed using Stata v.14 (College Station, TX, USA). The p<0.05 was considered statistically significant.

3. Results

Overall, after excluding by criteria 178 patients were included in this analysis. The median age was 66 (IQR 62–69) years and the median PSA was 8.2 (IQR 5.9–12.9) ng/mL. Only four (2.2%) patients presented bilateral ECE. The baseline characteristics of the cohort are shown in Table 1. Of the 356 prostate lobes included, 117 (32.9%) lobes had ECE in the histopathological analysis. The baseline characteristics of the cohort by prostate lobes are shown in Table 2. The median total biopsy cores by prostate lobe were 6 (IQR 5–6), and the median positive cores by prostate lobes were 2 (IQR 1–3). Most patients with ECE presented ISUP grade groups 3, 4, or 5 in the prostate biopsy, and only 10% presented ISUP grade group 1. On the other hand, a quarter of patients without ECE presented ISUP grade groups 3, 4, or 5. Patients with ECE had a statistically significant higher PSA level, percentage of positive digital rectal examination, percentage of hypoechoic nodes, percentage of MRI nodes or ECE suggestion, percentage of biopsy positive cores, ISUP grade group, and percentage of core involvement.

Table 1.

Baseline patients’ characteristics.

Characteristic	Value (n=178)
Age at surgery, median (IQR), year	66 (62–69)
PSA, median (IQR), ng/mL	8.2 (5.9–12.9)
PSA density, median (IQR), (ng/mL)/mL	0.29 (0.20–0.44)
Clinical stage, n (%)
T1c	106 (59.6)
T2a	34 (19.1)
T2b	22 (12.4)
T2c	11 (6.2)
T3a	4 (2.2)
T3b	1 (0.6)
Pathological stage, n (%)
T2	79 (44.4)
T3a	72 (40.4)
T3b	26 (14.6)
T4	1 (0.6)
Pathological Gleason grade group, n (%)
ISUP 1	4 (2.2)
ISUP 2	63 (35.4)
ISUP 3	57 (32.0)
ISUP 4	3 (1.7)
ISUP 5	51 (28.7)
Surgical margin, n (%)
Positive	56 (31.5)
Negative	122 (68.5)

IQR, interquartile range; PSA, prostate-specific antigen; ISUP, International Society of Urological Pathology.

Table 2.

Baseline characteristics with comparison between groups with positive and negative extracapsular extension.

Variable	All lobes (n=356)	ECE positive (n=117)	ECE negative (n=239)	p-Value
Age, n (%)				0.051
<50 years	12 (3.4)	3 (2.6)	9 (3.8)
50–59 years	60 (16.9)	16 (13.7)	44 (18.4)
60–69 years	218 (61.2)	67 (57.3)	151 (63.2)
≥70 years	66 (18.5)	31 (26.5)	35 (14.6)
Median PSA, n (%)				<0.001
<10 ng/mL	230 (64.6)	56 (47.9)	174 (72.8)
10–20 ng/mL	88 (24.7)	36 (30.8)	52 (21.8)
>20 ng/mL	38 (10.7)	25 (21.4)	13 (5.4)
Digital rectal examination, n (%)				<0.001
Positive	107 (30.1)	54 (46.2)	53 (22.2)
Negative	249 (69.9)	63 (53.8)	186 (77.8)
Hypoechoic nodule in TRUS, n (%)				<0.001
Present	87 (24.4)	46 (39.3)	41 (17.2)
Absent	269 (75.6)	71 (60.7)	198 (82.8)
Magnetic resonance imaging nodes, n (%)				<0.001
ECE positive	29 (8.1)	23 (19.7)	6 (2.5)
ECE negative	100 (28.1)	55 (47.0)	45 (18.8)
No nodule	227 (63.8)	39 (33.3)	188 (78.7)
Ipsilateral Gleason grade group, n (%)				<0.001
No cancer	81 (22.8)	7 (6.0)	74 (31.0)
ISUP 1	77 (21.6)	12 (10.3)	65 (27.2)
ISUP 2	73 (20.5)	32 (27.4)	41 (17.2)
ISUP 3	67 (18.8)	31 (26.5)	36 (15.1)
ISUP 4	37 (10.4)	21 (17.9)	16 (6.7)
ISUP 5	21 (5.9)	14 (12.0)	7 (2.9)
Positive core ratio, n (%)				<0.001
0–25%	185 (52.0)	40 (34.2)	145 (60.7)
26%–50%	95 (26.7)	28 (23.9)	67 (28.0)
51%–75%	32 (9.0)	16 (13.7)	16 (6.7)
76%–100%	44 (12.4)	33 (28.2)	11 (4.6)
Maximum core involvement ratio, n (%)				<0.001
0–25%	205 (57.6)	40 (34.2)	165 (69.0)
26%–50%	71 (19.9)	30 (25.6)	41 (17.2)
51%–75%	22 (6.2)	8 (6.8)	14 (5.9)
76%–100%	58 (16.3)	39 (33.3)	19 (7.9)

ECE, extracapsular extension; TRUS, transrectal ultrasound; PSA, prostate-specific antigen; ISUP, International Society of Urological Pathology.

Note: total percentages may not be 100% due to rounding.

Overall, the side-specific models presented higher AUC than the non-side-specific models (Table 3). Among the formers, the Soeterik nomogram presented excellent discrimination with the higher AUC, followed by the Patel, Sayyid, Martini, and Steuber models. However, the 95% confidence interval for most of these models overlapped. Among the latter, the MSKCC web calculator with and without the total number of cores involvement presented the higher AUC, followed by the Roach formula, and Partin tables. Receiver operating characteristic plots showing the AUC are shown in Fig. 1. The new versions of Partin tables performed slightly better than the previous versions. The MSKCC web calculators with or without biopsy cores performed similarly. Calibration plots for side-specific models are shown in Fig. 2. The Soeterik model had the better calibration performance. However, this model overestimated the risk of ECE in low and high ranges (predicted risk of <10% and predicted risk >80%), and slightly underestimated the risk of ECE in media ranges. The Patel and Martini models calibrated similarly, and both underestimated in low- and high-risk ranges. The Sayyid and Steuber models underestimated in low ranges and overestimated in medium ranges. Regarding the non-side-specific models, all of them presented a poor calibration performance (Supplementary Fig. 1).

Table 3.

Results for the validated models and their updates.

Predictive model	Parameters utilized in model	AUC (95% CI)
Side-specific model
Soeterik	-PSA density, MRI clinical, total Gleason score	0.81 (0.76–0.86)
Steuber	-PSA, clinical T-stage, total Gleason score, percentage of positive BCs, percentage of cancer in biopsy specimen	0.73 (0.68–0.78)
Sayyid	-Age, PSA, clinical T-stage, total Gleason score, maximum percentage of core involvement, percentage of positive BCs, hypoechoic nodule	0.77 (0.71–0.82)
Martini	-PSA, total Gleason score, maximum percentage of core involvement, ECE in MRI	0.75 (0.70–0.81)
Patel	-Age, PSA, total Gleason score in positive core, clinical T-stage, percentage of in tumor positive	0.78 (0.73–0.83)
Non-side-specific model
Roach formula	-PSA, total Gleason score	0.72 (0.64–0.79)
Partin tables
Makarov	-PSA, clinical T-stage, total Gleason score	0.60 (0.51–0.68)
Eifel	-PSA, clinical T-stage, total Gleason score	0.61 (0.53–0.70)
Tosoian	-PSA, clinical T-stage, total Gleason score	0.64 (0.56–0.73)
MSKCC
Web calculator (incl. BCs)	-Age, PSA, clinical T-stage, primary and secondary Gleason, percentage of BCs	0.74 (0.67–0.81)
Web calculator (excl. BCs)	-Age, PSA, clinical T-stage, primary and secondary Gleason	0.74 (0.67–0.81)

AUC, area under the curve; BCs, biopsy cores; ECE, extracapsular extension; excl., excluding; incl., including; MRI, magnetic resonance imaging; CI, confidence interval; MSKCC, Memorial Sloan Kettering Cancer Center; PSA, prostate-specific antigen.

Figure 1.

Receiving operator characteristic curves. (A) The side-specific models; (B) The non-side-specific models. MSKCC, Memorial Sloan Kettering Cancer Center; BCs, biopsy cores.

Figure 2.

Calibration plots for the models. (A) The Soeterik model; (B) The Patel model; (C) The Sayyid model; (D) The Steuber model; (E) The Martini model. The expected risk is divided into ten equally sized groups. The green dots and spikes are media risks and 95% confidence intervals. The dotted line is the reference line of calibration. The blue line is obtained by locally weighted scatterplot smoothing. The red spike plot gives the distribution of extracapsular extension (0=no; 1=yes).

In the decision curve analyses, all side-specific models showed net benefit, but they overlapped among them (Fig. 3). The non-side-specific models presented net benefit, except for the Partin tables (Supplementary Fig. 2).

Figure 3.

Decision curve analyses of the Soeterik, the Patel, the Sayyid, the Steuber, and the Martini models.

4. Discussion

Among the side-specific models, the Soeterik model presented excellent discrimination with the higher AUC (0.81) followed by the Patel model AUC (0.78), the Sayyid nomogram AUC (0.77), the Martini nomogram AUC (0.75), and the Steuber model AUC (0.73). On the other hand, among the non-side-specific models, the MSKCC web calculator presented higher AUC (0.74), followed by Roach formula AUC (0.72), and the Partin tables of 2016 AUC (0.64), 2013 AUC (0.61), and 2007 AUC (0.60). Thus, the side-specific models presented higher AUCs than the non-side-specific ones. The 95% confidence interval for most of these models overlapped. However, the Soeterik model also presented the best calibration performance.

There is heterogeneity in the variables included in the side-specific models. All of them include PSA level, except the Soeterik model that includes PSA density. Soeterik and Martini nomograms include MRI findings, and Sayyid nomogram includes transrectal ultrasound findings. The Patel model includes the biopsy core involvement and the ISUP grade group for each core, while the others include the greater percentage of core involvement. Another difference is that the Sayyid and Martini nomograms do not predict a probability of ECE in a lobe without tumor, contrary to Soeterik, Steuber, and Patel nomograms. These models consider that, despite not having carcinoma in biopsy cores, the probability to have ECE is higher than zero in lobes with contralateral higher ISUP grade group, percentage of cancer involvement, or PSA. In fact, in the present study, ECE was positive in 8.8% (7/80) of prostate lobes without tumor in the biopsy. The Patel model predicts the overall risk of ECE, and at 1, 2, 3, and 4 mm. However, we only validated the presence or not of ECE to compare with the other models included. On the other hand, the non-side-specific models are more homogeneous in the variables included and most of them included PSA, clinical T-stage, and ISUP grade group.

Rocco et al. [30] performed an external validation of several models predicting ECE using the data of 6360 prostate cancer patients undergoing radical prostatectomy. They found that most of the Partin tables were not reliable and showed poor discrimination and unsatisfactory calibration. Additionally, the Steuber nomogram presented acceptable discrimination, but lower than that in the development population, and unsatisfactory calibration. However, most of these patients were from the USA, and the models included did not have MRI among variables. Another study suggested that the validated models are not reliable in the context of high-risk prostate cancer [31].

Supplementary Table 1 shows an overview of previous validation studies. The Partin tables published in 2001 were validated in a Japanese cohort in 2008 [25]. However, several other updates to these tables were subsequently performed [[19], [20], [21]]. To the best of our knowledge, this is the first external validation of the MSKCC web calculator with or without the total number of cores involvement, the Patel model, Sayyid model, and Soeterik model. Moreover, this study is also the first validation of new versions of the Partin tables and the Steuber models in Japanese.

Meanwhile, we found that AUCs in this study ranged among AUCs found in previous validations conducted mainly in European and North American countries (Supplementary Table 1). When comparing to other validation studies, our AUCs differed because of several factors: different biopsy approaches (transrectal or transperitoneal), biopsy techniques with the inclusion or exclusion of MRI, and the number of cores taken in each case; the decision to perform NSRP; and the characteristics of the population included. The proportion of prostate lobes with ECE included in the models ranged between 7% and 32% [15,32]. Moreover, there were differences in the clinicopathological characteristics among included patients. In addition, some studies included a larger number of databases, surgeons, or pathologists. Moreover, some studies mixed different surgical approaches (open, laparoscopic, or RARP). This is a single-center study, and RARP was performed in all patients and included a few surgeons and pathologists. Finally, ethnic differences could explain differences in prostate cancer behavior [33,34].

In the calibration plots, the Soeterik, Partin, Martini, and Sayyid models presented adequate calibration and presented the lowest prediction variability. However, the four of them overestimated the risk of ECE in patients with predicted ECE lower than 10%. Thus, in these patients, the observed risk of ECE may be 10%–20% lower. Current urology guidelines do not set a limit to recommend against performing NSRP. Setting a limit in 20% and 25%, using the Soeterik model in this cohort, NSRP would have been incorrectly omitted in 7.5% and 9.8% of patients, respectively. Using the same cut-off values using the Patel model resulted in 14.3% and 10.7% of false negatives.

Several papers have shown that the extent of ECE is associated with a poor prognosis [35,36]. However, there is no consensus on how to evaluate the extent of ECE. The Epstein criteria define focal extension as a few neoplastic glands beyond the limits of the prostate on no more than two separate sections, and non-focal extension as more than focal [35]. Meanwhile, other authors used the radial distance to measure the extent of ECE [36]. Recently, two studies with a large population using the Epstein criteria showed that both non-focal ECE and focal ECE were independently associated with a worse prognosis than patients without ECE. Non-focal and focal ECE were independent risk factors for biochemical recurrence [37,38]. Ten-year cancer-specific survival and overall survival in patients with isolated ECE were not significantly different between patients with focal or non-focal ECE [38].

Recently, the inclusion of artificial intelligence in MRI models compared to experts presented promising results[39]. Moreover, there is still an unknown role for genetic in this field such as PTEN and endothelin-1. The development of nomograms that include artificial intelligence and genetic could be a future perspective to predict ECE and improve surgical management.

5. Conclusion

Among all evaluated models, the side-specific models showed acceptable to excellent performance with an AUC of 0.73–0.81 and predicted better than the non-side-specific models. The side-specific models presented adequate calibration, net benefit, and had the lowest prediction variability. The best-performing nomograms were the Soeterik and the Patel models. Then, these models can be utilized for the prediction of ECE in Japanese who undergo NSRP.

Author contributions

Study concept and design: Leandro Blas, Masaki Shiota.

Data acquisition: Leandro Blas, Masaki Shiota, Shohei Nagakawa, Shigehiro Tsukahara, Takashi Matsumoto, Ken Lee, Keisuke Monji, Eiji Kashiwagi, Junichi Inokuchi, Masatoshi Eto.

Data analysis: Leandro Blas, Masaki Shiota.

Drafting of manuscript: Leandro Blas, Masaki Shiota.

Critical revision of the manuscript: Leandro Blas, Masaki Shiota.

Conflicts of interest

The authors declare no conflict of interest.

Appendix A. Supplementary data

The following are the Supplementary data to this article:

Supplementary Fig. 1.

Supplementary Fig. 1. Calibration plots for the models. (A) Partin table of 2007; (B) Partin table of 2013; (C) Partin table of 2016; (D) The Roach formula; (E) The MSKCC web calculator without biopsy cores; (F) The MSKCC web calculator with biopsy cores. The expected risk is divided into ten equally sized groups. The green dots and spikes are media risks and 95% confidence intervals. The dotted line is the reference line of calibration. The blue line is obtained by locally weighted scatterplot smoothing. The red spike plot gives the distribution of extracapsular extension (0=no; 1=yes).

Supplementary Fig. 2.

Decision curve analyses of the Partin tables of 2007, 2013, 2016, the Roach formula, and the MSKCC web calculator without and with biopsy cores. MSKCC, Memorial Sloan Kettering Cancer Center; BCs, biopsy cores.