External validation of Memorial Sloan Kettering Cancer Center nomogram and prediction of optimal candidate for lymph node dissection in clinically localized prostate cancer

Abstract

Introduction

The aim of our study was to evaluate the external validity of the online Memorial Sloan Kettering Cancer Center (MSKCC) nomogram as a predictor for pelvic lymph node invasion (LNI) in men who underwent radical prostatectomy (RP) with pelvic lymph node dissection (PLND).

Material and methods

The study cohort consisted of 679 men with clinically localized prostate cancer (PCa) who underwent RP with PLND between 2005 and 2017. The area under curve (AUC) of the receiver operator characteristic analysis was used to quantify the accuracy of MSKCC nomogram to predict LNI. The specificity, sensitivity and negative predictive value were calculated to assess LNI probability cut-off.

Results

A total of 81 of 679 patients had LNI (11.9%). The AUC of MSKCC nomogram was 79%. Using the cut-off value of 7% (sensitivity 88.9%, specificity 45.2% and NPV 96.8%) a PLND could be omitted in 41% (279/679) of men. However, 3.2% (9/279) of men with LNI would be missed. MSKCC nomogram showed good calibration characteristics and high net benefit at decision curve analysis.

Conclusions

MSKCC nomogram in patients with PCa undergoing PLND has 79% discriminated accuracy for prediction of LNI in our cohort. Using a 7% nomogram cut-off, roughly 40% of men would be spared PLND with minimal risk to miss LNI.

INTRODUCTION

According to global cancer statistics, prostate cancer (PCa) remains one of most often diagnosed cancers among men. Although the death rate is decreasing, it is the fifth leading cause of cancer mortality in men with more than 307,000 deaths in 2012 [1]. There are several PCa treatment options; however, radical prostatectomy (RP) remains one of most frequently used [2] methods of treatment. Pelvic lymph node dissection (PLND) during RP remains the most accurate staging procedure for the detection of lymph node invasion (LNI) in PCa [3, 4]. The detection of LNI at PLND ranges from 1.1% to 28% [2–13] and is directly associated with the extent of PLND and the aggressiveness of PCa [5–8]. There is no doubt that extended PLND (ePLND) provides more accurate staging in comparison with limited PLND (lPLND) and has been recently recommended as a standard procedure by most international urological guidelines [2, 4, 10]. However, ePLND is associated with longer duration of general anesthesia, longer operation time and more postoperative complications compared to lPLND [14]. The indications for PLND continue to be controversial [2, 3, 4, 9]. There is a general agreement to recommend PLND for men with intermediate and high-risk PCa. However, the recently suggested Gleason grading system indicates new changes in the currently used risk stratification [2, 9, 15]. Several original and updated nomograms which predicted the risk of LNI have been proposed [5–8, 11]. However, only some of them are currently used because of the absence of accuracy of prediction and external validation [5, 7, 8].

The calculated 5% risk in used nomograms has been accepted as a reasonable cut-off for PLND [2, 8, 11]. Despite that, for almost 70% of men PLND remains an overtreatment. Finally, with the exception of a few positive studies, until now there is no evidence-based data about the therapeutic effect of PLND [9, 16, 17, 18]. Therefore, an ideal candidate for PLND still has to be defined.

Our initial results using the Memorial Sloan Kettering Cancer Center (MSKCC) preoperative PCa risk nomogram were shortly presented previously [19]. In this manuscript we aim to test the accuracy of online MSKCC nomogram [20] on the prediction of LNI for men undergoing PLND during RP in a more extended patients cohort and assess the possible cut-off for selection of patients for PLND.

MATERIAL AND METHODS

The study cohort consisted of 679 men with clinically localized PCa who underwent RP with PLND between January 2005 and December 2017. Descriptive measurements included preoperative clinical and biopsy data: age, clinical stage (cT), prostate-specific antigen (PSA), primary and secondary biopsy Gleason pattern and percentage of positive cores. The probability of LNI was calculated using MSKCC (https://www.mskcc.org/nomograms/prostate/pre-op) nomogram. The PLND template was changed during the study period. Up to 2012, lPLND removing fatty tissue located within obturator fossa was the most common procedure. Since 2012, the ePLND template has been adapted and involves removal of nodes overlying the external iliac vessels and internal iliac artery and obturator fossa. As an option, areas of the common iliac artery and the presacral region can be included. After surgery, the postoperative Gleason score, pathological stage, lymph node status, number of lymph nodes removed and number of positive lymph nodes were registered. Pathological stage was assessed using the 2002 TNM system, and tumour grading was classified using the revised 2005 Gleason grading system [21] splitting Gleason score 7 in subgroups 3+4 and 4+3. Such grading was performed during the whole study period. No patient received neoadjuvant androgen-deprivation or radiation therapy. The university’s ethical committee approved the prospective collection of the data (protocol Nr. BE-2-48).

Statistical analysis

Means, standard deviations, medians, interquartile ranges and frequencies were used for descriptive statistics. Chi-square and t tests were used to compare difference in means between lymph nodes positive vs. negative patients. The area under curve (AUC) of the receiver operator characteristic analysis was used to quantify the accuracy of MSKCC nomogram to predict LNI. The specificity, sensitivity and negative predictive value (NPV) were calculated for each nomogram-derived LNI probability cut-off. In addition, the extent of over and under estimation of the observed LNI rate was created graphically in logistic calibration plots and a decision curve analysis was performed to evaluate the net benefit associated with the used nomogram. All analyses were performed using the SPSS software (version 20.0, SPSS) and R statistical package (R Project for Statistical Computing). All tests were two sided and p-value of less than 0.05 was considered statistically significant.

RESULTS

Table 1 summarizes the descriptive statistics of this study cohort. Median (quartiles) patients’ age was 65 (60–69) years and median PSA was 10.75 (6.8–15.4) ng/ml. Median number of lymph nodes removed was 6 (4-10). Ten and more lymph nodes were removed (ePLND) in 195 of 679 men (29%). LNI was detected in 81 of 679 (11.9%) cases with a significantly higher rate in men with ePLND in comparison with lPLND (25.1 vs. 6.6%, p <0.0001). Median number of positive nodes was 2 (1–3). Preoperative clinical (PSA, cT) and biopsy (primary and secondary Gleason pattern, percentage of positive scores) characteristics differed comparing patients with and without LNI (p<0.0001). No patients with low risk features (according to D’Amico criteria) had LNI.

Table 1.

Descriptive characteristics of men who underwent lymph node dissection

Parameter	pN0 (n = 598, 88%)	pN1 (n = 81, 12%)	p value	All patients n = 679
Age (yr): median, (IQR)	65 (60–69)	65 (57–69)	0.3	65 (60–69)
PSA (ng/ml): median, (IQR)	10.54 (6.5–14.7)	13.3 (8.4–21.3)	<0.0001	10.75 (6.8–15.4)
Clinical stage: n, (%)  cT1  cT2  cT3	85 (14.2) 359 (60) 153 (25.6)	3 (3.7) 22(27.2) 56 (69.1)	<0.0001	88 (13) 381 (56.1) 209 (30.8)
Biopsy Gleason Score: n, (%)  6  3+4  4+3  8  9–Q10	241 (40.3) 213 (35.3) 49 (8.2) 65 (10.9) 30 (5)	10 (12.3) 20 (24.7) 17 (21) 17 (21) 17 (21)	<0.0001	251 (37) 233 (34.3) 66 (9.7) 82 (12.1) 47 (6.9)
% of positive cores: median, (IQR)	37.5 (18–62.5)	66 (37–87.5)	<0.0001	40 (25–62.5)
D’Amico risk groups: n, (%)  Low  Intermediate  High	30 (5) 334 (55.9) 234 (39.1)	0 (0) 20 (24.7) 61 (75.3)	<0.0001	30 (4.4) 354 (52.1) 295 (43.4)
Pathologic Gleason Score: n, (%)  6  3+4  4+3  8  9-10	104 (17.4) 285 (47.7) 95 (15.9) 54 (9) 60 (10)	1 (1.2) 11 (13.6) 17 (21) 9 (11.1) 43 (53.1)	<0.0001	105 (15.5) 296 (43.6) 112 (16.5) 63 (9.3) 103 (15.2)
Pathologic stage: n, (%)  pT2  pT3a  pT3b  pT4	282 (47.2) 243 (40.6) 72 (12) 1 (0.2)	5 (6.2) 21 (25.9) 53 (65.4) 2 (2.5)	<0.0001	287 (42.3) 264 (38.9) 125(18.4) 3 (0.4)
No. of LN removed: median, (IQR)	6 (4–9)	10 (7–16)	<0.0001	6 (4–10)
MSKCC: median, (IQR)	8 (5–16)	35 (11–50.5)	<0.0001	9 (5–21)

IQR – interquartile range; N0 – negative lymph node; N1 – positive lymph node; PSA – prostate-specific antigen; LN – lymph nodes; MSKCC – Memorial Sloan Kettering Cancer Center nomogram

The accuracy of MSKCC nomogram for prediction of LNI in the study cohort using ROC analysis was 79% (95% CI 73.8–84.2, Figure 1). A calibration plot of the MSKCC score for no LNI versus LNI showed a good calibration of the MSKCC nomogram score in our dataset with a deviation of the observed-predicted plot above a MSKCC nomogram score of 50% in which the observed frequency was lower than the predicted risk (Figure 2). The decision curve analysis demonstrated that MSKCC nomogram improved clinical risk prediction against threshold probabilities of LNI ≤20% (Figure 3).

Figure 1.

Receiving operator characteristic curve for the lymph nodes invasion prediction model. The area under the curve (AUC) for Memorial Sloan Kettering Cancer Center (MSKCC) nomogram is 79% (95% CI 73.8–84.2).

Figure 2.

Calibration plot of observed proportion versus predicted probability of lymph node invasions of the Memorial Sloan Kettering Cancer Center nomogram.

Figure 3.

Decision curve analysis demonstrating the net benefit associated with the use of Memorial Sloan Kettering Cancer Center (MSKCC) nomogram (red line) for the detection of lymph node invasion.

Table 2 summarizes nomogram-derived LNI probabilities. The number of men without and with LNI for each nomogram-derived cut-off is presented. Furthermore, specificity, sensitivity and NPV are calculated for each nomogram-derived LNI probability cut-off. Using the cut-off value of 7%, a PLND could be omitted in 41% of men (279 of 679 cases), and LNI would be missed in 9 of 279 patients (3.2%). Five of these nine men (55.5%) had one, three men (33.3%) had two and one (11.1%) had three positive lymph nodes. Sensitivity, specificity and NPV at the 7% cut-off were 88.9%, 45.2% and 96.8%, respectively.

Table 2.

Analysis of the nomogram-derived cut-offs used to discriminate between men with or without lymph node invasion

Cut-off	Patients in whom PLND is not recommended according to the cut-off (below cut-off)	Patients with LNI (%) that could be missed below cut-off	Patients below cut-off without histologic LNI	Patients below cut-off with histologic LNI	Patients above cut-off without histologic LNI	Patients above cut-off with histologic LNI	NPV
1	4 (0.6)	0	4 (0.7)	0 (0)	594 (99.3)	81 (100)	100
2	37 (5.4)	0	37 (6.2)	0 (0)	561 (93.8)	81 (100)	100
3	94 (13.8)	0	94 (15.7)	0 (0)	504 (84.3)	81 (100)	100
4	150 (22.1)	1.3	148 (24.7)	2 (2.5)	450 (75.3)	79 (97.5)	98.7
5	203 (29.9)	2.5	198 (33.1)	5 (6.2)	400 (66.9)	76 (93.8)	97.5
6	250 (36.8)	2.8	243 (40.6)	7 (8.6)	355 (59.4)	74 (91.4)	97.2
7	279 (41.1)	3.2	270 (45.2)	9 (11.1)	328 (54.8)	72 (88.9)	96.8
8	324 (47.7)	4.0	311 (52)	13 (16)	287 (48)	68 (84)	95.9
9	357 (52.6)	4.5	341 (57)	16 (19.8)	257 (43)	65 (80.2)	95.5
10	378 (55.7)	4.5	361 (60.4)	17 (20.1)	237 (39.6)	64 (79)	95.5
15	464 (68.3)	5.2	440 (73.6)	24 (29.6)	158 (26.4)	57 (70.4)	94.8

LNI – lymph node invasion; NPV – negative predictive value; PLND – pelvic lymph node dissection

DISCUSSION

The recent EAU guidelines (2019) recommend PLND for low and intermediate risk PCa when probability of LNI >5% and for high risk patients in all cases [22]. As more than 50% of men who underwent surgery had low or intermediate risk PCa, the nomogram for prediction of LNI plays essential role in patients’ selection for PLND. The requirements for nomogram are performance status, external validation and possibility to use in daily practice. Despite the fact that up to twenty different nomograms have been proposed, only a few of them are widely used. Very recently Hueting et al. externally validated 16 predictive models in a 1,000 men cohort [13]. The authors found that 2012 Briganti and MSKCC nomograms have the highest AUC (76% and 75%, respectively) and are the most accurate prediction models available. Bandini et al. compared four different nomograms: Cagiannos, Godoy, the 2012 Briganti and the online-MSKCC nomograms. Despite several comprehensive analytical steps, they did not prove that one nomogram is superior to another. Rather, the performance of each nomogram may be better or worse depending on clinical circumstances, including which patients are considered candidates for RP with PLND [23]. No benefits of some nomograms in comparison with others were detected in the recent comparison between Briganti, Partin and MSKCC nomograms for predicting LNI. Meta-analysis included more than 86,000 patients and detected pooled AUCs were 0.793, 0.778 and 0.780, respectively [24]. Two recent novel nomograms including extended biopsy data and pre-biopsy MRI data were proposed by Gandaglia et al. with a higher performance in comparison with MSKCC or 2012 Briganti nomograms [25, 26]. However, the external validation of these nomograms has not yet been done and probably requires a longer time period because of the specific parameters that are not routinely used in daily practice.

In the present study we analyze the online MSKCC nomogram which has been used in our center for a couple of years. The accuracy of MSKCC nomogram for detecting LNI in our cohort of men was high – AUC 79% with prevalence of LNI at 11.9%. Our findings are in agreement with those in other series, where accuracy using MSKCC different nomograms ranged from 74.4% to 86.2% and LNI rate ranged from 3.7% to 28% [7, 11, 13, 17, 27]. The highest 86% discriminated accuracy of the last updated MSKCC nomogram was presented by Godoy et al. in the cohort of 3,721 men with LNI rate 5.2% [11]. We should point out some differences that have become evident comparing our data sets with those presented by Godoy et al. The first one is preoperative characteristics of a patient. A comparison of median PSA values (10.75 vs. 5.31 ng/ml), frequencies of clinical stage cT1 (13% vs. 64%) and Gleason score 8-10 at biopsy (19% vs. 8%) shows higher PCa aggressiveness in our cohort of patients. Another difference is the number of lymph nodes removed. In our cohort, the median number of LN removed was 6 (4–10) compared to 11 (7–16) in the Godoy et al. series. Ten and more LN removed in the presented cohort was in 25% of cases vs. 60% in MSKCC data set. Surprisingly, LNI was found at a higher rate (11.9 vs. 5.2%) in our study cohort. A report by Hueting et al. showed that performance in external cohorts mostly is less comparing with original but reasons for that are not clear [13]. Indeed, patients’ characteristics and PLND template are important comparing AUCs achieved in different data sets.

The other aim of our study was to detect the optimal cut-off to performing PLND. According to the National Comprehensive Cancer Network (NCCN) guidelines, a cut-off is considered acceptable if 50% PLNDs is avoided and 12% men with LNI are missing [28]. Although there is no consensus regarding the optimal cut-off value, the borderline of 5% is generally recommended [2, 8, 29]. We examined several potential online-MSKCC nomogram-derived probability cut-offs to address the validity of previously recommended cut-offs. A detailed analysis of various cut-offs revealed that the optimal cut-off that could be used to discriminate between those in whom a PLND should be performed versus those who could be spared in our cohort should be at 7%. This value is based on the optimal trade-off between the numbers of avoided PLNDs (279 of 679, 41.1%) versus missed patients with LNI within all LNI cases (9 of 81, 11.1%) or within all who avoided PLND (9 of 279, 3.2%). Only one patient of the missed nine with LNI had 3 positive nodes that associated with very aggressive PCa [30]. An alternative cut-off of 5% would decrease the proportion of patients with missed LNI relative to the 7% cut-off (6.2% vs. 11.1% and 2.5% vs 3.2%), but would also decrease the number of spared PLNDs (29.9% vs. 41.1%), respectively. The NPV and sensitivity of both cut-offs remained similar (97.5 vs. 96.8% and 93.8 vs. 88.9%, respectively), but specificity was higher at 7% in comparison with the 5% cut-off – 45.2 vs. 33.1%. Taking into account the recommendations of the NCCN guidelines, the 7% cut-off could be recommended for PLND as optimal.

We should emphasize that nomograms based only on clinical and pathological preoperative parameters are not optimal for LNI prediction. Because of the stage and grade differences between biopsy and postoperative pathology, or the difference in the percentage of positive biopsy cores because of different number of cores taken, we will always miss some cases of LNI irrespective of the cut-off level chosen for PLND. What is even more important is whether very aggressive cancer will be missed at a used threshold. As our data show, only one case of very aggressive PCa (three or more positive nodes) will be missed at the proposed cut-off of 7%. Several recent reports also suggested 7% cut-off as optimal for selecting patients for PLND [25, 26] and some urological associations used an even higher threshold in daily practice [13]. However, the real benefit of the suggested cut-off should be confirmed in more studies that should also include cost effectiveness analysis.

To summarize our findings, we should address some limitations of the present study. The retrospective review of prospectively collected data is one of them. During the study period, the template of PLN has been recently changed from limited at the beginning of the study to extended. A variation between surgeons’ experience or surgical technique may have also biased the results [31, 32]. The number of positive events (LNI = 81) is at the lower boundary which is recommended for validation studies [33]. Finally, some changes in Gleason grading from 2014 could have a bias on the selection of patients for PLND.

Despite these limitations, in our cohort of men LNI detection rate was one of the highest, and predictive probability of nomogram used was within the usual range in comparison with other mentioned studies. The analysis of our data revealed that the threshold of 5% is not optimal because a lot of men at this level remain under high risk for unnecessary PLND. On the other hand, preoperative and postoperative stage and grade differences made all nomograms suboptimal, and new, more stable parameters should be incorporate into LNI predictive models.

Despite the differences between a patient’s preoperative clinical and pathological characteristics and PLND template, the detected high predictive accuracy indicates that MSKCC nomogram could be successfully used in a different than originally presented patients cohort.

CONCLUSIONS

We presented the external validation of MSKCC nomogram demonstrating high discriminative accuracy for prediction of lymph node invasion in men undergoing pelvic lymph node dissection at radical prostatectomy. Using the 7% nomogram cut-off, 41% of patients would avoid lymph node dissection, and lymph node invasion would be missed in 3.2% of patients with a minimal risk to miss a patient with very aggressive cancer features.

STATEMENT OF ETHICS

The study has been reviewed and approved by a certified Ethical Board.

FUNDING SOURCES

This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

AUTHOR CONTRIBUTIONS

Conception/design of the study: D.M; data collection or management: D.M., Z.V.; data analysis: D.M., Z.V, T.M; manuscript writing/editing: D.M, Z.V, T.M, M.J, S.J.

CONFLICTS OF INTEREST

The authors declare no conflicts of interest.