Skip to main content
NIHPA Author Manuscripts logoLink to NIHPA Author Manuscripts
. Author manuscript; available in PMC: 2020 May 27.
Published in final edited form as: BJU Int. 2009 Nov 13;105(12):1654–1659. doi: 10.1111/j.1464-410X.2009.09060.x

Re-calibration and external validation of an existing nomogram to predict aggressive recurrences after radical prostatectomy

Florian R Schroeck 1,2, Michael W Kattan 3, Judd W Moul 1, William J Aronson 4,5, Joseph C Presti Jr 6,7, Martha K Terris 8,9, Christopher J Kane 10, Christopher L Amling 11, Leon Sun 1,2, Stephen J Freedland 1,2,12
PMCID: PMC7252932  NIHMSID: NIHMS1587205  PMID: 19912203

Abstract

OBJECTIVE

To re-calibrate the previously published Duke Prostate Center (DPC) nomogram for the prediction of biochemical recurrence (BCR) after radical prostatectomy (RP) to not only predict overall BCR but also the clinically more relevant endpoint of an aggressive recurrence (i.e. a BCR with a postoperative PSA doubling time (PSADT) of < 9 months).

PATIENTS AND METHODS

Using the established point-scale system based upon the previously published DPC nomogram, we re-calibrated this point system to predict not just BCR, but also aggressive BCR within 2599 men treated with RP from the DPC database. PSADT was computed on all patients meeting the recurrence definition who had a minimum of two PSA values, separated by at least 3 months, and ≤ 2 years after recurrence. External validation was performed using data from 1695 men treated with RP within the Shared Equal Access Regional Cancer Hospital (SEARCH) database by calculating the concordance index c and by plotting calibration curves.

RESULTS

The median follow-up for patients with no BCR was 56 and 47 months for DPC and SEARCH, respectively. In the DPC modelling cohort and the SEARCH validation cohort, 645 (25%) and 557 (33%) men had BCR, while 83 (3.2%) and 71 (4.2%) patients had an aggressive recurrence. In external validation, predictive accuracy for an aggressive BCR was high (c= 0.83) and the nomogram showed good calibration.

CONCLUSIONS

We re-calibrated an existing nomogram to not only predict overall BCR after RP but also aggressive recurrence after RP. Our new tool can provide valuable information for patient counselling and patient selection for adjuvant therapy trials.

Keywords: prostate cancer, biochemical recurrence, PSA doubling time, aggressive recurrence, nomogram

INTRODUCTION

Despite early detection and advances in surgical technique, one out of three men will develop biochemical recurrence (BCR) after radical prostatectomy (RP) [1]. Randomized trials have shown that adjuvant radiation therapy in high-risk patients can significantly decrease the risk for BCR [2,3]. In the past, much effort was put into developing nomograms to identify patients at high risk for BCR [4]. However, not all men with BCR after RP ultimately die of prostate cancer [5]. Although patients at high risk for prostate cancer death can be identified based on a short PSA doubling time (PSADT) after BCR [5], it would be most important to identify these patients before BCR so they can receive adjuvant radiation therapy or be included in adjuvant therapy trials. To this end, we have found previously that current nomograms designed to predict overall BCR could in fact identify patients with aggressive recurrences (i.e. recurrence with a short PSADT) with even higher accuracy [6]. This is fortunate, as there is concern that it is increasingly difficult for the clinician to decide the best nomogram for his practice from a growing number of published models [7]. Rather than developing an additional new nomogram, but recognizing the importance of identifying men with a short PSADT who are at greatest risk of prostate cancer death, we therefore re-calibrated and externally validated an existing nomogram [8]. We used pre- and postoperative characteristics to not only predict overall BCR but also aggressive BCR defined as a BCR with a short PSADT after RP.

PATIENTS AND METHODS

The modelling cohort consisted of patients who underwent RP between 1988 and 2008 and who were not treated with neoadjuvant or adjuvant therapy before BCR. Their data were retrieved from the institutional review board approved Duke Prostate Center (DPC) database. As a recent history of previous TURP is likely to significantly lower preoperative PSA values, patients with clinical stage T1a and T1b (n = 24) were excluded. A total of 230 patients with missing preoperative PSA value and 35 patients with missing postoperative Gleason score were excluded. As patients with positive lymph nodes (LNs) are at high risk for aggressive prostate cancer regardless of their other risk factors, 52 patients with positive LNs were excluded. Patients who were missing any of the prediction variables essential for the existing nomogram [8] were excluded as well. Therefore, patients were excluded for missing surgical margin data (n = 176), seminal vesicle involvement (n = 243), extracapsular extension status (n = 5), prostate weight (n = 213), or race (n = 2), leaving 2599 patients available for analysis in the DPC modelling cohort. These patients were followed after RP per the attending urologist’s discretion with serial PSA determinations, and all postoperative PSA values were entered into the database.

The external validation cohort consisted of combined data from patients undergoing RP between 1988 and 2008 at the Veterans Affairs Medical Centers in West Los Angeles and Palo Alto, California, Augusta, Georgia, Durham, North Carolina, and Birmingham, Alabama which were collected in the Shared Equal Access Regional Cancer Hospital (SEARCH) database after obtaining Institutional Review Board approval from each institution to abstract and combine data [9]. Patients who had undergone neoadjuvant therapy were excluded. Similar to the modelling cohort, patients with clinical stage T1a and T1b (n = 48) or positive LNs (n = 31), or missing data for preoperative PSA value (n = 79), postoperative Gleason score (n = 132), surgical margin status (n = 23), seminal vesicle involvement (n = 11), extracapsular extension status (n = 11), prostate weight (n = 213), or race (n = 2) were excluded, leaving 1695 patients for analysis.

Within the DPC database, BCR was defined as a PSA level of ≥0.2 ng/mL at least 30 days after RP. In SEARCH, BCR was defined as a single PSA level of >0.2 ng/mL, two concentrations at 0.2 ng/mL, or secondary treatment for an elevated and/or rising PSA after RP. Men who received adjuvant treatment with an undetectable PSA level were censored as not recurred at the time of treatment in SEARCH.

For both databases, PSADT was calculated by dividing the natural log of 2 (0.693) by the slope of the linear regression line of the natural log of PSA over time [10]. PSADT was computed on all patients meeting the recurrence definition who had a minimum of two PSA values, separated by at least 3 months, and within 2 years after recurrence. All PSA values within the first 2 years after recurrence were used to calculate PSADT. For patients starting salvage hormone or radiation therapy within this time, only PSA values before salvage therapy were used. Therefore, all PSA values used were at least 0.2 ng/mL and obtained before subsequent treatment. Patients with a decline or no change in PSA level, or with a very long PSADT (>100 months) were assigned a value of 100 months for ease of calculations (n = 180 for DPC and n = 57 for SEARCH).

We a priori defined aggressive recurrence as recurrence with a PSADT of <9 months, based upon the high prostate cancer-specific mortality in a previous study [11]. Patients with no BCR were censored as not having an aggressive recurrence at the time of last follow-up. Patients with BCR but a PSADT of ≥9 months were censored as not having an aggressive recurrence at the time of their BCR, as these men were at low risk of prostate cancer death in a previous study [11]. Patients with BCR but an unavailable PSADT (n = 295 for DPC and n = 323 for SEARCH) could not be classified and were excluded from all analyses.

To re-calibrate the existing nomogram for prediction of aggressive recurrence, we calculated the nomogram based risk score (i.e. linear predictor) for each patient within the modelling cohort [8]. The risk score was calculated using the formula: 0.4366* natural logarithm of PSA + 0.7172 if surgical margin is positive + 0.5447 if seminal vesicle invasion is present + 0.2016 if extracapsular extension is present + 0.3233 if Gleason 3 + 4 + 0.8030 if Gleason 4 + 3 + 0.9877 if Gleason >7 −0.3721* natural logarithm of prostate weight +0.3140 if African American −0.0542* year of surgery. We then used this risk score as the single predictor variable in a Cox model for aggressive recurrence. Additional axes for the prediction of aggressive recurrence, which were based on this new Cox model, were then added to the existing previously published nomogram. We did not adjust or re-calibrate the axes for predicting BCR.

We performed external validation using the validation cohort (i.e. the SEARCH cohort). We calculated the concordance index c to determine predictive accuracy for the prediction of aggressive recurrence. This index provides the percentage of correct predictions by the model with values of 0.5 equalling the flip of a coin and 1.0 representing all patients predicted correctly. Based on the re-calibrated nomogram, we calculated each patient’s individual predicted risk for aggressive recurrence at 1, 3, and 5 years. These predicted risks were compared with the actual Kaplan–Meier based risk by plotting calibration plots.

Kaplan–Meier survival curves were plotted for the cohorts using either BCR or aggressive recurrence as the event. All statistical analysis was performed using STATA 9.2 (STATACorp, College Station, TX) and R 2.7.1 [12] with the packages ‘Hmisc’ and ‘Design’ [13].

RESULTS

CLINICOPATHOLOGICAL VARIABLES

The clinicopathological variables of the recalibration and external validation cohorts are given in Table 1. The DPC database included markedly fewer African-Americans, but overall the cohorts were relatively similar. The median follow-up for patients with no BCR was 56 months for DPC and 47 months for SEARCH. In both cohorts, patients with BCR tended to have higher risk disease (Table 1). In all, 645 (25%) and 557 (33%) patients had BCR in DPC and SEARCH, respectively. Of these, 83 (24% of the 350 patients with a recurrence and PSADT available) and 71 (30% of the 234 patients with a recurrence and PSADT available) had an aggressive recurrence in DPC and SEARCH, respectively. The median PSADT was 4.9 and 5.2 months for patients with aggressive recurrence and 55 and 23 months for patients with a non-aggressive recurrence in DPC and SEARCH, respectively. The actuarial BCR-free survival at 10 years was 67% (95% CI 65–70%) for DPC and 54% (95% CI 50–57%) for SEARCH (Fig. 1). Risk for aggressive recurrence was markedly lower with an aggressive recurrence-free survival of 95% (95% CI 94–96%) in DPC and of 93% (95% CI 90–95%) in SEARCH at 10 years (Fig. 1).

TABLE 1.

Clinicopathological variables of the two patient populations*

DPC database
SEARCH database
Variable All men Men with BCR All men Men with BCR
N 2599 645 1695 557
Median (IQR):
 Year of Surgery 2000 (1996–2004) 1998 (1994–2003) 2002 (1998–2004) 2000 (1997–2003)
 Age at surgery, years 63 (57–68) 64 (59–70) 62 (57–66) 62 (57–67)
 Length of follow-up, months 40.7 (15.2–87.0) 12.2 (3.0–31.8) 36.0 (12.0–66.0) 13.0 (4.0–38.0)
 PSA level, ng/mL 6.2 (4.5–9.2) 8.8 (5.8–16.7) 7.1 (5.0–10.7) 8.9 (5.9–14.8)
 Prostate weight, g 38 (31–50) 36 (29–49) 39 (31–51) 38 (30–49)
N (%):
 African-American 385 (15) 110 (17) 739 (44) 263 (47)
 Pathological Gleason score:
  2–6, 1188 (46) 161 (25) 681 (40) 129 (23)
  3 + 4 850 (33) 210 (33) 659 (39) 239 (43)
  4 + 3 285 (11) 114 (18) 176 (10) 88 (16)
  8–10 276 (11) 160 (25) 179 (11) 101 (18)
 Positive surgical margin 918 (35) 367 (57) 763 (45) 359 (65)
 Extracapsular extension 830 (32) 336 (52) 350 (21) 182 (33)
 Seminal vesicle invasion 254 (10) 171 (27) 169 (10) 114 (21)
*

Percentages may not add up to 100% due to rounding.

IQR, interquartile range.

FIG. 1.

FIG. 1.

Overall freedom from BCR (lower curves, green) and freedom from aggressive recurrence (upper curves, red) in the DPC database (panel A) and in SEARCH (panel B). For all curves, 95% CIs are indicated and the number of patients at risk is given for various time points.

NOMOGRAM, VALIDATION, AND CALIBRATION

The hazard ratio of the nomogram risk score for the prediction of aggressive recurrence was 5.92 (95% CI 4.63–7.57) and 4.87 (95% CI 3.68–6.46) in the DPC modelling cohort and the SEARCH validation cohort, respectively.

Figure 2 shows the re-calibrated nomogram with axes for prediction of both overall BCR as well as aggressive BCR. As can be seen in both the Kaplan–Meier curves (Fig. 1) as well as the nomogram (Fig. 2), overall BCR continues to occur past 3 years, whereas aggressive recurrences tend to occur early and therefore the 3- and 5-year risks for aggressive recurrence are essentially the same.

FIG. 2.

FIG. 2.

External calibration of the nomogram for the prediction of aggressive recurrence at 12, 36, and 60 months within the SEARCH validation cohort. The nomogram predicted freedom from aggressive recurrence (x-axis) is compared with the actual Kaplan–Meier estimate (y-axis) within the SEARCH validation cohort.

Predictive accuracy of the nomogram for prediction of aggressive recurrence was high in external validation (c = 0.83). Similarly, the nomogram showed good calibration in external validation (Fig. 3).

FIG. 3.

FIG. 3.

Nomogram for the prediction of both overall BCR as well as aggressive BCR (BCR with a PSADT of < 9 months).

DISCUSSION

We re-calibrated and externally validated an existing nomogram to not only predict overall BCR after RP, but also aggressive recurrence defined by a PSADT of < 9 months. Although aggressive recurrences can be identified based on a short PSADT after BCR [5], our nomogram can help identify patients at high risk for an aggressive recurrence based on clinicopathological variables before a patient actually develops BCR. Knowing the risk for aggressive recurrence in addition to the risk of overall BCR can significantly aid in patient counselling when adjuvant therapy is considered. Although salvage radiation therapy can be effective in patients with aggressive recurrences and a short PSADT [14], recent randomized trials suggest adjuvant therapy as the ideal therapy in patients with higher risk prostate cancer [2,3].

Regarding prediction of overall BCR with the current nomogram, predictive accuracy has been previously examined. In internal validation, the concordance index c was 0.75 [8], and subsequent external validation within the SEARCH cohort confirmed a high predictive accuracy (c = 0.75) [6].

Interestingly, despite that this nomogram was initially developed for the prediction of overall BCR, its accuracy for predicting aggressive recurrences is higher than for overall BCR (0.89 vs 0.75 for DPC and 0.83 vs 0.75 for SEARCH). We recently examined nine models that were developed for the prediction of BCR and found that all were able to predict aggressive BCR with higher accuracy [6]. We hypothesized this might be because the common predictor variables used in these nomograms (i.e. PSA level, stage, grade, etc.) mainly represent cancer biology, which are more closely associated with aggressive recurrences than overall BCR [6].

Our approach of re-calibrating an existing nomogram is supported by a recent study, which found that the updated ‘Kattan postoperative nomogram’ [15] developed for the prediction of BCR was also very good at predicting clinical failure defined as developing metastases, a rising PSA level on androgen deprivation therapy, or prostate cancer specific death [16].

Nevertheless, current prediction tools for metastasis or prostate cancer specific death rely heavily on PSA kinetics after BCR [17]. However, these tools cannot be applied until a patient has developed a BCR. As such, the current nomogram has the advantage of relying on readily available clinicopathological predictor variables, which are available immediately after RP while still estimating an outcome (BCR with a short PSADT) that has been shown to be strongly associated with a high-risk for prostate cancer death [11]. Therefore, patients at significant risk for prostate cancer-specific death can be identified earlier in their disease process and counselled regarding adjuvant therapy.

The present study has several limitations. While overall calibration was good, there was a trend towards underestimating the event rate in the external validation cohort (Fig. 3). At 5 years, maximal underestimation of aggressive recurrence was ≈5%.

About 300 patients in each database had a BCR but had no PSADT data available. These patients could not be classified regarding the aggressiveness of their recurrence and therefore had to be excluded. It is possible that if data were available on all these patients, it could have influenced the recalibration of the nomogram and its predictive accuracy. However, we have examined this problem previously and found in several sensitivity analyses that predictive accuracy remains high whether patients with BCR and missing PSADT are excluded, treated as having an aggressive recurrence, or censored as having a non-aggressive recurrence [6].

Some of the patients might have undergone adjuvant therapy without our knowledge and therefore their PSADT might have been recorded as higher than appropriate. Therefore, we might have underestimated the number of aggressive recurrences. However, the percentage of men who recurred with an aggressive recurrence in the present study (24% and 30% using our definition in DPC and SEARCH, respectively) was relatively similar to previous studies (38% with PSADT of <9 months [11] and 38% with PSADT of <12 months [18]). In addition, accuracy for the prediction of an aggressive recurrence remained high in external validation in SEARCH, which implies that misclassification in either database was probably not a major problem. Nevertheless, further external validation of the current nomogram in other independent datasets should be performed.

Finally, what degree of risk for recurrence merits treatment or enrolment into a trial is unknown. The decision to treat is up to the discretion of the patient and physician. Factors to consider include, but are not limited to, the risk of aggressive BCR, the benefit of the treatment, individual patient preferences, and the age and overall health of the patient (i.e. the expected life-span of the patient in the absence of aggressive recurrence).

In conclusion, we re-calibrated an existing nomogram to not only predict overall BCR but also aggressive recurrence after RP. External validation showed a high predictive accuracy as well as good calibration. Our new tool can provide valuable information for patient counselling and patient selection for adjuvant therapy trials.

ACKNOWLEDGEMENTS

Research support: Department of Defense, Prostate Cancer Research Program (S.J.F.); Department of Veterans Affairs, National Institute of Health R01CA100938 (W.J.A.), NIH Specialized Programs of Research Excellence Grant P50 CA92131-01A1 (W.J.A.), the Georgia Cancer Coalition (M.K.T.), the American Urological Association Foundation/ Astellas Rising Star in Urology Award (S.J.F.), and Committee for Urologic Research, Education, and Development (CURED) at Duke University (F.R.S., J.W.M., L.S., S.J.F.).

Abbreviations

VA

Veterans Affairs

DPC

Duke Prostate Center

BCR

biochemical recurrence

RP

radical prostatectomy

PSADT

PSA doubling time

SEARCH

Shared Equal Access Regional Cancer Hospital

LN

lymph node

Footnotes

CONFLICT OF INTEREST

None declared.

Disclaimers: Views and opinions of, and endorsements by the author(s) do not reflect those of the US Army or the Department of Defense.

REFERENCES

  • 1.Han M, Partin AW, Pound CR, Epstein JI, Walsh PC. Long-term biochemical disease-free and cancer-specific survival following anatomic radical retropubic prostatectomy. The 15-year Johns Hopkins experience. Urol Clin North Am 2001; 28: 555–65 [DOI] [PubMed] [Google Scholar]
  • 2.Thompson IM Jr, Tangen CM, Paradelo J et al. Adjuvant radiotherapy for pathologically advanced prostate cancer: a randomized clinical trial. JAMA 2006; 296: 2329–35 [DOI] [PubMed] [Google Scholar]
  • 3.Bolla M, van Poppel H, Collette L et al. Postoperative radiotherapy after radical prostatectomy: a randomised controlled trial (EORTC trial 22911). Lancet 2005; 366: 572–8 [DOI] [PubMed] [Google Scholar]
  • 4.Chun FK, Karakiewicz PI, Briganti A et al. Prostate cancer nomograms: an update. Eur Urol 2006; 50: 914–26 [DOI] [PubMed] [Google Scholar]
  • 5.Freedland SJ, Humphreys EB, Mangold LA et al. Death in patients with recurrent prostate cancer after radical prostatectomy: prostate-specific antigen doubling time subgroups and their associated contributions to all-cause mortality. J Clin Oncol 2007; 25: 1765–71 [DOI] [PubMed] [Google Scholar]
  • 6.Schroeck FR, Aronson WJ, Presti JC Jr et al. Do nomograms predict aggressive recurrence after radical prostatectomy more accurately than biochemical recurrence alone? BJU Int 2009; 103: 603–8 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 7.Ross RW, Kantoff PW. Predicting outcomes in prostate cancer: how many more nomograms do we need? J Clin Oncol 2007; 25: 3563–4 [DOI] [PubMed] [Google Scholar]
  • 8.Schroeck FR, Sun L, Freedland SJ, Jayachandran J, Robertson CN, Moul JW. Race and prostate weight as independent predictors for biochemical recurrence after radical prostatectomy. Prostate Cancer Prostatic Dis 2008; 11: 371–6 [DOI] [PubMed] [Google Scholar]
  • 9.Freedland SJ, Jalkut M, Dorey F, Sutter ME, Aronson WJ. Race is not an independent predictor of biochemical recurrence after radical prostatectomy in an equal access medical center. Urology 2000; 56: 87–91 [DOI] [PubMed] [Google Scholar]
  • 10.Pound CR, Partin AW, Eisenberger MA, Chan DW, Pearson JD, Walsh PC. Natural history of progression after PSA elevation following radical prostatectomy. JAMA 1999; 281: 1591–7 [DOI] [PubMed] [Google Scholar]
  • 11.Freedland SJ, Humphreys EB, Mangold LA et al. Risk of prostate cancer-specific mortality following biochemical recurrence after radical prostatectomy. JAMA 2005; 294: 433–9 [DOI] [PubMed] [Google Scholar]
  • 12.R-Development-Core-Team. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing, 2008 [Google Scholar]
  • 13.Harrell FE. Design Package (R Package, Version 2.1–1) and Harrell Miscellaneous Package (R Package, Version 3.4–2) 2007 [Google Scholar]
  • 14.Stephenson AJ, Shariat SF, Zelefsky MJ et al. Salvage radiotherapy for recurrent prostate cancer after radical prostatectomy. JAMA 2004; 291: 1325–32 [DOI] [PubMed] [Google Scholar]
  • 15.Stephenson AJ, Scardino PT, Eastham JA et al. Postoperative nomogram predicting the 10-year probability of prostate cancer recurrence after radical prostatectomy. J Clin Oncol 2005; 23: 7005–12 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 16.Eggener SE, Vickers AJ, Serio AM et al. Comparison of models to predict clinical failure after radical prostatectomy. Cancer 2009; 115: 303–10 [DOI] [PMC free article] [PubMed] [Google Scholar]
  • 17.Shariat SF, Karakiewicz PI, Roehrborn CG, Kattan MW. An updated catalog of prostate cancer predictive tools. Cancer 2008; 113: 3075–99 [DOI] [PubMed] [Google Scholar]
  • 18.Roberts SG, Blute ML, Bergstralh EJ, Slezak JM, Zincke H. PSA doubling time as a predictor of clinical progression after biochemical failure following radical prostatectomy for prostate cancer. Mayo Clin Proc 2001; 76: 576–81 [DOI] [PubMed] [Google Scholar]

RESOURCES