Do Nomograms Predict Aggressive Recurrence after Radical Prostatectomy More Accurately than Biochemical Recurrence Alone?

Abstract

Purpose

To compare the predictive accuracy of existing models in estimating risk of biochemical recurrence (BCR) vs. aggressive recurrence (BCR with a PSADT <9 months).

Methods

We included 1550 men treated with RP between 1988 and 2007 within the SEARCH database. The predictive accuracy of 9 different risk stratification models for estimating risk of BCR and risk of aggressive recurrence after RP was assessed using the concordance index c.

Results

The 10-year risks for BCR and aggressive recurrence were 47% and 9%, respectively. Across all 9 models tested, the predictive accuracy was on average 0.054 higher (range 0.024 to 0.074) for predicting aggressive recurrence than for predicting BCR alone (concordance index c=0.756 vs. 0.702). Similar results were obtained in four sensitivity analyses, (1) defining patients with BCR but unavailable PSADT (n=220) as having aggressive recurrence, (2) defining these patients as not having aggressive recurrence, (3) defining aggressive recurrence as PSADT <6 months or (4) defining aggressive recurrence as PSADT <12 months. Improvement of predictive accuracy was greater for preoperative models than for postoperative models (0.053 vs. 0.036, p=0.03).

Conclusion

Across 9 different models, prediction of aggressive recurrence after RP was more accurate than prediction of BCR alone. This is likely due to the fact current models mainly assess cancer-biology, which correlates better with aggressive recurrence than with BCR alone. Overall, all models had relatively similar accuracy for predicting aggressive recurrence.

Introduction

Despite early detection and aggressive intervention, 1 of 3 men will develop a biochemical recurrence (BCR) after radical prostatectomy (RP) [1, 2]. Ultimately, there are 2 major reasons men may experience BCR after surgery. The first is advanced stage and aggressive disease (i.e. poor biology). The second is poor surgical technique. We hypothesized men who develop a recurrence due to poor technique alone in the absence of poor biology are less likely to have an aggressive recurrence. Alternatively, men who recur with an aggressive recurrence likely do harbor poor biology. As such, it is noteworthy that much effort has been put into developing nomograms and risk stratification tools to identify men who are likely to develop BCR [3–9]. However, these models use almost exclusively variables associated with cancer biology, such as stage, grade, PSA [10]. Therefore, we hypothesized currently available models, which in fact estimate poor biology, will predict aggressive recurrence with greater accuracy than overall recurrence. Fortunately, aggressive disease can be identified early in the disease course based upon a short post-operative PSA doubling time (PSADT) [11–13]. Indeed, though time to PSA recurrence correlates with prostate cancer specific survival [14], it is not a surrogate for cancer death. On the contrary, a very short PSADT is a surrogate for prostate cancer death [11]. However, whether currently available nomograms designed to predict BCR predict the more clinically relevant end-point of aggressive recurrence defined by short PSADT is unknown.

To test this, we used seven different previously published models as well as basic pre- and post-operative models which include preoperative PSA, stage, and grade and assessed their ability to predict an aggressive recurrence (i.e. recurrence with a post-operative PSADT <9 months) versus their ability to predict any BCR among men undergoing RP within the SEARCH database.

Patients and Methods

Study population

After obtaining institutional review board approval from each institution to abstract and combine data, we combined data from patients undergoing RP between 1988 and 2007 at the Veterans Affairs Medical Centers in West Los Angeles and Palo Alto, California, Augusta, Georgia, and Durham, North Carolina into the SEARCH database [15]. Patients treated with preoperative androgen deprivation or radiation therapy were excluded. Men with missing follow-up data (n=59) or recurrence data (n=2) could not be classified and were therefore excluded. As a recent history of previous transurethral resection of the prostate is likely to significantly lower preoperative PSA values, men with clinical stage T1a or T1b (n=46) were excluded. Because men with node positive disease are likely to harbor aggressive disease and are at high-risk of prostate cancer death, they were excluded (n=23). We also excluded 54 and 13 men with missing preoperative PSA data or data for preoperative and postoperative Gleason score, respectively, resulting in a study population of 1550 men.

Definition of biochemical recurrence and aggressive recurrence

BCR was defined as a single PSA >0.2 ng/ml, 2 concentrations at 0.2 ng/ml, or secondary treatment for an elevated and/or rising postoperative PSA. Men who received adjuvant treatment with an undetectable PSA were censored as not recurred at the time of treatment. PSADT was calculated by dividing the natural log of 2 (0.693) by the slope of the linear regression line of the natural log of PSA over time [16]. PSADT was computed on all patients meeting the recurrence definition who had a minimum of 2 PSA values, separated by at least 3 months, and within 2 years after recurrence. All PSA values within the first 2 years after recurrence were used to calculate PSADT. For patients starting salvage hormone or radiation therapy within this time, only PSA values before salvage therapy were used. Therefore, all PSA values used were at least 0.2 ng/mL and obtained before subsequent treatment. Patients with a decline or no change in PSA, or with very long PSADT (>100 months) were assigned a value of 100 months for ease of calculations (n=51).

We a priori defined aggressive recurrence as recurrence with a PSADT <9 months, based upon the high prostate cancer-specific mortality seen in a prior study [12]. Patients without BCR were censored as not having an aggressive recurrence at the time of last follow-up. Patients with BCR but a PSADT of ≥9 months were censored as not having an aggressive recurrence at the time of their recurrence, as these men were at low risk of prostate cancer death in a prior study [12].

Statistical analysis

Predictive accuracy of multiple risk stratification models for estimating risk of BCR and risk of aggressive recurrence after RP was assessed by calculating Harrell’s concordance index c for each of these models for both end-points. This index provides the percentage of correct predictions by the model with values of 0.5 equaling the flip of a coin and 1.0 representing all patients predicted correctly. Predictive accuracy of the following nine models was assessed: a basic clinical model (based on clinical stage, biopsy Gleason score, and log-transformed preoperative PSA) developed in this patient cohort, a basic pathological model (based on pathological stage, prostatectomy Gleason sum, and log-transformed preoperative PSA) developed in this patient cohort, the D’Amico risk classification [3], Kattan’s preoperative [4] and postoperative [5] nomograms, a nomogram developed by our team based on a separate patient cohort from the Duke Prostate Center database [6], the CAPRA model [7], which has been previously validated in the SEARCH cohort for predicting BCR [17], a model from the Center for Prostate Disease Research (CPDR) [8], and a model from Johns Hopkins Hospital (JHH) [9]. Thus, none of the previously published models were developed based on data from the SEARCH database.

For our basic clinical and pathological models, we determined the risk of overall recurrence using a multivariate Cox proportional hazards model with only the variables described above included. Each patient was then assigned a risk score which was equivalent to the sum of the regression coefficients from the multivariate Cox model. This risk score was then assessed as a single continuous variable. For all other models we used the respective published formulas to calculate continuous risk scores for each patient.

Patients (n=220) who had a BCR but missing PSADT were excluded from the primary analysis. However, sensitivity analyses were performed redefining these patients as (1) having aggressive recurrence at the time of BCR or (2) as remaining without aggressive recurrence at the time of BCR and therefore censoring them at the time of BCR. Further sensitivity analyses were performed using different cut-off points for PSADT (<6 months and <12 months) to define aggressive recurrence. A cut-off of <3 months was not tested, as only 13 patients recurred with a PSADT<3 months.

Kaplan-Meier survival curves were plotted for the cohort using BCR and aggressive recurrence as the event. All statistical analysis was performed using STATA 9.2 (STATACorp, College Station, TX) and R 2.5.1[18] with the packages “Hmisc” and “Design”.[19]

Results

Clinicopathological variables

The clinicopathological variables of the study population are described in table 1. Approximately 40% of the patients had low risk, 41% intermediate risk, and 20% high risk disease, by the D’Amico classification. Median follow-up among non-recurrent patients was 52 months, during which 522 patients (34%) experienced a BCR. Patients with BCR tended to have higher risk disease compared to the entire cohort (table 1). Only 86 patients (7% of the whole patient population and 29% of the 302 patients with a recurrence and PSADT available) experienced an aggressive recurrence according to our definition. Figure 1 shows the recurrence-free survival curves for the end-points of biochemical and aggressive recurrence. The risks of overall recurrence and aggressive recurrence at 10 years were 47% (95% CI 43% – 50%) and 9% (95% CI 7% – 12%), respectively. Among men who recurred, median time to BCR was 17 months whereas among those who had an aggressive recurrence, median time to recurrence was 9 months. The median PSADTs for patients with aggressive and non-aggressive recurrence were 5 and 23 months, respectively.

Table 1.

Clinicopathological variables of the patient population.^*

	All men (n=1550)	Men with BCR (n=522)
Year of Surgery, median (IQR)	2000 (1996–2003)	1998 (1994–2001)
Age at surgery (years), median (IQR)	62 (57–66)	63 (58–67)
Length of follow up (months), median (IQR)	39 (13–72)	17 (5–41)
African American, number (%)	652 (42)	233 (45)
PSA (ng/ml), median (IQR)	7.3 (5.0–11.1)	9.4 (6.0–15.7)
Biopsy Gleason score
2–6, number (%)	961 (63)	254 (50)
3+4, number (%)	320 (21)	127 (25)
4+3, number (%)	106 (7)	55 (11)
8–10, number (%)	138 (9)	72 (14)
D’Amico risk
low, number (%)	609 (40)	118 (23)
intermediate, number (%)	624 (41)	248 (48)
high, number (%)	298 (20)	148 (29)
Pathological Gleason score
2–6, number (%)	620 (40)	134 (26)
3+4, number (%)	573 (37)	209 (40)
4+3, number (%)	173 (11)	84 (16)
8–10, number (%)	170 (11)	90 (17)
Positive Surgical Margin, number (%)	710 (47)	320 (63)
Extracapsular Extension, number (%)	374 (25)	198 (39)
Seminal Vesicle Invasion, number (%)	157 (10)	110 (22)
Prostate Weight, median (IQR)	39.0 (30.2–50.2)	38.0 (28.7–48.2)

^*Percentages may not add up to 100% secondary to rounding. BCR = biochemical recurrence, IQR = interquartile range

Figure 1.

Recurrence free survival curves for aggressive recurrence (upper curve) and biochemical recurrence (lower curve). For both curves, 95% confidence intervals are indicated and the number of patients at risk is given for various time points.

Predictive accuracy

The predictive accuracy (Harrell’s concordance index c) for estimating risk of biochemical and aggressive recurrence is given in table 2 for all nine models tested. The best performing models for prediction of BCR and aggressive recurrence in this patient cohort were the Duke Prostate Center model (c=0.748) and the postoperative Kattan nomogram (c=0.777), respectively. Overall, all models worked reasonably well to predict BCR with a difference in predictive accuracy between the best and worst model of 0.088. For all nine models, Harrell’s c was higher (average 0.054, range 0.024 to 0.074) for predicting aggressive recurrence than for predicting BCR alone. Moreover, the predictive accuracy of all the models for estimating risk of aggressive recurrence was quite good with a narrower range such that the difference between the best and worst model was only 0.049.

Table 2.

Predicitive accuracy (Harrell’s concordance index c) of various models for the prediction of biochemical and aggressive recurrence.

Model	c for prediction of overall recurrence	c for prediction of aggressive recurrence	Difference	no. patients in model
Basic preoperative model	0.699	0.758	0.059	1235
D’Amico classification [3]	0.660	0.728	0.068	1313
Kattan preoperative nomogram [4]	0.701	0.750	0.049	1297
Kattan postoperative nomogram [5]	0.724	0.777	0.054	999
Basic postoperative model	0.717	0.770	0.052	1299
Duke Prostate Center model [6]	0.748	0.772	0.024	1157
CAPRA [7]	0.694	0.767	0.074	1148
CPDR model [8]	0.687	0.730	0.043	1302
JHH model [9]	0.689	0.756	0.067	1001
Average:	0.702	0.756	0.054	1195

CAPRA = Cancer of the Prostate Risk Assessment, CPDR = Center for Prostate Disease Research, JHH = Johns Hopkins Hospital

As 220 men had experienced a BCR but had no PSADT available, they were unable to be classified and therefore were excluded from our primary analysis. However, we performed sensitivity analyses in which these men were included but classified as having an aggressive recurrence. Using this approach, all 9 models still performed an average of 0.017 better (range 0.004 to 0.043) in predicting aggressive recurrence than BCR (data not shown). When these 220 men were included but censored as not having an aggressive recurrence, Harrell’s c was an average 0.053 higher (range 0.022 to 0.083) for all 9 models for predicting aggressive recurrence than BCR (data not shown).

Further sensitivity analyses were performed using cut-offs of <12 and <6 months for defining aggressive recurrence. The average predictive accuracy of all 9 models for estimating risk of aggressive recurrence was high and similar across all three definitions (c=0.756, c=0.742, c=0.739, for cut-offs of <9 months, <12 months, and <6 months, respectively). Again, all models performed an average of 0.040 better (range 0.017 to 0.057) in predicting aggressive recurrence vs. BCR alone when <12 months were used (data not shown). All but one model (the Duke Prostate Center model [6]) performed better (average 0.037, range −0.011 to 0.071) when <6 months was used (data not shown). Across all three definitions of aggressive recurrence (i.e. PSADT <6, <9, or <12 months), preoperative models had a significantly higher improvement in predictive accuracy than postoperative models for estimating aggressive recurrence relative to their ability to predict BCR alone (0.053 improvement vs. 0.036 improvement, Wilcoxon rank-sum p=0.03).

Discussion

We found existing models designed to estimate risk of BCR actually had a greater degree of predictive accuracy for estimating risk of aggressive recurrence than for estimating risk of BCR. In our main analysis, the nine models tested were an average 5.4% more accurate in estimating risk of aggressive recurrence vs. BCR. Overall, improvement in predictive accuracy for aggressive recurrence vs. BCR was highest for models which were based only upon preoperative characteristics. Since a very short PSADT is a surrogate for prostate cancer death [11], the current findings are encouraging in that all models tested performed well at predicting these life-threatening aggressive recurrences.

Previous studies have shown that not all men with BCR are at high risk for prostate cancer death. For example, prior studies found overall 10-year survival is only 20–30% for men with a PSADT <3 months compared to >90% for men with a PSADT >15 months [11, 13]. Although aggressive disease can be identified relatively early in the disease course based upon a short post-operative PSADT [11–13], it is most important to predict clinically relevant recurrences (i.e. those with a short PSADT which are most likely to progress) prior to BCR, in order to identify candidates for adjuvant therapy and clinical trials. Unfortunately, most models used today were designed to predict BCR, not specifically high-risk BCR. However, given the findings of the present study, an individual predicted to be at significant risk for BCR based upon a currently available nomogram will also be at higher risk for aggressive recurrence.

We hypothesized that men who experience a BCR secondary to poor surgical technique are less likely to have an aggressive recurrence and conversely that aggressive recurrences are strongly influenced by unfavorable cancer biology. Our main finding supported this hypothesis. Specifically, the 9 models we tested, which all included primarily biology-related variables (i.e. PSA, stage, and grade), were more accurate in predicting aggressive recurrence vs. any BCR. In fact, a prior study also found factors mainly representing aggressive cancer biology (i.e. high-risk disease preoperatively and a prostatectomy Gleason score of 8 to 10, seminal vesicle invasion, or a time to PSA failure of less than 2 years postoperatively) to be independently predictive of a short PSADT after BCR [20]. Similarly, a recent study from the SEARCH database group found that only PSA, pathological findings, and prostatectomy Gleason grade were independently associated with aggressive recurrence [21]. Finally, yet another study found a rapid pre-operative PSA velocity, a strong predictor of cancer death, was associated with an aggressive recurrence [22]. Notably, in a follow-up to the latter study, a high PSA velocity while helpful for predicting any BCR was a much better discriminator of who would die from prostate cancer (i.e. have the most aggressive form of the disease) [23]. Thus, it does appear aggressive recurrences are most driven by poor biology.

Overall, the average predictive accuracy of the models was highest when <9 months was chosen as the definition for aggressive recurrence, which may imply this is the best definition for aggressive recurrence. Indeed, one prior study also suggested a 9-month cut-off may best discriminate between fatal and more indolent prostate cancer [13]. However, further studies focusing specifically on this question are needed to ultimately find the ideal PSADT cut-off value for defining aggressive recurrence.

To examine the robustness of our findings, we performed multiple sensitivity analyses, and found all models (with the exception of one model when aggressive recurrence was defined as PSADT <6 months) performed better when the outcome of interest was aggressive recurrence than BCR. Thus, whether the 220 men with recurrence but missing PSADT were excluded, considered as having an aggressive, or non-aggressive recurrence, and whether aggressive recurrence was defined using a PSADT cut-off of <6 months, <9 months, or <12 months, predictive accuracy was superior for estimating risk of aggressive recurrence relative to overall BCR. Ultimately, however, these results require validation in other datasets using more distant end-points such as metastasis and prostate cancer-specific death.

One strength of our study was a side-by-side comparison and external validation of seven previously published models. The best model for defining risk of BCR in this cohort was the model recently developed by our group using the independent Duke Prostate Center database [6]. Its predictive accuracy was nearly identical to the accuracy described in the initial report [6]. Thus, this study is the first to independently validate the usefulness of this model. Moreover, this model also worked well to predict aggressive recurrence.

Overall, the differences among all the models in accuracy for predicting any BCR were modest. Differences among the various models for estimating risk of aggressive recurrence were even less pronounced with a narrower range of predictive accuracies. The best model in this setting was the Kattan postoperative nomogram [5], however, the other models performed only marginally worse. It therefore appears predictive accuracy in itself might not be the most important factor in choosing a model for clinical use due to the overall similarity in performance across multiple models. Moreover, while validation is essential [10], the current study was able to equally well validate all external models. Therefore, it seems reasonable to base the choice of which model to use on personal preference and ease of use [24].

One limitation of the current study is the overall high positive margin rate and high 10-year BCR-rate. The degree to which this influenced the ratio of biology driven versus poor technique driven recurrences is unknown. Invariably, in a population of predominantly technique driven recurrences, we would hypothesize the improvement in model performance for predicting aggressive recurrence vs. any BCR would be greater. As such, it is noteworthy that the percent of men who recurred with an aggressive recurrence in the current study (29% using our definition) was similar to previous studies (38% with PSADT <9 months [12] and 38% with PSADT <12 months [25]). Moreover, other characteristics of our patients with BCR (table 1) are comparable to previously published series as well [12, 25]. This suggests the ratio of biology driven vs. technique driven recurrences may likewise be similar.

Another limitation is 220 patients had BCR but no PSADT available. Although we excluded these patients from our primary analysis and performed sensitivity analyses assuming these patients were either free of aggressive disease or recurred with aggressive disease, it is possible missing PSADT affected our results. Therefore, our results require validation in independent datasets.

We found predictive accuracy of existing models is higher when aggressive recurrence is used as the outcome of interest. However, this outcome is fortunately relatively rare. The 10-year risk of aggressive recurrence in the current cohort was only 9%. Therefore, risks of BCR after RP obtained from current nomograms or risk calculators cannot be directly transferred into risks of aggressive recurrence. For this end-point, new risk estimations based on existing patient populations with available PSADT after BCR would need to be calculated. Nevertheless our results are encouraging, as in clinical use it will likely be more important to identify patients at relatively high risk for aggressive recurrence who would be candidates for adjuvant therapy trials than to identify patients who are at risk for BCR alone. The current findings suggest this may be done employing currently existing nomograms and with even greater accuracy than predicting any BCR, provided baseline risks used in these nomograms are adjusted to the overall much lower risk of an aggressive recurrence.

Conclusion

Using seven different previously published models as well as a simple preoperative and postoperative model developed in this patient cohort, we found prediction of aggressive recurrence after RP is more accurate than prediction of BCR alone. This is likely due to the fact current models mainly measure cancer-biology, which correlates better with aggressive recurrence than with BCR alone.