External validation of the SEARCH model for predicting aggressive recurrence after radical prostatectomy: results from the Duke Prostate Center Database

Abstract

Objective

To validate a model previously developed using the Shared Equal Access Regional Cancer Hospital (SEARCH) database to predict the risk of aggressive recurrence after surgery, defined as a prostate-specific antigen (PSA) doubling time (DT) of < 9 months, incorporating pathological stage, preoperative PSA level and pathological Gleason sum, that had an area under the curve (AUC) of 0.79 using a cohort of men from the Duke Prostate Center (DPC).

Patients and methods

Data were included from 1989 men from the DPC database who underwent RP for node-negative prostate cancer between 1987 and 2003. Of these men, 100 had disease recurrence, with a PSADT of < 9 months, while 1889 either did not have a recurrence but had ≥36 months of follow-up or had a recurrence with a PSADT of ≥ 9 months. We examined the ability of the SEARCH model to predict aggressive recurrence within the DPC cohort, and examined the correlation between the predicted risk of aggressive recurrence and the actual outcome within DPC.

Results

The SEARCH model predicted aggressive recurrence within DPC with an AUC of 0.82. There was a strong and significant correlation between the predicted risk of aggressive recurrence based on the SEARCH tables and the actual outcomes within DPC (r = 0.68, P < 0.001), although the model predictions tended to be slightly higher than the actual risk.

Conclusions

The SEARCH model to predict aggressive recurrence after RP predicted aggressive recurrence in an external dataset with a high degree of accuracy. These tables, now validated, can be used to help select men for adjuvant therapy and clinical trials.

Introduction

Men who have a recurrence of prostate cancer with a short PSA doubling time (PSADT) after radical prostatectomy (RP) have a higher risk of all-cause and prostate cancer-specific death than men who either do not have a recurrence or who have a recurrence with a longer PSADT [1–5]. However, the PSADT is not known until men have already had a recurrence and have been followed for months to years. There are several algorithms for predicting various patient outcomes such as the risk of cancer extending outside of the prostate [6] and risk of PSA recurrence [7,8]. These algorithms generally rely on clinicopathological variables such as surgical margin status, biopsy or pathological Gleason sum, and preoperative PSA level. However, there are few studies examining the risk of clinically meaningful, or `aggressive' recurrences using data available before men have disease recurrence. In a previous study that included men from the Shared Equal Access Regional Cancer Hospital (SEARCH) cohort [9], we identified three factors that were available at the time of surgery that could be used to identify men who were at risk of an aggressive recurrence (defined as recurrence with a PSADT of < 9 months), i.e. preoperative PSA level, pathological Gleason sum, and pathological data [10]. However, the SEARCH database only includes men treated in equal-access care settings and thus it is unclear whether these results are applicable to men treated in different settings. Therefore, we sought to validate this model using a cohort of men treated at the Duke Prostate Center (DPC), a tertiary care-referral centre, by examining the ability of the SEARCH model to predict aggressive recurrence within the DPC cohort, as well as the correlation between the predicted risk of aggressive recurrence and the actual outcome within the DPC cohort.

Patients and methods

After obtaining Institutional Review Board approval from Duke University to abstract data, we entered data from patients undergoing RP at Duke University into the DPC database. This database includes information on patient age at surgery, race, height, weight, clinical stage, grade of cancer on diagnostic biopsies, preoperative PSA level, surgical specimen pathology (specimen weight, tumour grade, stage, and surgical margin status), follow-up PSA data, as well as sociodemographic and quality-of-life data. Patients treated with neoadjuvant androgen deprivation or radiotherapy were excluded. Biochemical recurrence was defined as a PSA level of ≥0.2 ng/mL obtained ≥30 days after surgery. Men who received adjuvant treatment with an undetectable PSA level were censored as not having recurrence at the time of treatment. The PSADT at the time of recurrence was calculated assuming first-order kinetics, by dividing the natural log 2 (0.693) by the slope of the linear regression line of the natural log of PSA level over time. A PSADT was computed for all patients meeting the definition of recurrence and who had at least two PSA values, separated by ≥3 months, and within 2 years after recurrence. All PSA values within the first 2 years after recurrence were used to calculate PSADT. For patients starting salvage hormone or radiotherapy within this time, only PSA values before salvage therapy were used. Patients with a PSADT of ≤ 0 (i.e. decline or no change in PSA) or a very long PSADT (> 100 months) were assigned a value of 100 months for ease of calculations (147 men).

We a priori defined aggressive disease as recurrence with a PSADT of < 9 months, based on the high prostate cancer-specific mortality rate in a previous study [4]. Non-aggressive disease was defined as recurrence with a PSADT of ≥9 months or no evidence of recurrence, as these men are at low risk of death from prostate cancer. Thus, of the 4689 men within the DPC, we excluded those who were lacking follow-up data (59) or who had a recurrence with an unknown PSADT (741), as they were unable to be classified.

Men whose disease recurs after 3 years are unlikely to have a life-threatening recurrence [11]. However, men with no recurrence but with a limited follow-up (< 36 months) remain at risk of recurrence with a short PSADT and cannot be reliably classified, and were thus excluded (1005 men). Because men who had surgery after 2003 (i.e. < 36 maximum potential months of follow-up) could only be included if they recurred, and as early recurrences are more likely to be aggressive, their inclusion would result in a potential selection bias. Therefore, all men treated after 2003 who otherwise met the inclusion criteria (162) were excluded.

Because men with node-positive disease are likely to have aggressive disease and are at high risk of death from prostate cancer, they were excluded (46). We also excluded 472, 194 and 21 men who were missing preoperative PSA values, margin status data, and extraprostatic extension (EPE) data, respectively. This resulted in a study population of 1989 men who had RP between 1987 and 2003.

We explored differences in the distribution of demographic and clinicopathological characteristics between aggressive and non-aggressive disease groups using the Mann–Whitney test for continuous variables (age, PSA level, prostate weight, and year of surgery) or the chi-squared test for categorical variables, i.e. race, body mass index (BMI), postoperative Gleason sum, EPE, surgical margin status, and seminal vesicle invasion (SVI). BMI was stratified as < 25, 25–29.9, 30–34.9, and ≥ 35 kg/m². Pathological findings were examined as a three-tiered categorical variable based upon recurrence risk noted in a previous SEARCH study: (i) organ-confined margin-negative; (ii) positive margins and/or EPE with no SVI; or (iii) SVI [10].

We examined the ability of our previously published model, that was developed to predict aggressive recurrence using the SEARCH database [10] to predict aggressive recurrence in the DPC database [12]. This model included preoperative PSA level (< 10, 10–20 and > 20 ng/mL), pathological Gleason sum (2–6, 7 and 8–10), and the three-tiered grouping of pathological characteristics described above. For each patient the risk of aggressive recurrence was estimated using the SEARCH database model. This risk was then examined for its ability to predict aggressive recurrences within the DPC cohort using the area under the receiver operative characteristics (ROC) curve (AUC). We also examined the correlation between the predicted risk of aggressive recurrence based on the SEARCH model for each risk category and the actual outcome within DPC using Spearman's correlation. In all tests, P < 0.05 was considered to indicate statistical significance.

Results

Men in the aggressive group were older, had surgery in earlier years, had higher PSA level, smaller prostates, higher pathological grade disease, more adverse pathological findings, and a greater percentage of the prostate involved with tumour than men in the non-aggressive group (Table 1).

Table 1.

The demographic, clinical and pathological characteristics of 1989 men undergoing RP, within the DPC database, stratified by disease aggressiveness

Mean (sd), median (IQR) or n (%) variable	Aggressive	Non-aggressive	P*
No. of patients	100	1889
Age at surgery, years	66.0 (7.1)	63.6 (7.3)	0.003†
Median year of surgery	1993	1997	< 0.001†
Race			0.006
Caucasian	74 (74)	1617 (86)
African American	24 (24)	249 (13)
Other	2 (2)	22 (1)
BMI, kg/m2
< 25	9 (19)	292 (24)	0.6
25–29.9	23 (49)	583 (49)
30–34.9	10 (21)	249 (21)
≥ 35	5 (11)	75 (6)
PSA, ng/mL	21.5 (22.4)	10.8 (29.8)	< 0.001†
	13.7 (8.4–25.0)	6.8 (4.7–10.7)
Pathological Gleason sum			< 0.001
2–6	13 (13)	988 (52)
7	45 (45)	736 (39)
8–10	42 (42)	165 (9)
Prostate weight, g	36.4 (15.0)	41.6 (18.9)	0.05†
	35.0 (26.5–44.0)	37.0 (29.0–49.0)
% of prostate involved with cancer	31.6 (19.2)	17.0 (15.0)	< 0.001
	25.0 (15.0–43.0)	10.0 (5.0–22.0)	< 0.001
EPE	68 (68)	683 (36)	< 0.001
Positive surgical margins	59 (59)	627 (33)	< 0.001
SVI	42 (42)	180 (10)	< 0.001
Pathological stage group			< 0.001
Organ-confined, margin negative	10 (10)	926 (49)
EPE and/or positive margins	48 (48)	783 (41)

^*Chi-square test unless otherwise specified

^†Mann-Whitney test

IQR, interquartile range.

In all, 100 men had disease recurrence with a PSADT of < 9 months. Of the 1889 men in the non-aggressive group, 454 had a recurrence with a PSADT of ≥9 months and 1535 did not have a recurrence. Men without recurrences had a median follow-up of 96.1 months, while those who had a recurrence with a PSADT of ≥ 9 months had a median time to recurrence of 15.8 months. Among the men with aggressive recurrences, the median time to recurrence was 3.4 months. The median PSADT in the aggressive recurrence group was 5.8 months vs 31.0 months in the non-aggressive group.

In the SEARCH cohort, we previously found that higher PSA levels, higher pathological grade disease, and adverse pathology were all significantly associated with aggressive recurrence. To validate that these same variables predicted recurrence in DPC, we ran a multivariate logistic regression model and found that all three variables significantly predicted aggressive recurrence (Table 2). Specifically, pathological Gleason sum of 7 (P = 0.002) and 8–10 (P < 0.001) relative to Gleason 2–6 tumours were significantly correlated with a short PSADT. EPE and/or positive margins (P = 0.001) and SVI (P < 0.001) were also associated with increased likelihood of aggressive recurrence relative to organ-confined, margin-negative disease. Overall, men with a higher PSA level were at higher risk of having an aggressive recurrence (P < 0.001). When PSA level was categorized using the thresholds of < 10, 10–19.9 and ≥ 20 ng/mL used in the SEARCH model, both a PSA level of 10–19.9 and > 20 ng/mL were significantly associated with an increased risk of aggressive disease (P = 0.004 and < 0.001, respectively). Stratified PSA levels remained significantly predictive of aggressive recurrence after adjustment for grade and stage (Table 2).

Table 2.

Multivariate analysis of factors predicting short PSADT (< 9 months) in men undergoing RP

Variable	Odds ratio (95% CI)	SEM	P
Model 1, using PSA as a continuous variable
logPSA	1.75 (1.37–2.24)	0.22	< 0.001
Pathological Gleason sum
2–6	Reference
7	2.65 (1.38–5.06)	0.88	0.003
8–10	7.91 (3.98–15.7)	2.77	< 0.001
Pathological stage group
Organ-confined, margin negative	Reference
EPE and/or positive margins	3.50 (1.72–7.10)	1.26	0.001
SVI	6.42 (2.97–13.9)	2.52	< 0.001
Model 2, using PSA as a categorical variable
PSA, ng/mL
< 10	Reference
10–19.9	2.20 (1.29–3.74)	0.60	0.004
≥ 20	4.04 (2.33–6.99)	1.13	< 0.001
Pathological Gleason sum
2–6	Reference
7	2.75 (1.44–5.25)	0.91	0.002
8–10	7.60 (3.81–15.1)	2.67	< 0.001
Pathologic stage group
Organ-confined, margin negative:	Reference
EPE and/or positive margins:	3.50 (1.72–7.10)	1.26	0.001
SVI	6.15 (2.85–13.3)	2.41	< 0.001

We examined the ability of the multivariate model developed to predict aggressive recurrence in the SEARCH cohort to discriminate between men at high risk of aggressive recurrence vs men at low risk of aggressive recurrence in DPC using the AUC (Fig. 1). In the DPC database, the AUC for this model was 0.82.

Fig. 1.

The predictive accuracy of the model in the DPC cohort.

When we examined the correlation between the predicted risk of aggressive recurrence based on the SEARCH model for each risk category and the actual outcome within DPC using Spearman's correlation, there was a strong and significant correlation between the actual and predicted outcomes (r = 0.68, P < 0.001; Fig. 2). The risk of aggressive recurrence predicted by the SEARCH model was slightly higher than the actual risk found among men in the DPC database, as shown in Fig. 2. Among men with lower risk, patient risk was overestimated by only 5–10%, but among the subset of men with higher risk, which contained relatively few men, patient risk was overestimated to a larger degree.

Fig. 2.

The risk of aggressive recurrence predicted by SEARCH vs the actual risk in the DPC cohort

Discussion

In a previous study from the SEARCH database, we identified three factors available at the time of surgery that could identify men at increased risk of an aggressive recurrence (defined as recurrence with a PSADT of < 9 months); preoperative PSA level, pathological Gleason sum, and pathological features [10]. We sought to validate this model using a cohort from the DPC, which includes men treated at Duke University Medical Center, a tertiary-care hospital, unlike the equal access Veterans' Affairs medical centres, at which patients in SEARCH were treated. We examined the ability of the SEARCH model to predict aggressive recurrence within DPC, as well as the correlation between the predicted risk of aggressive recurrence and the actual outcome within DPC.

This model uses the classic prostate cancer predictive triad (i.e. stage, grade and PSA level) that has been used in many studies to create models predicting various prostate cancer outcomes, such as the Kattan postoperative nomogram for predicting overall PSA recurrence [7], the risk-stratification criteria of D'Amico et al. [8], and the Partin tables for predicting pathological stage [6]. In the SEARCH cohort, this model had an AUC of 0.79 for predicting an aggressive recurrence, which is similar to or better than other models that are used to predict overall recurrence [7]. This is important, in that this model predicted, with equal or better accuracy, an endpoint (recurrence with a short PSADT), which is more strongly linked to cause-specific and overall mortality than simple PSA recurrence alone [1–4]. When this model was applied to men in the DPC database, it had an AUC of 0.82, which was actually higher than in the original development cohort. This underscores the strength and clinical utility of this model, given its high predictive accuracy in two different cohorts with different patient populations and racial compositions. Notably, the high predictive accuracy might result from the fact that in general, aggressive recurrence can be predicted more reliably than overall biochemical recurrence. In a study by Schroeck et al. [13] the authors found that Harrell's concordance index was 0.702 for a set of previously published nomograms predicting overall recurrence, whereas it was 0.756 for predicting aggressive recurrence.

When we examined the correlation between the predicted risk of recurrence (based on the SEARCH model) and the actual outcome for each risk category, there was a strong and significant correlation between the actual and predicted outcomes (r = 0.68, P < 0.001). However, the risk of aggressive recurrence predicted by the SEARCH model was slightly higher than the actual risk found among men in the DPC database (Fig. 2). Among men with lower risk features, the risk of aggressive recurrence was only slightly overestimated; however, it was overestimated to a larger degree among men with higher risk features. This is probably a reflection of the smaller subset of men with aggressive disease features that were included in the study.

Because this model has now been validated in an outside dataset and has been shown to have a similar, and even improved, predictive accuracy than in the development cohort, it can now be used in clinical practice. To facilitate this, we have posted the tables on the Duke Urology website at http://urology.surgery.duke.edu/nomograms.

Because this model can estimate a patient's risk of aggressive recurrence immediately after surgery, it can be used to risk-stratify men very early in the course of their disease. Therefore, men at high risk of developing an aggressive recurrence could be given earlier adjuvant therapy than they might otherwise receive, or be referred to clinical trials. It could also be used as a clinical trial entry criterion for studies examining new treatments for aggressive or high-risk prostate cancer.

Our study has several limitations. First, the predicted values for each risk category based upon the SEARCH data were higher than the actual values for each risk category in the DPC cohort; therefore the model might slightly overestimate the risk of disease in the DPC cohort, although the differences were often < 10% overestimates. These differences probably reflect other factors that influence prostate cancer outcomes that are not captured by this model. Additionally, patient risk is likely to vary slightly in different cohorts of men to which the model is applied, and the `true' risk might in fact lie somewhere between the values predicted by SEARCH and those actually seen in DPC. This model should be evaluated in a third cohort of men, preferably one that has many men with high-risk features, to further assess the correlation between predicted and actual patient risk. Inclusion of men with high-risk features would also allow further assessment of the accuracy of this model in this subset of men. Also, this study uses a surrogate endpoint (i.e. PSADT) rather than a harder endpoint, such as metastasis or death. Finally, the use of the 9-month PSADT threshold for defining aggressive disease is largely based on study data for one institution [3,4], although other studies have shown that recurrence with a short PSADT (using thresholds of 3–12 months to define aggressive disease) is associated with worse outcomes [1,2,5]. Despite these limitations, now that this model has been validated, it can be used in the clinical setting.

In conclusion, in the present study, we successfully validated the model for predicting aggressive recurrences after RP that we had initially developed in the SEARCH cohort, using the DPC cohort. Now that this model has been validated, it can be used to risk-stratify men immediately after surgery and allow high-risk men to be referred for adjuvant therapy. It could also be used as an entry criterion to help select men for clinical trials that are studying treatments for high-risk prostate cancer.

Glossary

Abbreviations

DT

doubling time

ROC

receiver operator characteristic

DPC

Duke Prostate Center

RP

radical prostatectomy

EPE

extraprostatic extension

SVI

seminal vesicle invasion

SEARCH

Shared Equal Access Regional Cancer Hospital

BMI

body mass index

AUC

area under the curve