Predicting Prostate Cancer Recurrence after Radical Prostatectomy

Abstract

BACKGROUND

Prostate cancer prognosis is variable, and management decisions involve balancing patients’ risks of recurrence and recurrence-free death. Moreover the roles of body mass index (BMI) and race in risk of recurrence are controversial [1,2]. To address these issues, we developed and cross-validated RAPS (Risks After Prostate Surgery), a personal prediction model for biochemical recurrence (BCR) within ten years of radical prostatectomy (RP) that includes BMI and race as possible predictors, and recurrence-free death as a competing risk.

METHODS

RAPS uses a patient’s risk factors at surgery to assign him a recurrence probability based on statistical learning methods applied to a cohort of 1276 patients undergoing RP at the University of Pennsylvania. We compared the performance of RAPS to that of an existing model with respect to calibration (by comparing observed and predicted outcomes), and discrimination (using the area under the receiver operating characteristic curve (AUC)).

RESULTS

RAPS’ cross-validated BCR predictions provided better calibration than those of an existing model that underestimated patients’ risks. Discrimination was similar for the two models, with BCR AUCs of 0.793, 95% confidence interval (0.766–0.820) for RAPS and 0.780 (0.745–0.815) for the existing model. RAPS’ most important BCR predictors were tumor grade, preoperative prostate-specific antigen (PSA) level and BMI; race was less important [3]. RAPS’ predictions can be obtained online at https://predict.shinyapps.io/raps.

CONCLUSION

RAPS’ cross-validated BCR predictions were better calibrated than those of an existing model, and BMI information contributed substantially to these predictions. RAPS predictions for recurrence-free death were limited by lack of co-morbidity data; however the model provides a simple framework for extension to include such data. Its use and extension should facilitate decision strategies for post-RP prostate cancer management.

INTRODUCTION

Among US men, prostate cancer is the most common non-cutaneous malignancy and the second leading cause of cancer death. In 2016, an estimated 180,890 new cases of prostate cancer will have been diagnosed and 26,120 deaths from the disease will have occurred [4]. However, many prostate cancers are indolent, and do not need aggressive surveillance after RP. Choosing optimal strategies for postoperative care requires using each patient’s personal characteristics to predict his likelihood of disease recurrence within a future time period and before he dies from other causes. Consideration of such death is needed because prostate cancer occurs late in life (with 67 years as median age at diagnosis) [5] and progresses slowly, and thus treatment decisions for patients with serious comorbidities should consider their likelihoods of dying before their cancers recur. Yet, although several risk models have been proposed for prostate cancer recurrence after RP (see Shariat for a review) [6] and one model considers the competing risks of death from prostate cancer and from other causes among patients with recurrent disease [7], we are unaware of any models examining the competing risks of recurrence and recurrence-free death after prostate surgery.

Here we describe RAPS, a predictive model that specifies a patient’s 10-year risk of prostate cancer recurrence after radical prostatectomy (RP), while accommodating the possibility that he will instead die from causes other than prostate cancer. We compare the performance of RAPS with that of a model proposed by Kattan [8], and we evaluate whether additional information on patients’ race and body mass index (BMI) improves model performance. Although race and BMI have been associated with recurrence risk [9–11], these characteristics have not been included in existing post-operative recurrence risk models.

The primary measures of model performance are: a) calibration (agreement between model predictions and patients’ actual outcomes); and b) discrimination (ability to distinguish patients whose cancers do and do not recur). These two measures reflect different aspects of model performance (2003); both are important in personalized care, with calibration important for integrating the multiple risks involved in treatment decisions, and discrimination important for assessing a model’s utility for effective and cost-efficient treatment guidelines. Such assessment across a broad range of settings is needed to ensure the model’s robustness across populations with different co-morbidities and different demographic characteristics [12].

MATERIALS AND METHODS

Study Population

As part of the Study of Clinical Outcomes, Risk and Ethnicity (SCORE) conducted within the University of Pennsylvania Health Care System, patients undergoing a RP for prostate cancer between 1995 and 2012 were invited to participate in a longitudinal study evaluating their risks of subsequent disease recurrence. A patient’s time to prostate cancer recurrence was defined as the date of first detectable blood levels of prostate-specific antigen (PSA), hereafter called biochemical recurrence (BCR). Patients’ covariates recorded at surgery included age, date of surgery, preoperative PSA level, race (white, black, other), and height and weight (combined here as BMI (weight in kilograms divided by the square of height in meters)). Patients’ postoperative tumor characteristics included binary variables for primary and secondary Gleason grade >= 4, surgical margin involvement (SMI), extra-capsular extension (ECE), seminal vesicle involvement (SVI), and lymph node invasion (LNI). Among 1464 potentially eligible SCORE patients, we excluded 160 with missing BCR status or year of BCR occurrence, 27 whose tumors lacked primary or secondary Gleason grade, and one with missing race, leaving 1276 patients for analysis. For 29 of these patients, we imputed the following missing covariates: pre-operative PSA level (N = 12), SMI status (N=8), SVI status (N=5) and ECE (N = 4).

Observed Outcome Data

For each SCORE patient we recorded an event time, defined as the time from his RP to the earliest of the following events: a) BCR; b) death without BCR; c) elapse of 10 years post-RP; and d) last observation alive and recurrence-free. Patients with event times of type (a) were classified as positive for BCR and negative for death; those of type (b) were positive for death & negative for BCR; those of type (c) were negative for both BCR and death; and those of type (d) were outcome-uncertain or censored. We used competing risk theory for survival data [13] to estimate the cumulative incidence functions for BCR and death in specified subgroups of patients, while allowing for censoring. For example, the cumulative BCR incidence function for white patients estimates the probability that these patients develop BCR within 10 years of RP and before dying of other causes.

Risk Models

We compared the performance of two prediction models that specify each patient’s probability of BCR within 10 years of RP, based on his personal and tumor characteristics at RP. Model A, due to Kattan [8], estimates patients’ 10-year BCR risks using Cox proportional hazards methods to regress time to BCR against preoperative PSA level, year of RP, and presence/absence of SMI, ECE, SVI, LNI, and high-grade cancer (i.e., both primary and secondary Gleason grade >= 4). Model B (the RAPS model) estimates each patient’s 10-year risks of both BCR and death using random survival forest (RSF) methods [14,15]. Model B includes all the covariates used by Model A; however it also includes patients’ ages at RP, race (black, white, other) and BMI as additional covariates, and both primary and secondary Gleason grade indicators are included (to allow a data-driven combination of the two to influence risk, rather than a pre-specified summary indicator). The two models differ in how they handle patients who die recurrence-free within 10 years of surgery: Model A treats them as BCR-uncertain (or censored) at their time of death, while Model B treats them as BCR-negative.

The statistical learning methods underlying Model B extend random forests [16,17] to accommodate censored survival data. A random forest is a set of classification and regression trees applied to multiple bootstrap samples of the patients. Each bootstrap sample is used to create a “tree” consisting of successive “nodes” at which sets of patients are split into two subsets (called “daughter nodes”), each of which contains patients more homogeneous with respect to outcome than the parent set. The split is based on patients’ values of one covariate, and the covariate and the split cutpoint are chosen to minimize the prediction error within the two daughter nodes. The splitting continues as long as both daughter nodes contain a pre-specified minimum number of patients. Each patient’s outcome probability is based on the survival experience of all patients in his terminal node. A convenient feature of these bootstrap-based trees is that each patient is excluded from approximately 1/3 of all the trees (such patients are deemed out-of-bag (OOB) for those trees) and therefore the cumulative hazard functions of those trees can be averaged to provide unbiased prediction probabilities for BCR occurrence before dying from other causes and for BCR-free death. In addition, the RSF methods also provide estimates of the importance of each covariate in correctly predicting their outcomes. The importance of a covariate is the change in prediction error when all binary splits involving the covariate are replaced by random splits of the relevant patients [14]. More detail about the methods and their extension to censored survival data can be found in Breiman [16] and Ishwaren [14,15].

Model Evaluation

We compared Models A and B with respect to their calibration to the observed BCR outcomes and their discrimination between patients who do and do not develop BCR within 10 years of RP. To assess a model’s calibration, we examined its standardized residuals (SRs), where the SR corresponding to a subgroup of patients is the difference between observed and model-predicted outcomes (BCR or death) in the 10-year period, divided by its standard deviation. For Model A, we computed SRs for the death outcome by assuming that the mortality rates predicted by this model are neglible [18]. We plotted SRs for subgroups of patients determined by quartile of model-assigned risk, age at RP, race and BMI level. Under the null hypothesis of good model fit, these SRs have standard normal distributions; thus SRs outside the range (−3,3) provide statistically significant (P= 0.003) evidence of poor model calibration [18]. We also constructed plots of observed and mean model-assigned outcome probabilities in various subgroups of patients, and compared observed and predicted BCR incidence in quartiles of assigned risk [19].

We assessed each model’s discrimination by estimating the area under its receiver operating characteristic (ROC) curve (also called the AUC) or concordance [20,21]. The AUC is the probability that the model-assigned risk for a randomly selected BCR-positive subject exceeds the model risk for a randomly selected BCR-negative subject. Since outcome-uncertain patients are neither BCR-positive nor BCR-negative, we assigned each such patient a pseudo-BCR-outcome, which we randomly generated with probability given by his risk of BCR within 10 years post RP, given the time he was last observed alive and BCR-free. This approach gives AUC estimates that are less biased and more efficient than those obtained by simply deleting outcome-uncertain subjects from the analysis (see Melcon [22] for details).

RESULTS

Patient Characteristics

Table I shows the distribution of SCORE patients according to selected patient and tumor characteristics, by status at last observation (BCR within 10 years, died BCR-free within 10 years, alive and BCR-free within 10 years, and alive and BCR-free at 10 years). Overall, 215 (16.8%) of the patients developed BCR within 10 years, 34 (2.7%) died recurrence-free within 10 years, 126 (9.9%) were alive and BCR-free at 10 years, and the remaining 901 patients (70.6%) were last observed alive and BCR-free at their last observation time before 10 years. A higher proportion (21.8%) of black patients developed BCR than did white patients (15.8%). As expected, preoperative PSA and tumor grade were strongly corrected with BCR occurrence but uncorrelated with death from other causes.

Table I.

Distribution of SCORE Patients according to Selected Personal and Tumor Characteristics, by Outcome Status at Last Observation

	BCRa within 10 yrs		Died BCR-free within 10 yrs		Alive & BCR-free at <10 yrs		Alive & BCR-free at 10 yrs		Total
	N	%	N	%	N	%	N	%	N	%
All subjects	215	16.8	34	2.7	901	70.6	126	9.9	1276	100
Age (yrs) at RP
<60	95	7.5	12	0.9	470	36.8	49	3.8	626	49.1
60–79	120	9.4	22	1.7	431	33.8	77	6.0	650	50.9
Race/ethnicity
EA	161	12.6	28	2.2	729	57.9	98	7.7	1016	79.6
AA	53	4.2	6	0.5	158	12.4	26	2.0	243	19.1
Other	1	0.1	0	0	14	1.1	2	0.2	17	1.3
BMI (kg/m2)
<25	31	2.4	8	0.6	168	13.2	34	2.7	241	18.9
25–29	103	8.1	18	1.4	460	36.1	67	5.3	648	50.8
30+	81	6.3	8	0.6	273	21.4	25	2.0	387	30.3
PSA levelb (ng/ml)
<5.6	46	3.6	12	0.9	521	40.8	48	3.8	1089	85.3
≥5.6	169	13.2	22	1.7	380	29.8	78	6.1	187	14.7
Pathologic Primary Gleason Grade
<4	124	9.7	29	2.3	823	64.5	113	8.9	1089	85.3
4+	91	7.1	5	0.4	78	6.1	13	1.0	187	14.7
Pathologic Secondary Gleason Grade
<4	108	8.5	23	1.8	572	44.9	81	6.3	784	61.4
4+	107	8.4	11	0.9	329	25.8	45	3.5	492	38.6

^aBCR = biochemical recurrence

^bpreoperative PSA level (median level = 5.6 ng/ml)

Comparison Of Assigned Risks

Fig. 1 shows a scatterplot of patients’ risks of recurrence within 10 years of RP and before dying of other causes, as assigned by Models A and B. Points close to the diagonal line correspond to patients for whom Models A and B assigned similar recurrence risks. The figure indicates that the risks assigned by Model A tend to be lower than those assigned by Model B. The correlation coefficient between the two sets of risks was 0.742, 95% CI (0.717, 0.766).

Figure 1.

Scatterplot of ten-year risks assigned by Models A and B.

Model Calibration

Table II and Fig. 2 show SRs based on differences between observed counts of outcomes (BCR and death) and the numbers predicted by Models A and B, with SRs for Model B based on cross-validation. The predicted numbers of outcomes were obtained using competing risk methods that account for competing risks and incomplete followup [18]. It is evident from both Table II and Fig. 2 that Model A’s predicted counts for both recurrence and death are systematically smaller than those observed. These discrepancies are also evident in Fig. 3, which shows that subjects’ actual BCR incidence rates are higher than the mean risks of patients in each quartile of model-assigned risk. In contrast, the figures and table show little evidence of poor fit for Model B.

Table II.

Standardized residuals (SRs) comparing observed (Obs) and predicted (Pred) counts of biochemical recurrence and recurrence-free death, with predictions based on Models A and B

		Recurrence						Death
		Model A			Model B			Model A			Model B
		Obs	Exp	SR	Obs	Exp	SR	Obs	Exp	SR	Obs	Exp	SR
All patients		215	81.04	14.88	215	215.84	−0.57	34	0	5.83	34	30.85	0.21
Risk quartile a	1	14	4.42	4.55	12	8.75	1.78	9	0	3.00	9	5.00	1.79
	2	20	6.62	5.20	22	23.13	−0.24	9	0	3.00	12	8.49	1.21
	3	43	14.52	7.48	49	46.84	0.32	24	0	4.90	3	10.06	−2.23
	4	138	55.47	11.08	132	137.13	−0.44	9	0	3.00	10	7.31	1.00
Age (yrs) b	<60	95	29.82	11.94	95	91.16	0.40	12	0	3.46	12	10.74	0.39
	60+	120	51.22	9.61	129	124.0	−0.42	22	0	4.69	22	20.11	0.42
Race c	White	161	56.19	13.98	161	161.72	−0.06	28	0	5.29	28	24.14	0.79
	Black	53	23.85	5.97	53	51.63	0.19	6	0	2.45	6	6.31	−0.12
	Other	1	1.00	0.003	1	2.50	−0.95	0	0	0	0	0.40	−0.63
BMI (kg/m2) d	<25	31	14.89	4.17	31	36.17	−0.86	8	0	2.83	8	6.85	0.44
	25–29	103	38.17	10.49	103	105.52	−0.25	18	0	4.24	18	17.14	0.21
	30+	81	27.98	10.02	81	74.16	0.79	8	0	2.83	8	6.85	0.44
PSA level e	<5.6	46	16.95	7.05	46	56.80	−1.43	29	0	5.39	29	11.63	5.09
	5.6+	169	64.08	13.11	169	159.04	0.79	5	0	2.24	5	19.22	−3.24

SRs are shown by:

^aquartile of model-assigned risk;

^bage at radical prostatectomy;

^crace;

^dbody mass index (BMI);

^epre-operative PSA level.

The SRs for a well-calibrated model fall outside the interval (−3, 3) with probability 0.003.

Figure 2.

Attribute diagrams (ADs) showing estimated 10-year cumulative BCR incidence versus mean assigned BCR risk for Models A and B.

Figure 3.

Receiver operating characteristic curves for Models A and B.

Model Discrimination

Fig. 4 shows the ROC curves that reflect the ability of Models A and B to distinguish the 215 BCR-positive patients (those with BCR within 10 years of RP) from the 160 BCR-negative patients (those alive and BCR-free at 10 years post-RP plus those who died BCR-free within 10 years of RP). The remaining 901 outcome-uncertain patients were assigned pseudo-outcomes, as described in the Methods section. Each point on a model’s ROC curve corresponds to a threshold T between 0 and 1, with its ordinate and abscissa given by the proportions of BCR-positive and BCR-negative patients, respectively, whose model-assigned risks exceed T. Clearly the two models provide similar discrimination, based on the similarity of their ROC curves and their AUCs of 0.780 (95% confidence interval 0.745–0.815) for Model A and 0.793, (0.766–0.820) for Model B.

Figure 4.

Relative importance of RAPS covariates in determining patients’ recurrence and mortality risks.

Covariate Importance

The RSF methods used to develop Model B also provide measures of the relative importance (Imp) of each of its covariates in determining patients’ risks of both recurrence and death from other causes within 10 years of RP. Fig. 4 shows that the most important recurrence predictor was the primary Gleason grade of a patient’s tumor (Imp = 0.062), followed by preoperative PSA level (Imp = 0.049) and BMI (Imp = 0.043). The race of a patient was not an important predictor, which agrees with the findings of some [23] but not all [24] other investigators. In contrast, none of the covariates played a major role in predicting the 34 deaths from other causes; importance exceeded 0.01 only for age at RP.

DISCUSSION

We have used longitudinal cohort data from patients undergoing RP for prostate cancer to develop and evaluate RAPS, a prediction model for prostate cancer recurrence that accounts for the competing risk of death from other causes. We used cross-validation to evaluate how well the model would perform if applied to an independent sample of patients from a population with the same demographic, co-morbid and tumor characteristics as that of the patients in the present SCORE sample. We found RAPS to be well calibrated to patients’ observed recurrences and deaths, while the recurrences predicted by the comparison model [8] were substantially lower than those observed. However the two models provided similar discrimination between patients with and without a BCR after surgery.

Some study limitations warrant consideration in interpreting these results. Because we lacked data on patients’ co-morbid conditions, the RAPS-assigned mortality risks are not based on patients’ baseline co-morbidities, and they were evaluated against few (N = 34) deaths from causes other than prostate cancer. Moreover, the mortality risks assigned to SCORE patients may be inappropriate for cohorts with higher prevalence of comorbid conditions. That is, while the current cross-validated performance measures accurately reflect those expected from independent samples of the same overall patient population as that containing the SCORE patients, they may not apply to different patient populations. This limitation emphasizes a need to extend RAPS by including existing co-morbid conditions, and to validate its predictions by application to patients from populations with distributions of racial/ethnic, demographic, co-morbid and tumor characteristics that differ from those of the SCORE patients. Because all model performance metrics depend on the population distribution of these outcome-determining characteristics, validation across different populations is important to establish a model’s validity across a wide range of clinical settings.

To our knowledge, RAPS is the first post-operative recurrence model to include a patient’s BMI in its predictions, and the data suggest that this risk factor is important for predicting his likelihood of recurrence, with more obese men at higher risk. This finding is consistent with conclusions from a systematic review and meta-analysis of the relation between BMI and prostate cancer recurrence [31], and from a previous analysis of the SCORE data [32]. BMI is largely determined by body weight, and obesity has been found to be associated with increased risk of BCR [9,33,34]. Moreover obesity has been associated with prostate cancer aggressiveness, even after adjusting for the pathologic features of the excised tumor [23]. These observations raise the possibility that obese RP patients might reduce their recurrence risks by weight reduction, an important question in need of investigation.

In contrast, patients’ self-identified race was not an important predictor of prostate cancer recurrence after RP in these data. This finding is consistent with some, but not all previous studies [9,24–30].

In summary, RAPS provided excellent cross-validated calibration and good cross-validated discrimination to the observed occurrences of both BCR and death from other causes in a cohort of largely Caucasian and African-American patients. Application of an existing risk model to this cohort showed similarly good discrimination but poor calibration, substantially under-predicting patients’ recurrence risks. Cross-validation of RAPS using the SCORE cohort also indicated that a patient’s BMI was an important predictor of his likelihood of recurrence, suggesting that for obese patients, weight loss may offer a way to reduce the morbidity and mortality associated with this disease. While RAPS’ predictions for recurrence-free death were limited by lack of co-morbidity data, the model nevertheless provides a simple framework for extension to include such data. Its use should facilitate decision strategies for post-RP prostate cancer management.

The application is deployed for immediate use online at https://predict.shinyapps.io/raps, and a version suitable for replication of our work at https://hub.docker.com/r/vanessa/prostate-raps/.

CONCLUSIONS

RAPS, which includes patients BMI and race, was better calibrated to the SCORE cohort than was the existing model, and BMI contributed substantially to its predictions. While these results were validated internally within the SCORE data by cross-classification, they need replication in cohorts with different covariate distributions. Moreover RAPS predictions for recurrence-free death need extension to include data on patients’ co-morbidities.