A Comparison of Models to Predict Clinical Failure Following Radical Prostatectomy

Abstract

Purpose

Models are available to accurately predict biochemical recurrence (BCR) following radical prostatectomy (RP). Since not all patients experiencing BCR will progress to metastatic disease, it is appealing to determine post-operatively which patients are likely to manifest systemic disease.

Methods

The study cohort consisted of 881 patients undergoing RP between 1985 and 2003. Clinical failure (CF) was defined as metastases, a rising PSA in a castrate state, or death from prostate cancer. The cohort was randomized into training and validation sets. The accuracy of four models to predict clinical outcome within five years of RP were compared: “post-operative BCR nomogram” and “Cox regression CF model” based on standard clinical and pathologic parameters, and two CF “systems pathology” models which integrate clinical and pathologic parameters with quantitative histomorphometric and immunofluorescent biomarker features (“systems pathology models #1 and #2”).

Results

When applied to the validation set, the concordance index for the post-operative BCR nomogram was 0.85, Cox regression CF model 0.84, systems pathology model #1 0.81, and systems pathology model #2 0.85.

Conclusions

Models predicting either biochemical recurrence or clinical failure following radical prostatectomy exhibit similarly high levels of accuracy since standard clinical and pathologic variables appear to be the primary determinants of both outcomes. Patients and clinicians interested in predicting clinical failure can recalibrate standard biochemical recurrence models to estimate the likelihood of systemic disease. It is possible that introducing current or novel biomarkers found to be uniquely associated with disease progression may further enhance the accuracy of the systems pathology-based platform.

Introduction

There are many prognostic models that predict biochemical recurrence (BCR) after radical prostatectomy (RP)^{1, 2}. BCR, however, is suboptimal as it lacks a standardized assay or definition, suffers from false-positives stemming from non-prostatic or non-cancerous pathologies, and, most importantly, poorly predicts subsequent metastases or death from prostate cancer³. Much effort has focused on identifying patients with a high likelihood of manifesting these endpoints^{3, 4} so that patient expectations are realistic, appropriate surveillance initiated, adjuvant treatment considered, and entry into clinical trials discussed.

We analyzed a series of patients undergoing RP and compared the accuracy of several different models to predict clinical endpoints of unequivocal importance to patients and physicians, namely metastases, biochemical progression of cancer following androgen deprivation therapy (ADT), or death from disease.

Methods

Following IRB approval, information was compiled on 971 patients treated with radical prostatectomy by multiple surgeons at Memorial Sloan-Kettering Cancer Center (MSKCC) between 1985 and 2003 for localized and locally advanced prostate cancer and for whom tissue samples were available. We excluded patients who received treatment either before prostatectomy or immediately after but before biochemical recurrence, leaving 881 patients in the full cohort. Secondary treatment following a BCR was acceptable and administered at the discretion of the treating physician.

BCR was defined as a PSA > 0.2 ng/ml confirmed by a subsequent rising value and clinical failure (CF) as unequivocal radiographic or pathologic evidence of metastases (skeletal and/or soft tissue disease in lymph nodes or solid organs), a rising PSA while on androgen deprivation therapy, or death attributed to prostate cancer. Patients were censored at CF or last known follow-up. Time to CF was measured from RP to the first of these events.

To obtain training and validation sets of equivalent size and case mix, patients were randomized using the following strata: preoperative PSA (by quartile), pathologic Gleason score (≤ 6, 7, ≥ 8), surgical margin (SM), extracapsular extension (ECE), seminal vesicle invasion (SVI), lymph node involvement (LN), BCR, and CF. Patients without exact matches on all aforementioned characteristics (n=58) were randomized based on PSA, pathologic Gleason, BCR, and CF status. The remaining patients (n=13) were randomized based on BCR and CF only.

Our objective was to compare four models constructed to predict outcome within five years following RP. The first was a model (“Kattan post-operative nomogram”) that predicts the probability of BCR based on PSA, Gleason grade, and pathological stage⁵. The second model (“Standard Cox model”) was to create a Cox proportional hazards model on the training set using CF as the outcome event. Predictors included PSA (using restricted cubic splines with knots at the tertiles), pathological Gleason grade (≤ 6, 7, ≥ 8) and stage (coded as a binary categorical variable: 1 if the patient had any of SM, ECE, SVI, LN, and 0 otherwise).

The final two models utilized a “systems pathology” approach developed by Aureon Laboratories (Yonkers, NY) which integrates standard clinical and pathologic parameters with automated histomorphometric and immunofluorescent biomarker variables⁶. In brief, hematoxylin and eosin (H&E) stained sections from prostatectomy blocks were assessed by two pathologists for tumor content and quality prior to construction of tissue microarray (TMA) blocks. Twelve blocks containing triplicate 0.6 mm cores from 881 prostatectomy specimens were constructed at MSKCC. Digitized H&E images were acquired from the TMA slides and analyzed using a proprietary histology labeling tool software designed by Aureon. Image objects were classified into histopathological classes according to their spectral (e.g. color, channel values), generic shape (e.g. area, length), and spatial relationship properties. A multiplex immunofluorescence assay consisting of 5 antibodies [androgen receptor (AR), α-methyl CoA racemase (AMACR), cytokeratin 18 (CK18), high molecular weight keratin (HMWK), and p63] was performed on all 12 TMA blocks. Images were acquired and evaluated using image analysis software with extraction of selected features reflecting intensity, area and distribution of selected antigens.

The systems pathology predictive models were constructed using a version of the support vector machine regression (SVR) machine learning algorithm. To accomplish this, a modified loss/penalty function was defined within the support vector regression algorithm, which allows censored (left and right censored) and non-censored data to be processed.

The systems pathology models were generated utilizing preoperative data, prostatectomy pathologic findings, and quantitative histologic and biomarker features. The model provided a risk score from 0 to 100, with a higher value connoting an increased risk of CF. If the score is greater than the threshold, a patient is predicted as being an “early” CF (prior to 5 years after RP); if the score is less than or equal to the threshold, he is predicted as being at either low risk of CF or a possible “late” CF. The thresholds, as determined by Aureon were 40.1 and 36.9 for Model 1 and Model 2, respectively.

Pre-treatment clinical information, pathologic findings at prostatectomy, and post-treatment outcome (e.g. BCR, CF) for all patients in the training set were provided to Aureon. Following construction of the training models, preoperative and pathologic data from patients in the validation set were forwarded to Aureon and two predictions, one for each model, made for each patient. The accuracies of these models, as well as the post-operative nomogram and the standard Cox model, were then calculated by MSKCC statisticians. At no point in the study did Aureon have access to outcome data for the validation set.

The systems pathology platform did not provide predictions for 217 (25%) patients due to insufficient tumor content as assessed from the H&E analysis (<50% tumor on a core). The systems pathology models were therefore developed on 345 evaluable patients in the training set and the standard Cox model developed on all 440 training patients. All models were then applied to the 319 patients in the validation set that were evaluable by systems pathology.

Predictive accuracy was defined in terms of the concordance index, ranging from 0.5 (chance) to 1.0 (perfect accuracy) and reflecting the probability that in two randomly selected patients, one who had an event and the other who was event-free for at least a similar period of time, the patient with the event had a higher risk prediction. As a second measure of accuracy, we estimated the sensitivity and specificity of the models to predict CF within 5 years. To calculate sensitivity and specificity for survival time data, we first define x = 1 if the patient is classified as being at ‘high risk’ by the systems pathology score and x = 0 otherwise; s(t) is the Kaplan-Meier survival probability at time t, predefined as five years. The sensitivity is calculated as [1 − (s(t) | x = 1)] · P(x = 1) ÷ [1 − s(t)] and the specificity as (s(t) | x = 0) · P(x = 0) ÷ s(t)⁷. Sensitivity and specificity for the systems pathology models were developed in the training set and applied to the validation set. Specificities for the postoperative BCR and Standard Cox models predicting CF were chosen to match the specificities of the systems pathology models.

Binary predictions for the post-operative nomogram and standard Cox model were obtained by dichotomizing the model predictions so that specificity was close to that of the systems pathology models (∼80%). Statistical analyses were performed using Stata 8.2 (Stata Corp, College Station, TX).

Results

The clinical and pathologic characteristics of the study sample are summarized in Table 1. Training and validation set data include only those patients evaluable by systems pathology.

Table 1.

Characteristics of the study cohort, training set, and validation set. Training and validation set data included only patients successfully analyzed by the systems pathology platform.

	Entire cohort	Training	Validation
Number of patients	881	345	319
Follow-up from prostatectomy (years)
Median	6.1	6.4	6.6
IQR	4.6, 8.0	4.8, 8.5	5.3, 8.7
Age (years)
Mean (SD)	60.8 (6.6)	61.0 (6.6)	60.5 (6.7)
Pre-operative PSA (ng/dl)
< 4.0	72 (8%)	22 (6%)	31 (10%)
4.0 – 9.9	495 (56%)	193 (56%)	163 (51%)
10.0 – 20.0	222 (25%)	90 (26%)	87 (27%)
> 20.0	92 (11%)	40 (12%)	38 (12%)
Clinical Stage
T1	439 (50%)	176 (51%)	152 (48%)
T2	427 (48%)	160 (46%)	162 (51%)
T3	15 (2%)	9 (3%)	5 (1%)
Biopsy Gleason Score
≤ 6	564 (64%)	224 (65%)	199 (62%)
7	256 (29%)	93 (27%)	97 (31%)
≥ 8	61 (7%)	28 (8%)	23 (7%)
Pathologic Gleason Score
≤ 6	316 (36%)	128 (37%)	118 (37%)
7	478 (54%)	182 (53%)	167 (52%)
≥ 8	87 (10%)	35 (10%)	34 (11%)
Surgical Margins
Positive	305 (35%)	117 (34%)	120 (38%)
Negative	576 (65%)	228 (66%)	199 (62%)
Extracapsular Extension
Yes	262 (30%)	103 (30%)	97 (30%)
No	619 (70%)	242 (70%)	222 (70%)
Seminal Vesicle Involvement
Yes	79 (9%)	29 (8%)	28 (9%)
No	802 (91%)	316 (92%)	291 (91%)
Lymph Node Involvement
Yes	27 (3%)	13 (4%)	11 (3%)
No	854 (97%)	332 (96%)	308 (97%)
Biochemical Recurrence (PSA > 0.2 ng/ml)
Yes	214 (24%)	87 (25%)	79 (25%)
No	667 (76%)	258 (75%)	240 (75%)
Clinical Failure*
Yes	81 (9%)	30 (9%)	27 (8%)
No	800 (91%)	315 (91%)	292 (92%)

^*Metastases, rising PSA in castrate state, or death from prostate cancer

The systems pathology modeling began with a set of 40 variables (10 clinicopathologic, 12 morphometric, and 18 immunofluorescence features). Table 2 lists the 11 features selected by Model #1 (5 clinicopathologic, 5 morphometric, and 1 molecular) and 7 features selected by Model #2 (3 clinicopathologic, 3 morphometric, and 1 molecular) in order of weighted importance. The standard Cox model predicting CF included PSA, Gleason grade, and pathologic stage (Table 3).

Table 2.

Features selected for systems pathology clinical failure models, in order of weighted importance. AR: androgen receptor, AMACR: α-methyl CoA racemase

Systems Pathology Model #1
1) Biopsy Gleason sum 2) Texture variation within cytoplasm 3) Lymph node involvement 4) Relative area of epithelial nuclei 5) Mean brightness of cytoplasm 6) Pre-op PSA 7) Prostatectomy Gleason sum 8) Variation of color between cytoplasm 9) Mean brightness of AR in AR+/AMACR+ epithelial nuclei 10) Surgical margins 11) Variation of color between cytoplasm
Systems Pathology Model #2
1) Mean brightness of cytoplasm 2) Biopsy Gleason sum 3) Lymph node involvement 4) Texture variation in stroma 5) Color variation between epithelial nuclei 6) Prostatectomy Gleason sum 7) Mean brightness of AR in AR+/AMACR+ epithelial nuclei

Table 3.

Standard Cox model predicting CF derived from training set

	Hazard ratio	95% C.I.		p value
PSA (ng / ml)*	1.02	1.00	1.05	0.10
Gleason score
Gleason ≤ 6	Reference
Gleason 7	0.91	0.40	2.08	0.8
Gleason ≥ 8	3.47	1.43	8.41	0.006
Pathologic stage
Organ-confined	Reference
Non-organ confined**	2.74	1.19	6.32	0.02

^*non-linear terms with splines at tertiles

^**extracapsular extension, seminal vesicle invasion, or lymph node involvement

The concordance indices, sensitivity, and specificity of the four models are summarized in Table 4. In a sensitivity analysis, the standard Cox model was developed using only the 345 patients in the training set that were evaluable for the systems pathology models. This did not have an important impact on model accuracy (c-index reduced from 0.838 to 0.826).

Table 4.

Accuracy of each predictive model on the training and validation sets ; BCR (biochemical recurrence), CF (clinical failure), CI (concordance index)

Model	Outcome	CI: Validation set	Sensitivity	Specificity
Post-operative nomogram	BCR	0.853	93.4%	82.4%
Standard Cox model	CF	0.838	81.8%	81.0%
Systems Pathology Model 1	CF	0.807	76.4%	79.9%
Systems Pathology Model 2	CF	0.849	85.7%	79.0%

Discussion

A rising PSA after RP does not invariably culminate in symptomatic local or distant recurrence. For example, 28% of patients with BCR, defined as a PSA > 0.4 ng/ml, may never experience a subsequent increase in PSA or clinical symptoms⁸ and among patients experiencing BCR defined as a PSA > 0.2 ng/ml, half remained without evidence of metastatic disease 8 years later⁹. Yet, despite the broad spectrum of clinical behavior for patients with BCR, nearly all will eventually undergo secondary forms of therapy as Bianco et al estimated the 10-year probability of being alive and free from secondary treatment as only 13%¹⁰.

Given the heterogeneous nature of BCR, its loose association in heralding systemic disease, and uncertain management, the utility of identifying an individual at high risk of BCR is unknown. For these reasons, it is a more appealing objective to accurately determine which patients are destined to manifest metastases, the lethal variant of prostate cancer, so that secondary therapy can be administered judiciously and appropriately.

We sought to construct CF models to predict outcome following RP and found no evidence that incorporation of the extra variables included in the systems pathology models improved predictive accuracy. We feel there are multiple reasons for this finding. A PSA recurrence serves as an intermediate endpoint for subsequent metastases, occurring prior to but not guaranteeing its development. Nevertheless, D'Amico et al have shown that for patients treated by external beam radiation, those at high-risk for BCR are similarly at high-risk for metastases and cancer-specific death, regardless of competing risk¹¹. Models predicting BCR and CF are, logically, destined to have similar predictive variables and accuracy. Patients and clinicians wishing to predict CF can therefore use standard models for BCR and recalibrate the resulting probabilities. For example, if we hypothetically assume a 10-year probability of BCR of 30% and a 10-year probability of CF of 10%, then by using a standard nomogram for BCR⁵ and multiplying the probability by 0.33 (10/30) the estimated likelihood of CF can be obtained.

Another finding was the addition of 27 histologic morphometric features and 5 candidate biomarkers to standard clinicopathologic features did not enhance the predictive accuracy of the CF model. All three CF models provided a similarly high level of accuracy and their indistinguishable outcome, while disappointing, is not surprising. First, among the myriad of tumor markers studied in prostate cancer, few have been shown to be clinically useful. It is possible the addition of current or novel biomarkers found to be uniquely associated with disease progression may further enhance the systems pathology approach. Second, the models based on standard clinical and pathologic parameters (“Kattan post-operative nomogram” and “Standard Cox Model”) presumably benefit from the expertise of an experienced pathologist at an oncology referral center. How the systems pathology approach would perform compared to a standard model based on data from less experienced pathologists is not known. Third, the finding of model equivalence lends further support to the force of traditional and universally predictive disease characteristics, such as Gleason grade and lymph node involvement.

To enhance the likelihood of improving upon currently available predictive models, how should future modeling studies be designed? Ideally, the model would be created from and validated on a sufficiently large cohort with a diverse spectrum of demographic and disease characteristics followed for an extensive period of time, treated uniformly, and analyzed with contemporary modeling procedures using both established and novel markers¹². For a novel feature to be included it must improve the accuracy of the model without it. Following RP, validated instruments exist that predict BCR with a relatively high degree of accuracy (CI: 0.81)⁵. Therefore, to improve the accuracy of such a model would require the addition of robust markers not significantly confounded by prior parameters. Paradoxically, the success of current models has impeded their further improvement; however previous studies demonstrating that gene expression data enhances a post-prostatectomy model predicting BCR supports the idea that additional genetic or molecular tumor characteristics are important¹³.

Our objective was to reliably predict CF based on information available immediately following prostatectomy. This is an arduous task, largely due to the typically prolonged natural history of prostate cancer. Most men not cured by local definitive treatment live many years⁹, frequently die of non-prostate cancer related causes¹⁴, and their rate of cancer-related death even in large, mature series is relatively small^{15, 16}. Pre-treatment PSA, PSA velocity, and certain pathologic characteristics are associated with cancer-specific mortality^15-17 and additional predictive factors available at the time of BCR (PSADT, time to BCR, and response to androgen deprivation therapy) have individually, and in combination, been successful in predicting cancer-specific survival^{3, 18, 19}.

Multiple limitations should be considered when interpreting our study. All analyses were performed retrospectively and are therefore subject to the inherent limitations of this approach. Systems pathology is a novel platform, of which only an early version was available at the onset of our study. Subsequent developments and ongoing modifications may further improve its ability to evaluate smaller volume cancers and enhance its predictive accuracy. The post-operative BCR nomogram in this study outperformed previous applications to similar cohorts (CI: 0.85 versus 0.79 - 0.81)⁵ and may represent a fortuitously optimistic model. Such an improved performance may also be explained by a significant portion of the 881 patients in our study being present in the cohort used to develop the BCR nomogram. Additionally, our cohort consisted solely of patients undergoing RP and therefore it is unknown how the accuracy of these models would compare when analyzed in biopsy specimens before therapy.

Nevertheless, the ‘systems pathology’ approach described herein may well prove valuable, with further refinements, in elucidating prostate cancer biology or predicting prognosis. Through multiplexing of numerous immunofluorescent markers, members of our group have previously reported prostate cancer mechanistic insights²⁰. Further, as candidate biomarkers are discovered and proposed, ‘systems pathology’ instruments may aid in defining their role. Studies are underway to apply ‘systems pathology’ to biopsy specimens pre-treatment and also to a cohort of men treated conservatively without primary therapy.

Conclusions

Models to predict either biochemical recurrence or CF (metastases, androgenindependent disease, or death from prostate cancer) following RP exhibit similarly high levels of accuracy, regardless of statistical techniques or inclusion of selected morphometric and quantitative immunofluorescent features. Standard clinical and pathologic variables such as Gleason score and lymph node involvement appear to be the driving forces of clinical outcome. Clinicians and investigators wishing to predict CF can use standard models for biochemical recurrence and recalibrate the resulting probabilities.