The effect of race on the discriminatory accuracy of models to predict biochemical recurrence after radical prostatectomy: results from the Shared Equal Access Regional Cancer Hospital and Duke Prostate Center databases

Abstract

To evaluate whether race modifies the accuracy of nomograms to predict biochemical recurrence (BCR) after radical prostatectomy among subjects from the Shared Equal Access Regional Cancer Hospital (SEARCH) and Duke Prostate Center (DPC) databases. Retrospective analysis of 1721 and 4511 subjects from the SEARCH and DPC cohorts, respectively. The discrimination accuracy for BCR of seven previously published predictive models was assessed using concordance index and compared between African-American men (AAM) and Caucasian men (CM). AAM represented 44% of SEARCH and 14% of DPC. In both cohorts, AAM were more likely to experience BCR than CM (P<0.01). In SEARCH, the mean concordance index across all seven models was lower in AAM (0.678) than CM (0.715), though the mean difference between CM and AAM was modest (0.037; range 0.015–0.062). In DPC the overall mean concordance index for BCR across all seven nomograms was 0.686. In contrast to SEARCH, the mean concordance index in DPC was higher in AAM (0.717) than CM (0.681), though the mean differences between CM and AAM was modest (−0.036; range −0.078 to −0.004). Across all seven models for predicting BCR, the discriminatory accuracy was better among CM in SEARCH and better among AAM in DPC. The mean difference in discriminatory accuracy of all seven nomograms between AAM and CM was approximately 3%–4%. This indicates that currently used predictive models have similar performances among CM and AAM. Therefore, nomograms represent a valid and accurate method to predict BCR regardless of race.

Introduction

Risk stratification in prostate cancer is essential to determine which patients are more likely to respond to specific interventions, which patients may benefit from adjuvant therapy and which patients are likely to progress despite our best efforts.¹ Several models and nomograms to predict the probability of cancer recurrence after radical prostatectomy have been published in the last 10 years.² These models were predominantly developed from data sets with few African-American men (AAM) because the large majority in these data sets was composed of Caucasian men (CM). However, AAM are known to have worse disease at diagnosis and higher failure rates after surgery than CM.³ Thus, given these significant racial disparities, it is not clear whether predictive models currently available are as accurate in AAM as among CM. Until now, only one study has compared the performance of two commonly used nomograms between AAM and CM: the preoperative and post-operative Kattan nomograms.⁴ In that article, the two nomograms had similar discrimination and calibration regardless of race. However, whether these findings apply broadly to multiple predictive models across multiple patient populations remains unknown. The Shared Equal Access Regional Cancer Hospital (SEARCH) cohort is particularly suitable to explore racial disparities given its equal-access nature and the fact AAM represent nearly half of its population.³ The Duke Prostate Center (DPC) cohort is also well suited to investigate the effect of race on prostate cancer as it has a significant representation of AAM.⁵ Therefore, we sought to perform a more comprehensive comparison of the performance of multiple commonly used nomo-grams to predict biochemical recurrence (BCR) after radical prostatectomy between AAM and CM in two distinct multiracial cohorts: the SEARCH and the DPC databases.

Materials and methods

Study population

After obtaining Institutional Review Board approval from each institution, data from patients undergoing radical prostatectomy between 1988 and 2008 at 4 Veterans Affairs Medical Center (Greater Los Angeles and Palo Alto, CA; Augusta, GA; Durham, NC, USA) were combined into the SEARCH Database.³ Data from patients operated at Duke University Medical Center during the same period were abstracted into the DPC Database.⁶ Both databases include information on patient age at surgery, race, height, weight, clinical stage, cancer grade on diagnostic biopsies, preoperative PSA, surgical specimen pathology (specimen weight, tumor grade, stage and surgical margin status) and follow-up PSA.^6,7 Patients treated with preoperative hormonal therapy or radiotherapy were excluded in both data sets. Of 1975 patients in SEARCH, we excluded 71 (4%) patients because of missing follow-up data. We also excluded 183 (9%) men who were neither CM nor AAM. This resulted in a study population of 1721 subjects from SEARCH. Of 4627 patients in DPC, we excluded 38 (1%) patients due missing follow-up data and 78 (1%) men who were considered neither CM nor AAM. This resulted in a study population of 4511 subjects.

All patients were followed with serial PSA determinations and clinical visits at intervals according to the attending physician discretion. In SEARCH, BCR was defined as a single PSA above 0.2 ng/ml, 2 concentrations at 0.2 ng/ml or secondary treatment for an elevated PSA.⁸ In DPC, BCR was defined as a PSA level ≥0.2 ng/ml or secondary treatment for an elevated PSA.⁵ Secondary treatment after surgery was at the judgment of the patient and treating physician.

Statistical analysis

Comparison of baseline patients' and disease characteristics between AAM and CM was performed using χ²-test for categorical data and rank-sum test for continuous variables. The univariable association between race and BCR-free survival was analyzed using Kaplan– Meier plots and log-rank test. The association between race and BCR adjusted for each nomogram score was done using Cox proportional hazards. The overall discrimination for BCR of the various models was determined using Harrell's concordance index.⁹ Concordance index represents the probability that in a pair of randomly selected men, the one having the higher estimated probability of recurrence actually experienced an earlier recurrence. The difference in concordance index was calculated by subtracting the concordance index obtained in AAM from the one obtained in CM. The predictive discrimination of the following seven models was assessed: (1) the risk classification published by D'Amico et al.¹⁰ (2) the pre-¹¹ and (3) post-operative¹² nomograms developed by Kattan et al. (4) the DPC nomogram⁵ (5) the Cancer of the Prostate Risk Assessment (CAPRA) model¹³ (6) a nomogram from Center for Prostate Disease Research and Cancer of the Prostate Strategic Research Endeavor (CPDR/CaPSURE)¹⁴ and (7) a model published by investigators at Johns Hopkins Hospital (JHH).¹⁵ All statistical analyses were performed using Stata 10.0 (StataCorp, College Station, TX, USA) and R 2.8.1 (R Foundation for Statistical Computing, Vienna, Austria) with Design 2.1–2 and Hmisc 3.5–2 libraries. A P<0.05 was considered statistically significant.

Results

Of the 1721 patients included in the SEARCH cohort, AAM composed 44% (n=761). AAM were treated in more contemporary years and had a significantly younger age, higher preoperative PSA, lower preoperative stage and less extracapsular extension (Table 1). In DPC, AAM represented 14% of the total population (650 subjects). AAM were significantly younger, were treated in more contemporary years and had higher preoperative PSA values, higher biopsy and pathological Gleason scores and higher prevalence of positive surgical margins (Table 2).

Table 1.

SEARCH population baseline characteristics

Variables	African American	Caucasian	P-value
Number of patients, N (%)	761 (44)	960 (56)	—
Age at surgery (years)			<0.001
Median (IQR)	61 (56–66)	63 (59–67)
VA center, N (%)			<0.001
1	258 (34)	282 (29)
2	43 (5)	178 (12)
3	181 (24)	169 (18)
4	279 (37)	331 (35)
Preoperative PSA (ng ml−1)			<0.001
Median (IQR)	7.4 (5.1–11.5)	6.9 (4.8–10.4)
Preoperative stage			<0.001
T1	439 (62)	399 (45)
T2–T3	268 (38)	479 (55)
Biopsy Gleason score, N (%)			0.158
2–6	449 (61)	672 (62)
7	225 (31)	259 (28)
8–10	56 (8)	93 (10)
Positive biopsy scores (%)			0.542
Median (IQR)	33 (16–50)	33 (16–50)
Positive surgical margins, N (%)	338 (46)	440 (47)	0.818
Extracapsular extension, N (%)	144 (20)	261 (28)	<0.001
Seminal vesicle invasion, N (%)	81 (11)	96 (10)	0.540
Positive lymph nodes, N (%)	8 (1)	16 (2)	0.361
Pathology Gleason score, N (%)			0.003
2–6	282 (38)	381 (40)
7	393 (53)	440 (47)
8–10	62 (9)	122 (13)
Prostate weight (g)			0.716
Median (IQR)	39 (30–61)	38 (30–50)
Follow-up (months)			<0.001
Median (IQR)	44 (20–79)	56 (27–95)

Abbreviation: IQR, interquartile range.

Table 2.

DPC population baseline characteristics

Variables	African American	Caucasian	P-value
Number of patients, N (%)	650 (14)	3861 (86)	—
Age at surgery (years)			<0.001
Median (IQR)	61 (56–66)	64 (58–69)
Preoperative PSA (ng ml−1)			<0.001
Median (IQR)	7.3 (5.2–12.2)	6.5 (4.6–10.3)
Preoperative stage			0855
T1	363 (74)	1935 (74)
T2–T3	130 (26)	679 (26)
Biopsy Gleason score, N (%)			<0.001
2–6	366 (65)	2408 (72)
7	127 (23)	705 (21)
8–10	67 (12)	250 (7)
Positive biopsy scores (%)			0.963
Median (IQR)	50 (33–100)	50 (33–100)
Positive surgical margins, N (%)	246 (43)	1234 (35)	<0.001
Extracapsular extension, N (%)	193 (31)	1324 (35)	0.032
Seminal vesicle invasion, N (%)	79 (12)	476 (12)	0.911
Positive lymph nodes, N (%)	6 (1)	71 (2)	0.103
Pathology Gleason score, N (%)			<0.001
2–6	242 (37)	1877 (49)
7	322 (50)	1485 (38)
8–10	86 (13)	499 (13)
Prostate weight (g)			0.973
Median (IQR)	38 (30–50)	38 (30–50)
Follow-up (months)			<0.001
Median (IQR)	48 (22–89)	69 (29–117)

Abbreviation: IQR, interquartile range.

In SEARCH, over a median follow-up of 50 months, 598 individuals (35%) experienced BCR. The median follow-up after radical prostatectomy in subjects that did not develop BCR was significantly higher in CM (56 versus 44 months in AAM, P<0.001). In DPC, 1536 men (34%) developed BCR over a median follow-up of 66 months. A significant difference in follow-up between CM and AAM was also observed among those that did not recur (69 versus 48 months, respectively, P<0.001). In both SEARCH (P=0.002) and DPC (P=0.005), AAM were more likely to experience BCR than CM (Figures 1 and 2). We also found AAM, in both cohorts, were more likely to develop BCR even after adjustments for each nomogram score, though this did not always reach statistical significance (Table 3).

Figure 1.

Biochemical recurrence-free survival after radical prostatectomy in the SEARCH cohort.

Figure 2.

Biochemical recurrence-free survival after radical prostatectomy in the DPC cohort.

Table 3.

Association between race and biochemical recurrence adjusted for nomogram score in the SEARCH and DPC cohorts

Nomogram	SEARCH			DPC
	HRa	95% CI	P-value	HRa	95% CI	P-value
Preoperative
D'Amico	1.25	1.06–1.48	0.010	1.10	0.94–1.29	0.241
CAPRA	1.21	1.00–1.46	0.055	1.18	0.93–1.50	0.182
Kattan preop	1.30	1.07–1.57	0.008	1.30	1.02–1.65	0.035
Postoperative
DPCb	—	—	—	—	—	—
Kattan postop	1.36	1.14–1.62	0.001	1.16	0.97–1.38	0.098
CPDR/CaPSUREb	—	—	—	—	—	—
JHH	1.35	1.14–1.60	0.001	1.05	0.90–1.24	0.517

Abbreviations: DPC, Duke Prostate Center; JHH, Johns Hopkins Hospital.

^aThe HR represents the risk of biochemical recurrence among AAM as compared with the reference group (CM).

^bThe DPC and CPDR/CaPSURE nomograms were excluded from this analysis given they include race as a covariate in their models.

Among all men in SEARCH (AAM and CM), the mean concordance index for BCR across all 7 nomograms was 0.699 (Table 4). The mean concordance index in AAM and CM were, respectively, 0.678 and 0.715. The mean difference in concordance index between CM and AAM was 0.037 (range 0.015–0.062). The highest accuracy variation between AAM and CM was observed in the preoperative Kattan model (0.062). Thus, in SEARCH, across all seven models for predicting BCR, the discriminatory accuracy of the models was better among CM for all models.

Table 4.

Concordance indexes of various models for predicting biochemical recurrence by race in the SEARCH cohort

Nomogram	All patients			Caucasian			African American			C-index differencea
	C-index	Nb	Rc	C-index	Nb	Rc	C-index	Nb	Rc
Preoperative
D'Amico	0.6358	1614	551	0.6550	889	286	0.6137	725	265	0.0412
CAPRA	0.6769	1318	419	0.7052	700	206	0.6463	617	213	0.0588
Kattan postop	0.6683	1311	418	0.6993	694	205	0.6371	617	213	0.0622
Postoperative
DPC	0.7440	1423	455	0.7577	770	232	0.7256	653	223	0.0322
Kattan postop	0.7278	1512	504	0.7416	847	269	0.7160	665	235	0.0256
CPDR/CaPSURE	0.6939	1599	543	0.7157	889	286	0.6867	710	257	0.0289
JHH	0.7226	1581	535	0.7295	896	288	0.7178	685	247	0.0117
Average	0.6956	1480	489	0.7149	812	253	0.677	666	8236	0.0372

Abbreviations: DPC, Duke Prostate Center; JHH, Johns Hopkins Hospital.

^aC-index difference denotes the C-index in Caucasian minus C-index in African American.

^bN: number of subjects in the model.

^cR: number of recurrences in the model.

In DPC we found the opposite results. Of all seven models, the discriminatory accuracy was better among AAM in all models. The overall mean concordance index for BCR among the nomograms in DPC was 0.686. The mean concordance index in AAM and CM were, respectively, 0.718 and 0.681. The mean difference in concordance index between CM and AAM was −0.042 (range from −0.078 to −0.004). The highest accuracy variation between AAM and CM was observed in the Johns Hopkins Hospital model (0.078, Table 5).

Table 5.

Concordance indexes of various models for predicting biochemical recurrence by race in the DPC cohort

Nomogram	All patients			Caucasian			African American			C-index differencea
	C-index	Nb	Rc	C-index	Nb	Rc	C-index	Nb	Rc
Preoperative
D'Amico	0.6331	3491	1052	0.6266	2952	872	0.6604	539	180	−0.0338
CAPRA	0.6816	1987	447	0.6759	1678	366	0.7055	309	81	−0.0296
Kattan preop	0.6829	1979	445	0.682	1670	364	0.6826	309	81	−0.0004
Postoperative
DPC	0.7354	2948	746	0.7259	2488	614	0.7771	460	132	−0.0511
Kattan postop	0.7231	3323	950	0.7173	2831	801	0.7551	492	149	−0.0378
CPDR/CAPSURE	0.6759	3642	1117	0.6782	3084	926	0.7389	562	176	−0.0248
JHH	0.6717	4044	1279	0.6611	3482	1103	0.7389	562	176	−0.0778
Average	0.6862	3059	862	0.6810	2598	721	0.7175	461	141	−0.0365

Abbreviations: DPC, Duke Prostate Center; JHH, Johns Hopkins Hospital.

^aC-index difference denotes the C-index in Caucasian minus C-index in African American.

^bN: number of subjects in the model.

^cR: number of recurrences in the model.

To explore why in SEARCH all models performed modestly better among CM, whereas the opposite finding was noted in DPC, we hypothesized that the predicted risks across the nomograms would be highly correlated given that similar variables are used in all the nomograms tested (that is, PSA, stage and grade). To assess this, we compared the degree of correlation between the models within each data set. We found all predicted survivals were moderately to highly correlated in both SEARCH (Spearman r=0.39–0.92) and DPC (Spearman r=0.31–0.91).

Discussion

Racial disparities in prostate cancer have been attributed to multiple causes including socioeconomic factors,¹⁶ lifestyle behaviors,¹⁷ increased genetic predisposition,¹⁸ decreased detection rates,¹⁹ advanced stage and a worse tumor grade at diagnosis⁷ and increased risk of recurrence after treatment.^3,5,20 These features certainly affect the way prostate cancer patients are treated.²¹ However, in light of these differences, whether race impacts on the accuracy of models to predict BCR after radical prostatectomy has not been fully explored. Indeed, only one study has compared the performance of two models (Pre- and Post-operative Kattan nomograms) between AAM and CM.⁴ In that study, the two models had similar discrimination and calibration regardless of race. However, the accuracy of other widely used predictive tools such as the D'Amico classification, CAPRA, Johns Hopkins Hospital, CPDR/CaPSURE and DPC models have not been evaluated between AAM and CM. This is particularly relevant given these nomograms were developed in cohorts predominantly composed by CM. As both SEARCH and DPC cohorts have a significant representation of AAM, we sought to compare the performance of these nomograms to predict BCR after radical prostatectomy in AAM and CM among subjects from the SEARCH and DPC databases.

In SEARCH, we found that AAM presented at younger ages and with higher preoperative PSA levels, yet had lower clinical stage and less extracapsular extension. Despite more favorable clinical and pathological staging, AAM were more likely to develop BCR than CM. In DPC, we found similar results: AAM presented with a significantly younger age, higher preoperative PSA and had a higher prevalence of positive surgical margins. The incidence of BCR was also significantly higher in AAM compared with CM. These findings mirror data from other centers that have also found that AAM present with lower risk disease, but have higher risk of recurrence.^14,22 Given these differences in baseline disease characteristics and dissimilar risk for recurrence between races and the fact that many of the previously published nomograms were created using cohorts composed of few AAM,^5,10–15 it is unclear whether these nomograms accurately assess risk among AAM.

Despite these baseline disease characteristic differences between AAM and CM, we found that across all seven previously published models for predicting BCR, the discriminatory accuracy of the models was better among CM in SEARCH and better among AAM in DPC. Furthermore, the mean difference in discriminatory accuracy of all seven nomograms to predict BCR between AAM and CM was only 3–4%. Although the clinical relevance of a 3–4% difference can be debated, the fact that there was no clear direction in that in SEARCH the models performed better in CM whereas in the DPC the models performed better in AAM suggests that there is no systematic bias in the performance of commonly used predictive models towards one race or another. Similarly, an earlier study evaluating only two nomograms came to a similar conclusion that race does not modify the accuracy of prognostic models.⁴ Moreover, in an earlier study from SEARCH, we found that race does not alter the prognostic ability of the Partin tables to predict pathological stage at the time of radical prostatectomy.²³ As such, despite significant differences at presentation and in terms of recurrence rates, in clinical practice we conclude that commonly used models perform equally well regardless of race.

We hypothesize that the reason for all nomograms having a similar performance within the same population (that is, all better in AAM or all better in CM) is related to the high degree of correlation among all models. For example, if a patient was high-risk on one nomogram, it was highly likely that he would be high-risk on all nomograms. This is explained by the fact that all nomograms used many of the same variables (that is, PSA, grade and stage) as predictive covariates. Thus, it suggests that if a given nomogram performed slightly better in one subset of patients for one reason or another, then it is highly likely all nomograms would perform slightly better in that subset of patients.

Two out of the seven published nomograms include race as a covariate (CPDR/CaPSURE and DPC). The mean concordance index difference of these two nomograms between races was 3–5%, very similar to the mean difference observed in the models that do not include race as a cofactor. Intuitively this makes sense in that though the overall model includes race, when analyses are restricted to a single racial group, the use of race does not help discrimination as there is no variability within the group. As such, these findings cannot and should not be used to assess whether adding race to a nomogram improves or decreases predictive accuracy either on the whole or among specific racial groups.

This study is limited by the retrospective nature of our cohort. Furthermore, one of the models tested (DPC) was developed within the DPC cohort. However, the overall accuracy of this nomogram by race was similar to the other nomograms tested (that is, better in CM in SEARCH and better in AAM in DPC) and thus this fact did not appear to influence our results. In addition, both of our populations were considerably different from the various populations used for model development. Moreover, other demographic, geographic, historical and methodological differences between SEARCH, DPC and the cohorts used to derive the various models tested may influence their accuracies. These significant variations, however, are, in fact, strengths of our study because they allow for the assessment of generalizability of the various models and transportability among considerably dissimilar populations.²⁴

In summary, across all seven previously published models for predicting BCR, the discriminatory accuracy of the models was better among CM in SEARCH, and better among AAM in DPC. The mean difference in discriminatory accuracy of all seven nomograms to predict BCR between AAM and CM was approximately 3–4%. Overall, these results indicate that currently used predictive models have a similar performance among CM and AAM. Therefore, these nomograms represent a valid and accurate method to predict BCR regardless of race.