Accuracy of the Kattan nomogram across prostate cancer risk-groups

Abstract

OBJECTIVE

• To investigate the predictive ability of nomograms at the extremes of preoperative clinical parameters by examining the predictive ability across all prostate cancer risk groups.

PATIENTS AND METHODS

• The Columbia University Urologic Oncology Database was reviewed: 3663 patients underwent radical prostatectomy from 1988 to 2008. Patients who had received neoadjuvant or adjuvant therapy, or had insufficient clinical parameters for estimation of 5-year progression-free probability using the preoperative Kattan nomogram were excluded.

• A total of 1877 patients were included and stratified by D’Amico risk criteria. Mean estimated nomogram progression rates were compared with actuarial Kaplan–Meier survival statistics.

• A regression model to predict progression-free survival was fitted with estimated nomogram score and concordance indices were calculated for the entire model and subsequently for each risk group.

RESULTS

• Of 1877 patients, 857 (45.6%) were low risk, 704 (37.5%) were intermediate risk, and 316 (16.8%) were high risk by D’Amico criteria.

• Mean estimated nomogram survival and actuarial Kaplan–Meier survival at 5 years were 90.5% and 92.2% (95% CI 89.2–94.3) for low-risk, 76.7% and 77.8% (73.3–81.7) for intermediate-risk, and 65.8% and 60.4% (52.0–67.7) for high-risk groups, respectively. Using nomogram score in the regression model, the c-index for the full model was 0.61.

• For low-, intermediate- and high-risk patients independently the c-index was 0.60, 0.59 and 0.57, respectively. When low-, intermediate- and high-risk patients were independently removed from the model the c-index was 0.64, 0.65 and 0.55, respectively.

• The c-index for the full model using the categorical nomogram risk scores was 0.67. Similar to the D’Amico model, the c-index improved to 0.69 when intermediate-risk patients were removed from the model.

CONCLUSIONS

• The study confirms the ability of preoperative nomograms to accurately predict actuarial survival across all risk groups.

• The predictive ability of the nomogram varies by risk group, yet even at the extremes of high-risk and low-risk prostate cancer the nomogram accurately predicts outcome.

INTRODUCTION

Carcinoma of the prostate is the most common solid organ malignancy to afflict men in the USA and accounted for 27 360 deaths in 2009 [1]. With the availability of serum PSA and transrectal ultrasound-guided needle biopsy of the prostate, asymptomatic and clinically organ-confined prostate cancer are increasingly diagnosed, with continuing uncertainty regarding the biological significance of some tumours. Several predictive tools have been developed to help guide patients and their physicians in the decision-making process after the cancer diagnosis is rendered to the patient. Predictive models have been shown to perform as well as or better than a physician’s clinical judgment when predicting probabilities of outcome [2]. One of the most commonly used tools is the nomogram developed by Kattan et al. [3], which incorporates clinical stage, Gleason grade on diagnostic biopsy and pretreatment serum PSA to predict biochemical recurrence 5 years after radical prostatectomy in those patients with clinically localized prostate cancer. An inherent limitation of nomograms is the reliance on the most common combinations of clinical features in a given population; as a result, rare cases are often under-represented. The aim of this study was to investigate the predictive ability of nomograms at the extremes of preoperative clinical parameters by examining the predictive accuracy of the Kattan nomogram across different risk groups.

PATIENTS AND METHODS

The Columbia University Comprehensive Surgical Urologic Oncology Database contains the details of 3663 patients who underwent radical prostatectomy from January 1988 to December 2008. Patients who had received neo-adjuvant or adjuvant therapy, had insufficient clinical parameters for estimation of 5-year progression-free probability using the preoperative Kattan nomogram, or had less than 1 year of follow-up were excluded. For purposes of risk stratification of the remaining 1877 patients in the study, D’Amico’s criteria [4,5] were applied. Low risk was defined as clinical Stage T1c or T2a, a PSA level of 10 ng/mL or less, and a Gleason sum of ≤6; intermediate risk was defined as either clinical Stage T2b, a PSA level >10 ng/mL but less than 20 ng/mL, or a Gleason sum of 7; and high risk was defined as clinical Stage T2c or greater, PSA level >20 ng/mL, or a Gleason sum of ≥8. Mean estimated nomogram progression rates were compared with actuarial Kaplan–Meier survival statistics. A regression model to predict progression-free survival was fitted with estimated nomogram score and concordance indices were calculated for the entire model and subsequently for each risk group.

To estimate the predictive ability of the current Kattan nomogram, we used two statistics: the concordance index and the Somers’ D statistic [6]. The concordance index is the probability that given two randomly selected patients, the patient with the worse outcome is predicted to have the worse outcome. The index ranges from 0.5, indicating the model performed no better than a random coin flip, to 1, indicating the model has perfect ability to rank patients. The Somers’ D statistic is the difference between the fraction of pairs for which the full model is more concordant than the reduced model and the fraction of pairs for which the reduced model is more concordant than the full. In this measure, a correlation coefficient of 0 represents no discriminating ability and a value of 1 represents perfect discrimination. It can be converted to a concordance index by dividing by 2 and adding to 0.5. All tests of statistical significance were two-sided. All analyses were conducted with STATA, version 9.0 (STATA, College Station, TX).

RESULTS

Of 1877 patients, 857 (45.6%) were classified as low risk, 704 (37.5%) as intermediate risk and 316 (16.8%) as high-risk before prostatectomy. Clinical and pathological characteristics are listed in Tables 1 and 2, respectively. In our cohort, 163 (8.7%) patients had biopsy Gleason score 7–10, 65 (3.5%) had PSA >20 ng/mL and 130 (6.9%) had clinical stage T2c or greater at diagnosis.

TABLE 1.

Clinical characteristics of 1877 patients

	No. of patients	%
D’Amico risk
Low	857	45.7
Intermediate	703	37.5
High	316	16.8
Biopsy Gleason score
2–6	1151	61.3
7	563	30.0
8–10	163	8.7
Prostate-specific antigen (ng/mL)
≤4	310	16.5
4.1–10	1263	67.3
10.1–20	239	12.7
>20	65	3.5
Clinical stage (1992 TNM)
T1a/b	16	0.9
T1c	1035	55.1
T2a	495	26.4
T2b	201	10.7
T2c	119	6.3
T3a	11	0.6

TABLE 2.

Pathological characteristics of 1877 patients treated with radical prostatectomy between 1988 and 2006

	Low		Intermediate		High		Overall
	n	%	n	%	n	%	n
Stage
pT2a	102	65.0	44	28.0	11	7.0	157
pT2b	54	54.5	43	43.4	2	2.0	99
pT2c	537	52.1	365	35.4	129	12.5	1031
pT3	120	24.4	218	44.3	154	31.3	492
pT4	5	16.7	14	46.7	11	36.7	30
Gleason score
2–6	502	71.8	151	21.6	46	6.6	699
7	295	33.0	471	52.7	127	14.2	893
8–10	19	8.8	64	29.6	133	61.6	216
Surgical margins
Negative	670	49.1	493	36.1	201	14.7	1364
Positive	148	33.3	191	43.0	105	23.6	444
Extracapsular extension
Absent	476	58.5	257	31.6	80	9.8	813
Into, not through	224	46.0	198	40.7	65	13.3	487
Present	119	24.5	209	43.1	157	32.4	485

Mean estimated nomogram survival and actuarial Kaplan–Meier survival at 5 years were 90.5% and 92.2% (95% CI 89.2–94.3) for low-risk, 76.7% and 77.8% (73.3–81.7) for intermediate-risk, and 65.8% and 60.4% (52.0–67.7) for high-risk groups, respectively (Table 3 and Fig. 1). Using nomogram score in the regression model, the c-index for the full model was 0.61. For low-, intermediate- and high-risk patients independently the c-index was 0.60, 0.59 and 0.57, respectively. The Somers’ D statistic for each risk group was 0.27, 0.30 and 0.10. To formally test whether the predictive ability of the nomogram varied across the risk groups, we computed the pairwise differences in the Somers’ D statistic with 95% CI. No significant differences were found in the three pairwise comparisons.

TABLE 3.

Mean estimated Kattan nomogram progression-free survival and actuarial 5-year progression-free survival stratified by D’Amico risk groups

	Estimated progression-free survival, %	Actuarial progression-free survival, % (95% CI)
Low risk	90.5	92.2 (89.2–94.3)
Intermediate risk	76.7	77.8 (73.3–81.7)
High risk	65.8	60.4 (52.0–67.7)

FIG. 1.

Progression-free survival stratified by D’Amico risk groups.

In a separate analysis, we computed the concordance index after removing patients from each of the risk groups (Table 4). When low-, intermediate- and high-risk patients were independently removed from the model the c-index was 0.64, 0.65 and 0.55, respectively. The c-index for the full model using the categorical nomogram risk scores was 0.67. Similar to the D’Amico model, the c-index improved to 0.69 when intermediate-risk patients were removed from the model.

TABLE 4.

Concordance (c) indices for the complete model, for each risk group and for combination models

	c-Index
Standard Kattan nomogram
Overall	0.61
Subgroup analysis
Low	0.60
Intermediate	0.59
High	0.57
Selective exclusion
Excluding low risk	0.64
Excluding intermediate risk	0.65
Excluding high risk	0.55
DAmico risk grouping
Overall	0.67
Low-High	0.69

DISCUSSION

Nomograms are designed to provide an individualized estimate of the predicted probability of the event of interest. However, development of these prognostic models entails analysis of outcomes in a large cohort of patients, usually from within a large academic centre. The Kattan nomogram was developed by analysing a cohort of men treated by a single surgeon at a US tertiary referral centre, and was based on a cohort of men in which the median PSA was 6.1 ng/mL, 84% were clinical stage < T2c and 68% had Gleason score 2–6 disease [3]. The nomogram was subsequently internally validated in patients at the same institution treated by five other surgeons, as well as several other national and international cohorts [7,8]. However, in all of these studies, most of the cohort consisted of what would be classified as low- and intermediate-risk population. In our study we found that while taken independently, each risk group performs similarly well compared with the complete model, the high-risk cohort made a significant contribution to the predictive accuracy of our model, with a robust deterioration in the concordance index when high-risk patients were removed from the analysis.

Other studies have also attempted to assess performance of the Kattan nomogram in various risk groups. In a study by Mitchell et al. [9] the 5-year recurrence-free probability after radical prostatectomy was calculated using a continuous multivariable preoperative nomogram among patients classified as low, medium and high risk using D’Amico criteria. Although low-risk patients uniformly had a high likelihood of being free of biochemical recurrence based on the probability calculated using the nomogram, a substantial proportion of intermediate-risk and even high-risk patients had a calculated 5-year recurrence-free probability of >90%. Moreover, a considerable overlap in the risk-grouping predictions was evident among intermediate-risk and high-risk patients. In our study the 5-year progression-free probability ranged from 72 to 97% for the low-risk group, from 26 to 93% for the intermediate-risk group and from 2 to 89% for the high-risk group. Although these results may be explained in part by the particular study population, these findings may show the difference between risk group classification and nomograms.

Risk stratification using the D’Amico criteria is dependent upon preoperative stage, biopsy grade and preoperative PSA. A unique feature of this stratification method is that risk is determined by the most clinically advanced variable rather than a consideration of all three. This provides the potential for a patient to be readily placed in a higher risk group based on a single clinical variable. As an example, a patient with clinical stage T2b, Gleason score 7 disease on biopsy and with a PSA of 20 ng/mL is lower risk than a patient of similar age with clinical stage T1c, Gleason score 6 disease on biopsy and a PSA of 21 ng/mL. Calculation of the above two patients’ progression-free probability at 5 years using the Kattan nomogram would yield 75% and 94%, respectively. Yossepowitch et al. [10] illustrated that categorical risk stratification of patients produces a wide range of predicted progression rates when a continuous multivariable analysis is used, especially in patients defined as ‘high risk’.

Very few studies have specifically assessed the accuracy of the Kattan nomogram at the extremes of the patient spectrum. The performance of the nomogram of Stephenson et al. [11] in the prediction of recurrence-free probability in patients with pretreatment PSA <2.5 ng/mL was investigated by Berglund et al. [12] in a large cohort of patients treated with radical prostatectomy. The study found that the preoperative nomogram functioned as a robust prediction model with no significant difference in biochemical recurrence outcomes than those predicted by the nomogram in the lower extreme of PSA values. Another study by Thanigasalam et al. [13] investigated the consequences of stage migration in the era of PSA testing on the prognostic accuracy of the Kattan nomogram. The study compared two groups of patients with localized prostate cancer treated with radical prostatectomy between 1991 and 1996 (Group 1) and 1997 and 2001 (Group 2). Group 2 had lower pathological stage disease and fewer cases with Gleason grade above 8. No difference was shown in the predictive accuracy of the Kattan nomogram between the two groups.

Despite their advantages, nomograms are not without limitations. Most nomograms are created from a cohort from a single centre of excellence or from highly specialized tertiary care centres, which may bias the outcome of the data collected. Furthermore, nomograms predicting biochemical recurrence after radical prostatectomy represent established and clinically useful decision aids. However, prediction of biochemical recurrence after treatment represents a surrogate endpoint. Definitive assessment of the effect of any risk factor will require analysis of either local or distant recurrence, cancer-specific or overall survival. These types of analysis depend on follow-up information and therefore follow-up and competing comorbidities records may represent a major limitation. Several nomograms and multivariable risk assessment models have been developed that assess such endpoints. Svatek et al. [14], for example, developed a nomogram that predicts cancer-specific survival in men with an androgen-independent variant of prostate cancer. The nomogram incorporated the following clinical variables: PSA at initiation of androgen deprivation therapy, PSA nadir during androgen deprivation therapy, time from androgen deprivation therapy to development of androgen-insensitive prostate cancer, and PSA doubling time since androgen-insensitive prostate cancer diagnosis. Bootstrap-corrected predictive accuracy of this nomogram was 80.9%. The limitation of this nomogram is that it has not been validated with an external data set and was developed from a single institution’s patient population. Another risk assessment model developed at UCSF uses variables such as age at diagnosis, PSA at diagnosis, Gleason score of the biopsy, clinical stage and percentage of biopsy cores involved with cancer to calculate Cancer of the Prostate Risk Assessment (CAPRA) score [15]. The CAPRA score, which has been validated in multiple studies involving more than 9000 patients treated with radical prostatectomy has also been recently shown to accurately predict an individual’s likelihood of metastatsis, cancer-specific mortality, and overall mortality across various treatment modalities [16].

Another limitation of Kattan nomogram is that it assesses the biochemical recurrence risk at 5 years. Twenty-seven percent of biochemical recurrences occur beyond 5 years following radical prostatectomy [17,18]. Stephenson et al. [11] created a nomogram that looks at recurrence at 10 years of follow-up and showed from 79 to 81% accuracy in independent validation sets. A more recent study by Suardi et al. [19] described a model capable of predicting biochemical recurrence up to at least 15 years following RP. The nomogram predictor variables included pathological stage, surgical margin status, pathological Gleason score, type of radical prostatectomy and use of adjuvant radiotherapy. After 200-bootstrap internal validations, the predictive accuracy of the nomogram was 79.3%, 77.2%, 79.7% and 80.6% at 5 years, 10 years, 15 years and 20 years, respectively. In the second external validation cohort, the predictive accuracy of the nomogram was 77.9%, 79.4% and 86.3% at 5 years, 10 years and 15 years after radical prostatectomy, respectively. External validity could not be tested at 20 years because of insufficient follow-up.

Novel molecular markers, which reflect biological behaviour of prostate cancer, are increasingly being incorporated into nomograms. Kattan et al. [20] describe a nomogram which incorporates pretreatment plasma levels of interleukin-6 soluble receptor and TGF-β₁ in addition to the standard pretreatment PSA level, clinical stage and biopsy Gleason grade. Addition of pretreatment interleukin-6 soluble receptor and TGF-β₁ improved the ability of the nomogram to predict biochemical progression by a statistically significant margin.

Continuous multivariable models such as nomograms currently represent the most accurate tools for predicting the outcome of patients who undergo definitive therapy for localized prostate cancer. Our study confirms the ability of the preoperative Kattan nomogram to accurately predict actuarial survival across all risk groups. The predictive ability of the nomogram varies by risk group, yet even at the extremes of high-risk and low-risk prostate cancer the model accurately predicts outcome.

What’s known on the subject? and What does the study add?

The Kattan nomogram is one of the most commonly used preoperative prediction tools for estimating individualized risk of biochemical recurrence after radical prostatectomy. However, little is known about this nomogram’s accuracy for patients at the extremes of the risk spectra, as only a small fraction of such patients comprised the cohort used in its development. We examined the accuracy of the Kattan nomogram across various risk groups, and confirmed its ability to accurately estimate risk of recurrence, even for patients with high and low-risk prostate cancer.