Abstract
Objectives
To compare the predictive performance and potential clinical usefulness of risk calculators of the European Randomized Study of Screening for Prostate Cancer (ERSPC RC) with and without information on prostate volume.
Methods
We studied 6 cohorts (5 European and 1 US) with a total of 15,300 men, all biopsied and with pre-biopsy TRUS measurements of prostate volume. Volume was categorized into 3 categories (25, 40, and 60 cc), to reflect use of digital rectal examination (DRE) for volume assessment. Risks of prostate cancer were calculated according to a ERSPC DRE-based RC (including PSA, DRE, prior biopsy, and prostate volume) and a PSA + DRE model (including PSA, DRE, and prior biopsy). Missing data on prostate volume were completed by single imputation. Risk predictions were evaluated with respect to calibration (graphically), discrimination (AUC curve), and clinical usefulness (net benefit, graphically assessed in decision curves).
Results
The AUCs of the ERSPC DRE-based RC ranged from 0.61 to 0.77 and were substantially larger than the AUCs of a model based on only PSA + DRE (ranging from 0.56 to 0.72) in each of the 6 cohorts. The ERSPC DRE-based RC provided net benefit over performing a prostate biopsy on the basis of PSA and DRE outcome in five of the six cohorts.
Conclusions
Identifying men at increased risk for having a biopsy detectable prostate cancer should consider multiple factors, including an estimate of prostate volume.
Keywords: PSA, Risk, Prostate cancer, Prostate volume, Calibration, Net benefit
Introduction
Although screening for prostate cancer (PC) can reduce disease-specific mortality [1–3], key problems are over diagnosis and unnecessary testing (i.e., prostate biopsies). Applying a purely PSA-based biopsy indication (PSA ≥ 3.0 ng/ml as indication for prostate biopsy) resulted in 76% of biopsies being performed in men who were found not to have cancer on that biopsy [2]. Increasing the PSA threshold above which men are recommended to undergo a biopsy causes a considerable number of cancers to be missed [4]. A more promising approach is to refine the biopsy indication based on the multivariable combination of PSA with other patient and disease characteristics [5].
We previously developed a multistep PC risk calculator based on data from the European Randomized Study of Screening for Prostate Cancer (ERSPC) Rotterdam (http://www.prostatecancer-riskcalculator.com). The calculator is meant as a decision aid for lay people (RC#1), general practitioners (RC#2), and urologists (RC#3 for men previously unscreened, RC#4 for men with a previous PSA test, RC#5 for men with a previous negative biopsy, and RC#6 to predict indolent PC) by providing individualized estimates of the risk of detecting PC on biopsy [6, 7].
The calculators have some limitations (i.e., based on sextant biopsy information), but have been validated in several cohorts with satisfactory results [8–10]. Clinical implementation may however be hampered by the need to perform a TRUS before biopsy. We therefore developed variants of the original calculators without the need for TRUS measurements. Next to PSA and DRE outcome (abnormal vs. normal), the DRE can also estimate prostate volume and may hence avoid the need for TRUS before biopsy [11].
We aimed to test the validity of the DRE-based volume ERSPC risk calculator and to compare its performance to a biopsy strategy based on the PSA value and the outcome of the DRE (i.e., abnormal vs. normal with no information on prostate volume included in the risk prediction).
Patients and methods
Patients
Data were from the Prostate Biopsy Collaborative Group, which includes ten European and US cohorts as previously described [12]. We excluded 4 cohorts where prostate volume was not available, leaving 6 cohorts with a total of 15,300 men for analysis. These included four screening cohorts from the ERSPC: Göteborg, Sweden Round 1 screening (GOTEBORG-R1), Göteborg Rounds 2–6 (GOTEBORG-R2–6), Rotterdam, the Netherlands, Round 1 (PSA ≥3.0 ng/ml) (ROTTERDAM-R1), Rotterdam Rounds 2–3 (ROTTERDAM-R2–3); one other screening cohort (Tyrol, Austria) (TYROL); and one US clinical cohort [Cleveland Clinic, Ohio (CCF)]. Biopsy decisions for all ERSPC cohorts were indicated by PSA ≥3 ng/ml (4 ng/ml for Rotterdam during some years) and for the other cohorts by clinical indication typically an elevated PSA without strict protocol. Biopsy schemes for all ERSPC cohorts were 6-core, for CCF, greater than 8-core, Tyrol 6, 10, or 12 depending on year. All cohorts except for GOTEBORG-R1 and ROTTERDAM-R1 included some men who had been previously screened. Biopsies after a positive biopsy for PC were excluded from the analysis.
Methods
The ERSPC DRE-based volume calculators were developed on a cohort of 3,624 men who had never before had a biopsy (DREvol-RC3) and a cohort of men (N = 2,896) previously screened/biopsied (DREvol-RC4/5) [11]. The model based on PSA and DRE outcome only was developed on similar cohorts resulting in a model suitable for men previously unscreened (PSADRE-model) and men previously screened and/or biopsied (PSADRE-model, see “Appendix” for formulas). Characteristics of each cohort were summarized by descriptive statistics. For each biopsy in the dataset, risks were computed using the DREvol-RC3 (men not previously biopsied) or DREvolRC4/5 (men previously biopsied) [11] as well as the model based on PSA and DRE alone (PSADRE-model) and compared to the outcome of presence of cancer at biopsy. Missing values were imputed (filled in) based on correlations between any of the predictor variables and correlations with the end point. A single imputation was performed, using the first set of imputed values from a multiple imputation procedure (aregImpute, as implemented in R v 2.10.1 software) [13].
Since no data on prostate volume as assessed with DRE were available in the validation cohorts, TRUS-assessed prostate volume was recoded as in [11] into three volume classes as can be estimated by DRE. TRUS-assessed volumes <30 cc were recoded as 25 cc, volumes between 30 and 50 cc as 40 cc, and volumes ≥50 cc as 60 cc.
The predictive performance was evaluated by validation plots, comparing the frequency of observed outcome to predicted risks [14]. Perfect calibration results in a 45° line of a loess smoother in this plot. The calibration slope should be 1, with values below 1 indicating too extreme predictions, as might result from overfitting [13].
The discriminative ability of predictions is reflected in the spread of the predictions, stratified by the presence of cancer at biopsy. The area under the ROC curve (AUC, equivalent to c statistic) summarized discriminative ability, with pairwise testing by the DeLong test.
The clinical usefulness of the calculators was evaluated by decision curves [15].
We focused on the range from 10 to 40% for the decision threshold. This is a reasonable range of thresholds since some patients might opt for biopsy with a chance cancer of 10% while few patients would refuse biopsy if their risk was greater than 1 in 2–3.
Results
Study population
Median age ranged from 61 to 67 years and median PSA levels from 3.5 to 5.8 ng/ml. The percentage of men with an abnormal DRE result ranged from 10 to 26% in the screening cohorts and was remarkably lower in the clinical cohort where only 6% of the men had an abnormal DRE. The missing data on prostate volume ranged from 0.4 to 30%. Despite the fact that two of the ERSPC cohorts and the Tyrol-cohort included men who were previously screened and/or biopsied, the cancer detection rates in the screening cohorts were similar (26–28%, Table 1).
Table 1.
Descriptives of the 6 validation cohorts
| Goteborg round 1 | Goteborg rounds 2–6 | Rotterdam round 1 | Rotterdam rounds 2–3 | Cleveland clinic | Tyrol | |
|---|---|---|---|---|---|---|
| Number of patients | 740 | 1,241 | 2,895 | 1,494 | 2,631 | 4,199 |
| Number of biopsies | 740 | 1,241 | 2,895 | 1,494 | 3,286 | 5,644 |
| Age | ||||||
| Median (range) | 61 (51, 70) | 63 (53, 71) | 66 (55, 75) | 67 (59, 75) | 64 (50, 75) | 63 (50, 75) |
| PSA median (range) | 4.7 (0.5, 226.0) | 3.6 (2.0, 88.8) | 5.0 (0.0, 245.0)a | 3.5 (0.4, 99.5) | 5.8 (0.2, 491.7) | 4.2 (0.1, 3,210.0) |
| DRE result | ||||||
| Normal | 614 (83%) | 1,117 (90%) | 2,137 (74%) | 1,182 (79%) | 3,083 (94%) | 5,076 (90%) |
| Abnormal | 126 (17%) | 124 (10%) | 758 (26%) | 312 (21%) | 203 (6%) | 568 (10%) |
| Unknown | 0 | 0 | 0 | 0 | 0 | 0 |
| Prostate volume | ||||||
| Median (range) | 44 (12–241) | 40 (3–131) | 51 (5–239) | 46 (15–130) | 42 (0–737) | 40 (7–652) |
| Unknown (N, %) | 7 (0.9) | 274 (22.1) | 13 (0.4) | 7 (0.5) | 986 (30.0) | 476 (8.4) |
| Family history | ||||||
| No | 0 | 0 | 1,708 (59%) | 875 (59%) | 1,690 (51%) | 0 |
| Yes | 0 | 0 | 328 (11%) | 160 (11%) | 373 (11%) | 0 |
| Unknown | 740 (100%) | 1,241 (100%) | 859 (30%) | 459 (31%) | 1,223 (37%) | 5,644 (100%) |
| African origin | ||||||
| No | 0 | 0 | 0 | 0 | 2,818 (86%) | 0 |
| Yes | 0 | 0 | 0 | 0 | 422 (13%) | 0 |
| Unknown | 740 (100%) | 1,241 (100%) | 2,895 (100%) | 1,494 (100%) | 46 (1%) | 5,644 (100%) |
| Prior biopsy | ||||||
| Yes | 0 | 0 | 0 | 0 | 1,091 (33%) | 1,555 (28%) |
| No | 740 (100%) | 1,241 (100%) | 2,895 (100%) | 1,494 (100%) | 2,195 (67%) | 4,089 (72%) |
| Unknown | 0 | 0 | 0 | 0 | 0 | 0 |
| Cancer | 192 (26%) | 322 (26%) | 800 (28%) | 388 (26%) | 1,292 (39%) | 1,562 (28%) |
| Biopsy Gleason grade | ||||||
| ≤6 | 152 (79%) | 269 (84%) | 508 (64%) | 297 (77%) | 669 (52%) | 911 (58%) |
| 7 | 33 (17%) | 45 (14%) | 234 (29%) | 78 (20%) | 478 (37%) | 319 (20%) |
| ≥8 | 7 (4%) | 8 (2%) | 52 (6%) | 13 (3%) | 145 (11%) | 137 (9%) |
| Unknown | 0 | 0 | 6 (1%) | 0 | 0 | 195 (12%) |
aPSA values were re-measured at the laboratory of Prof. Lilja in NY resulting in PSA levels <3.0 ng/ml
Discrimination
In all validation cohorts, the AUC of the DREvol-RC was larger than that of the PSADRE-model (each pair wise comparison: p < 0.001). In the strictly protocol-based screening cohorts of the ERSPC centers, the AUCs for the DREvol-RC and PSADRE-model were 0.776 (0.735–0.816) and 0.719 (0.673–0.764) for GOTEBORG-R1; 0.609 (0.573–0.646) and 0.563 (0.525–0.601) for GOTEBORG-R2–6; 0.746 (0.724–0.767) and 0.692 (0.669–0.715) for ROTTERDAM-R1; and 0.659 (0.628–0.690) and 0.606 (0.571–0.638) for ROTTERDAM-R2–3, respectively. Hence, the improvement in AUC by using the DREvol-RC versus the PSADRE-model was approximately 0.05, although we note that Rotterdam R1 is part of the development cohort and hence a likely optimistic validation. In the two large, not ERSPC validation cohorts, the differences in AUCs of the DREvol-RC and the PSADRE-model were similar: for the CCF cohort, the AUCs were 0.665 (0.646–0.684) versus 0.620 (0.600–0.640) and for the Tyrol-cohort 0.720 (0.705–0.735) versus 0.675 (0.659–0.690).
Calibration
As expected, the calibration of the DREvol-RC was very good for the ROTTERDAM-R1 cohort (Fig. 1). The validation cohort ROTTERDAM-R2–3 differs from the development cohort in that men with previous negative biopsy were included in the development but not in the validation cohort, and results from round 3 were included only in the validation but not the development cohort. Yet, the validation plot showed good calibration with a calibration slope close to 1 (0.96). Calibration in the large (intercept of 0.20) indicates minor underprediction of cancer at biopsy. Calibration was also satisfactory for the GOTEBORG-R1 and GOTEBORG-R2–6 cohorts, but with some over prediction for probabilities above 40%. In the large, screening-based Tyrol-cohort, minor underprediction was noted. In the clinically derived CCF cohort, a considerable underprediction was noted, especially in the risk range below 50% (Fig. 1).
Fig. 1.
Calibration plots for the DRE-based ERSPC RC
Net benefit
The DREvol-RC provided a net benefit over performing a prostate biopsy in every men and performing a prostate biopsy on the basis of PSA and DRE for threshold probabilities of approximately 10% or higher in the cohorts representing men screened for the first time (GOTEBORG and ROTTERDAM-R1) and the Tyrol-cohort (Fig. 2). Net benefit over performing a prostate biopsy in all men in GOTEBORG-R2–6 and ROTTERDAM-R2–3) was less but nevertheless present for risk thresholds between 25 and 40%. Ignoring information on prostate volume decreased net benefit slightly. The net benefit curve for CCF suffers from the observed underprediction, with a net benefit only for higher thresholds (>35%), while the net benefit curves on the Tyrol-cohort clearly show the additional value of (a) a multivariable approach (PSA + DRE) and (b) the inclusion of information on prostate volume.
Fig. 2.
Net benefit curves for the DRE-based ERSPC RC (large dashed black) versus the screening strategies of referring no men to biopsy (horizontal line at 0), referring all men to biopsy (solid black) and referring men on the basis of PSA + DRE (small dashed black)
Discussion
A key measure for a prediction model is its ability to distinguish those who will have a biopsy detectable PC from those who will not. In the ideal situation, this will lead to biopsy only in those men that indeed have PC and hence making the prostate biopsy a procedure to assess disease aggressiveness.
The recently developed DREvol-RC was previously validated in men screened at repeat screening rounds of ERSPC Rotterdam (4th and 5th screening round) [11], showing that volume estimation by DRE underestimated the TRUS-assessed prostate volume. However, the median values of the TRUS-assessed prostate volume (26.5, 45.6, and 69.3 cc) were close to the three predefined volume classes of 25, 40, and 60 cc, respectively. Since the sample size of the validation cohort in [11] was relatively small, additional studies are needed. This study performed a validation of this DREvol-RC in six different cohorts, both of screening and clinical origin. The AUCs covered a wide range (from 0.61 to 0.78), which reflects both the fit of the prediction model to the cohort and the varying case-mix. Higher AUC values will be found in more heterogeneous cohorts [16]. In all cohorts, AUC values were substantially larger than found for the model based on PSA and DRE alone (PSADRE-model). This was also true for the two largest validation cohorts, i.e., the Tyrol screening cohort and the clinical CCF cohort.
Better discrimination does not automatically imply better clinical usefulness. A better discriminatory model can suffer from major under or over prediction making it in fact a dangerous tool to use in clinical decision making. In the current study, the DREvol-RC was well calibrated with the exception of the CCF cohort where a systematic underprediction was observed. Baseline data of the CCF cohort did not indicate it as a high risk cohort. The cancer detection rate was however considerably higher (39%) as compared to the other cohorts. This most likely is the result of the biopsy procedure comprising of at least 10 cores or more while the ERSPC RCs were based on sextant biopsies. Moreover, men who were actually biopsied in the CCF cohort were preselected based on clinical judgment. Biopsy may have been avoided in men with elevated PSA levels due to the presence of BPH. This idea is supported by the larger net benefit when using the DREvol-RC in cohorts where men were biopsied purely based on an elevated PSA level. Application of the DREvol-RC in clinical cohorts hence needs further study, and a correction for the miscalibration might need to be made.
Similar analysis for the Prostate Cancer Prevention Trial (PCPT) RC [17, 18] (including PSA, DRE outcome, age, and family history) shows limited to no clinical benefit as compared to the policy of biopsying all men. These differences in predictive performance have been observed in previous studies [8, 9, 19]. Prostate volume was highly predictive in a screening-based validation cohort of 525 men [9]. While prostate volume is lacking in the PCPT RC, it is crucial within the ERSPC RCs. The predictors PSA and prostate volume play an opposing role [6, 7, 20]: high PSA level is positively related to PC at biopsy, but a large prostate volume has a negative relation. Indeed, prostate volume is related to serum PSA in men with Benign prostatic hyperplasia and no evidence of PC, and the relationship depends on age [21, 22]. In middle-aged men, BPH may hence explain a high PSA. Histologically, distinguishable BPH is present in about 8% of men aged 31–40 years, and this prevalence increases markedly with age to about 70% by the seventh decade of life [23].
In conclusion, identifying men at increased risk for having a biopsy detectable PCa should consider multiple factors. Prostate volume is a key element in such risk prediction, and approximate estimation through DRE might enable a more widespread use of this calculator.
Acknowledgments
Grants to support the work of the ERSPC include: European Union Grants SOC 95 35109, SOC 96 201869 05F022, SOC 97 201329, SOC 98 32241, the 6th Framework Program of the EU: PMark:LSHC-CT-2004-503011; The Dutch Cancer Society (KWF 94-869, 98-1657, 2002-277, 2006-3518); the Netherlands Organization for Health Research and Development (ZonMW-002822820, 22000106, 50-50110-98-311). Prostate Cancer Research Foundation of Rotterdam (SWOP); Beckman-Coulter-Hybritech Inc; Abbott Pharmaceuticals, Sweden; Af Jochnick’s foundation; Catarina and Sven Hagstroms family foundation; Gunvor and Ivan Svensson’s foundation; Johanniterorden, King Gustav V Jubilée Clinic Cancer Research Foundation; Sahlgrenska University Hospital; Schering Plough, Sweden, Swedish Cancer Society (Contract numbers 090107, 080315, and 083455); Wallac Oy, Turkku, Finland. Ewout Steyerberg was supported by a grant from the Center for Translational Molecular Medicine (PCMM project). Grants to support the Tyrol study: Supported by the International Agency for Research on Cancer, Lyon and the Tyrolean Prostate Cancer Early Detection Group. The SABOR project is supported by the San Antonio Center of Biomarkers of Risk for Prostate Cancer U01 CA86402. Statistical center supported in part by funds from David H. Koch provided through the Prostate Cancer Foundation, the Sidney Kimmel Center for Prostate and Urologic Cancers and P50-CA92629 SPORE grant from the National Cancer Institute to Dr. P. T. Scardino.
Conflict of interest
No conflict of interest declared.
Open Access
This article is distributed under the terms of the Creative Commons Attribution Noncommercial License which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
Appendix: Formulas used to calculate volume classes and the DRE ERSPC RC risks [11]
Volume classes:
if (TRUS volume < 30.0) volumeclasses = 25.
if (TRUS volume ≥ 30.0 and TRUS volume < 50) volumeclasses = 40.
if (TRUS volume ≥ 50) volumeclasses = 60.
Model on the basis of PSA and DRE for men not previously biopsied:
If (priorscreening = 0) lpPSADREriskcalc3 = −1.743 + 0.788 × (log2 (PSA1) − 2.0) + 1.105 × (DRE1).
Model on the basis of PSA and DRE for men previously biopsied:
If (priorscreening = 1) lpPSADREriskcalc45 = −1.315 − 0.899 × prevbiopsy + (0.421 − 0.362 × prevbiopsy) × (log2(PSA) − 2) + (0.726 × DRE).
DRE-based volume ERSPC Riskcalculator for men not previously screened (DREvol - RC #3):
if (priorscreening = 0) lpDRE_ERSPC_riskcalc = −1.826 + 1.024 × (log2(PSA) − 2.0) − 1.50 × (log2(volumeclasses)/lg10(2)) − 5.4) + 0.992 × (DRE).
DRE ERSPC Riskcalculator for men previously screened and/or biopsied (DREvol - RC #45)
if (priorscreening > 0) lpDRE_ERSPC_riskcalc = −1.470 − 0.677 × priorbiopsy + (0.576 − 0.423 × priorbiopsy) × (log2(PSA) − 2) − 1.043 × (log2(volumeclasses) − 5.5) + (0.68 × DRE).
Log2 denotes the 2 log, which can also be calculated as log(x)/log(2), with x = PSA (in ng/ml) or x = volume in 3 classes (25, 40, or 60). DRE denotes a positive DRE examination (coded 0/1), prevbiosy denotes a previous biopsy (coded 0/1).
References
- 1.Andriole GL, Crawford ED, Grubb RL, III, Buys SS, Chia D, et al. Mortality results from a randomized prostate-cancer screening trial. N Engl J Med. 2009;360:1310–1319. doi: 10.1056/NEJMoa0810696. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2.Schröder FH, Hugosson J, Roobol MJ, Tammela TL, Ciatto S, et al. Screening and prostate-cancer mortality in a randomized European study. N Engl J Med. 2009;360:1320–1328. doi: 10.1056/NEJMoa0810084. [DOI] [PubMed] [Google Scholar]
- 3.Hugosson J, Carlsson S, Aus G, Bergdahl S, Khatami A, et al. Mortality results from the Goteborg randomised population-based prostate-cancer screening trial. Lancet Oncol. 2010;11:725–732. doi: 10.1016/S1470-2045(10)70146-7. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4.Thompson IM, Ankerst DP, Chi C, Lucia MS, Goodman PJ, Crowley JJ, Parnes HL, Coltman CA., Jr Operating characteristics of prostate-specific antigen in men with an initial PSA level of 3.0 ng/ml or lower. JAMA. 2005;294(1):66–70. doi: 10.1001/jama.294.1.66. [DOI] [PubMed] [Google Scholar]
- 5.Schröder F, Kattan MW (2008) The comparability of models for predicting the risk of a positive prostate biopsy with prostate-specific antigen alone: a systematic review. Eur Urol 54(2):274–290. (Epub 2008 May 22. Review) [DOI] [PubMed]
- 6.Kranse R, Roobol M, Schröder FH. A graphical device to represent the outcomes of a logistic regression analysis. Prostate. 2008;68(15):1674–1680. doi: 10.1002/pros.20840. [DOI] [PubMed] [Google Scholar]
- 7.Steyerberg EW, Roobol MJ, Kattan MW, van der Kwast TH, de Koning HJ, Schröder FH. Prediction of indolent prostate cancer: validation and updating of a prognostic nomogram. J Urol. 2007;177(1):107–112. doi: 10.1016/j.juro.2006.08.068. [DOI] [PubMed] [Google Scholar]
- 8.Trottier G, Roobol MJ, Lawrentschuk N et al (2011) Comparison of risk calculators from the Prostate Cancer Prevention Trial and the European Randomized Study of Screening for Prostate Cancer in a contemporary Canadian cohort. BJU Int 108(8B):E237–E244 [DOI] [PubMed]
- 9.Cavadas V, Osorio L, Sabell F, et al. Prostate cancer prevention trial and European Randomized Study of Screening for Prostate Cancer risk calculators: a performance comparison in a contemporary screened cohort. Eur Urol. 2010;58(4):551–558. doi: 10.1016/j.eururo.2010.06.023. [DOI] [PubMed] [Google Scholar]
- 10.Dong F, Kattan MW, Steyerberg EW, Jones JS, Stephenson AJ, Schröder FH, Klein EA. Validation of pretreatment nomograms for predicting indolent prostate cancer: efficacy in contemporary urological practice. J Urol. 2008;180(1):150–154. doi: 10.1016/j.juro.2008.03.053. [DOI] [PubMed] [Google Scholar]
- 11.Roobol MJ, van Vugt HA, Loeb S, Zhu X, Bul M, Bangma CH, van Leenders AG, Steyerberg EW, Schröder FH (2011) Prediction of prostate cancer risk: the role of prostate volume and digital rectal examination in the ERSPC risk calculators. Eur Urol [Epub ahead of print] [DOI] [PubMed]
- 12.Vickers AJ, Cronin AM, Roobol MJ, Hugosson J, Jones JS, Kattan MW, Klein E, Hamdy F, Neal D, Donovan J, Parekh DJ, Ankerst D, Bartsch G, Klocker H, Horninger W, Benchikh A, Salama G, Villers A, Freedland SJ, Moreira DM, Schroeder FH, Lilja H. The relationship between prostate-specific antigen and prostate cancer risk: the Prostate Biopsy Collaborative Group. Clin Cancer Res. 2010;16:4374–4381. doi: 10.1158/1078-0432.CCR-10-1328. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Steyerberg EW. Clinical prediction models: a practical approach to development, validation, and updating. New York: Springer; 2009. [Google Scholar]
- 14.Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, Kattan MW, Pencinac MJ. Assessing the performance of prediction models: a framework forctraditional and novel measures. Epidemiology. 2010;21(1):128–138. doi: 10.1097/EDE.0b013e3181c30fb2. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–574. doi: 10.1177/0272989X06295361. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16.Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol. 2010;172(8):971–980. doi: 10.1093/aje/kwq223. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17.Ankerst DP, Boeck A, Thompson IM, Cronin AM, Roobol MJ, Hugosson J, Jones S, Kattan MW et al. (in press) Mass validation of the Prostate Cancer Prevention Trial risk calculator: results from the Prostate Biopsy Collaborative Group. WJU [DOI] [PMC free article] [PubMed]
- 18.Thompson IM, Ankerst DP, Chi C, Goodman PJ, Tangen CM, Lucia MS, Feng Z, Parnes HL, Coltman CA., Jr Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst. 2006;98:529–534. doi: 10.1093/jnci/djj131. [DOI] [PubMed] [Google Scholar]
- 19.van den Bergh RC, Roobol MJ, Wolters T, van Leeuwen PJ, Schröder FH. The Prostate Cancer Prevention Trial and European Randomized Study of Screening for Prostate Cancer risk calculators indicating a positive prostate biopsy: a comparison. BJU Int. 2008;102(9):1068–1073. doi: 10.1111/j.1464-410X.2008.07940.x. [DOI] [PubMed] [Google Scholar]
- 20.Roobol MJ, Schröder FH, Kranse R, ERSPC, Rotterdam A comparison of first and repeat (four years later) prostate cancer screening in a randomized cohort of a symptomatic men aged 55–75 years using a biopsy indication of 3.0 ng/ml (results of ERSPC, Rotterdam) Prostate. 2006;66(6):604–612. doi: 10.1002/pros.20352. [DOI] [PubMed] [Google Scholar]
- 21.Roehrborn CG, Boyle P, Gould AL, Waldstreicher J. Serum prostate-specific antigen as a predictor of prostate volume in men with benign prostatic hyperplasia. Urology. 1999;53(3):581–589. doi: 10.1016/S0090-4295(98)00655-4. [DOI] [PubMed] [Google Scholar]
- 22.Chung BH, Hong SJ, Cho JS, Seong DH. Relationship between serum prostate-specific antigen and prostate volume in Korean men with benign prostatic hyperplasia: a multicentre study. BJU Int. 2006;97(4):742–746. doi: 10.1111/j.1464-410X.2006.06016.x. [DOI] [PubMed] [Google Scholar]
- 23.Berry SJ, Coffey DS, Walsh PC, Ewing LL. The development of human benign prostatic hyperplasia with age. J Urol. 1984;132:474–479. doi: 10.1016/s0022-5347(17)49698-4. [DOI] [PubMed] [Google Scholar]