Properties of the four kallikrein panel outside the diagnostic grey zone: meta-analysis of patients with positive digital rectal exam or prostate-specific antigen 10 ng / mL and above

Abstract

Purpose

The four-kallikrein panel, commercially available as the 4Kscore™, is a reflex test for prostate cancer early detection that has been extensively validated in multiple international cohorts. It has been suggested that use of such reflex tests be limited to those with PSA less than 10 ng/mL and negative DRE. We aimed to determine the value of the panel in men outside this “diagnostic grey zone”.

Methods

We conducted an individual patient data meta-analysis using data from prior studies on the four-kallikrein panel. We calculated the properties of the panel for predicting high-grade (Gleason 7+) cancer in a subgroup of men with either positive DRE or PSA 10 – 25 ng/mL.

Results

A total 2,891 men from 8 cohorts were included. An important proportion of patients – 32% in the US validation study - had either a PSA 10 – 25 ng/mL or a positive DRE. For men with PSA of 10–25 ng/ml, the fixed–effects estimate for the discrimination of the kallikrein model was 0.84 versus. 0.69 for the base model (difference: 0.128; 95% C.I. 0.098, 0.159); for the positive DRE group, discrimination was 0.82 versus 0.72 (difference: 0.092; 95% C.I. 0.069, 0.115). Decision analysis showed clinical net benefit for use of the panel in this subgroup, with a reduction in biopsy rates of about 20% and only a small number of high-grade cancers missed, less than 3% of those not biopsied.

Conclusion

The use of the kallikrein panel in men with positive DRE or PSA 10 – 25 ng/mL is justified.

Introduction

It has been estimated that each year approximately 30 million US men undergo measurement of prostate-specific antigen (PSA) for prostate cancer early detection¹. Although men with low PSA frequently do harbor prostate cancer, such men have an extremely low long-term risk of metastatic disease^{2, 3}. On the other hand, most men with modestly elevated PSA do not have prostate cancer⁴. Efforts to improve on the PSA test have therefore focused on increasing specificity rather than sensitivity. As a result, novel prostate screening tests are typically developed as “reflex tests” to be used when PSA is elevated.

One such test is the four-kallikrein panel, now commercially available as the 4Kscore™. This is a statistical model that includes blood levels of three PSA isoforms (total, free and intact PSA), plus an additional kallikrein marker (hK2), and clinical data on age, digital rectal exam (DRE) status and history of prior negative biopsy. The panel was initially validated in a series of studies on cohorts from the European Randomized trial of Screening for Prostate Cancer (ERSPC)^5-11. A subsequent statistical model was developed for use on plasma samples, with an initial study on the ProtecT trial¹², subsequently validated in both Swedish¹³ and US⁴ clinical cohorts. In all studies, the area-under-the-curve (AUC) for discrimination of high-grade prostate cancer on biopsy was close to or in excess of 0.80. Improvement in AUC compared to models including clinical data alone, such as PSA and DRE, was typically in the order of 0.08.

Several prostate cancer markers are intended for use in what has been termed the “diagnostic grey zone”, men with PSA less than 10 ng / mL and negative DRE. An obvious example is the Prostate Health Index, which has approval from the Food and Drug Administration only within the grey zone¹⁴. Previously reported studies of the four kallikrein panel have, by contrast, included men with positive DRE and PSA > 10 ng / mL. The favorable result of these studies might be seen as justifying the use of the panel outside the diagnostic grey zone. However, it is perfectly possible for the panel have poor properties in this small group of men, but for the average, across all men, to indicate good diagnostic performance. In this paper, we conduct an individual patient data meta-analysis using data from prior studies of the kallikrein panel to determine its value for prediction of high grade prostate cancer in men with a positive DRE or PSA ≥ 10 ng / mL.

Methods

Although a meta-analysis typically starts with a literature review, the kallikrein assays were developed and then patented by the senior author, and the statistical algorithm developed by the first author. Studies on the panel are not practical without the involvement of the investigators – to advise on assays and algorithms – hence all relevant studies are personally known to the investigators and no literature search was indicated. Studies were included if the kallikrein panel was measured on men undergoing biopsy for prostate cancer, where the panel was measured blindly and all men were biopsied irrespective of kallikrein results. Raw data were obtained on age, PSA, DRE, stage, prior biopsy status, biopsy outcome, biopsy grade and kallikrein marker levels. DRE was defined as in the original study, typically, the presence of nodularity. Data were received from the ProtecT¹², Rotterdam ERSPC^{5, 8}, Rotterdam Repeat Biopsy¹⁰, UPCA¹³, Tarn ERSPC⁹, Göteborg ERSPC^{6, 7} and OPKO cohorts⁴. In the ProtecT cohort, some men had measurements from both plasma and serum PSA samples, in which case the plasma measurements were used. We did not obtain permission to use data from the Stockholm 2 cohort in this analysis¹⁵. One study in which the kallikrein panel was evaluated¹¹ was not included due to very small sample size (only 9 total high grade cancers).

Patients who had PSA levels higher than 25 ng/ml were excluded from all analyses, as these patients are given a uniformly high risk by the kallikrein panel. The models were assessed in two separate cohorts: men with high PSA, defined as a PSA ≥10 and ≤25 ng/ml, and men with a positive DRE. The ProtecT cohort was not included in the latter analysis as DRE results were not available.

A base model and a kallikrein model including the panel of four kallikrein markers were used to generate predictions for the risk of high grade cancer for each patient. The four kallikrein and base models used for these predictions were the same as those used in the original study publications. Further details of the exact models used for each cohort are given in the supplementary material. The following cohorts represented independent validations of a previously specified statistical model: Göteborg previously screened, OPKO, Rotterdam, Rotterdam repeat biopsy, Tarn, UPCA. The OPKO and UPCA cohorts constituted independent validation for the kallikrein statistical model currently used in contemporary practice.

For each cohort, we assessed discrimination by calculating the area under the curve (AUC). The AUC and the standard error of the AUC were meta-analyzed across cohorts and the fixed-effects and random-effects estimates reported. The difference in AUC between the base model and kallikrein model was calculated for each cohort, with the standard error of this difference estimated by bootstrap methods. The change in AUC with the addition of the kallikrein markers and the bootstrapped standard errors were then entered into a meta-analysis. Clinical impact was assessed by calculating the number of men at low risk and reporting the number of these low-risk men who were found to have high grade disease (i.e. false positives). Low risk was defined by having a predicted probability of high-grade disease that was less than the pre-specified thresholds of 7.5% or 10%. These results were then converted into net benefit^{16, 17} and compared to the default strategy of biopsying all men at risk. These analyses were restricted to two cohorts that used contemporary biopsy schemes and in which a prespecified statistical model was applied, that is, independent validation studies (OPKO and UPCA).

We also conducted several sensitivity analyses. First, the analyses were repeated for the outcome of any grade cancer in men with high PSA. This is on the grounds that, in some risk stratification schemes, a patient with low-grade cancer is not considered low risk if PSA>10 ng/mL. Note that we used the same models as for high-grade cancer and assessed discrimination, but not calibration or clinical utility, as the models are not calibrated for the endpoint of any cancer. Analyses were also performed that assessed serum and plasma model predictions separately, that assessed Rotterdam rounds 1, 2 and 3 as separate cohorts, and that assessed model predictions from models with and without DRE separately. A sensitivity analysis was performed that included only the cohorts that were independent validations. Among men with positive DREs, a sensitivity analysis was performed that included only men with PSA < 10 ng/ml. All analyses were conducted using Stata 13 (Stata Corporation, College Station, TX).

Results

A total of 1,198 men from eight cohorts were included in the individual patient data meta-analysis of men with a PSA of 10-25 ng/ml. There were 1,835 men from seven cohorts with positive DRE. Patient and disease characteristics for these patients are presented in Table 1; supplemental tables 1 and 2 show characteristics separately by cohort. Cohort characteristics were similar, with the exception that, as expected, contemporary cohorts had higher prevalence of Gleason 7+ disease due to extended biopsy and changes in grading practice. Table 2 presents the AUCs and overall meta-analytic estimates among men with a PSA of 10-25 ng/mL and, separately, men with positive DRE for the base models and kallikrein models.

Table 1.

Patient and disease characteristics for men with PSA 10-25 and for men with positive DRE. Data are presented as median (IQR) or frequency (%).

	PSA 10–25 ng/ml (N=1198)	Positive DRE (N=1835)
Age at biopsy	65 (61, 69)	66 (62, 70)
Total PSA	12.9 (11.2, 16.0)	4.8 (3.6, 7.1)
Prior negative biopsy	133 (11%)	248 (14%)
DRE result
Normal	485 (40%)	0 (0%)
Abnormal	272 (23%)	1835 (100%)
Unknown	441 (37%)	0 (0%)
Biopsy outcome
Negative	564 (47%)	1048 (57%)
Low grade	266 (22%)	427 (23%)
High grade	368 (31%)	360 (20%)
Clinical stage	N=634	N=787
T1	232 (37%)	29 (3.7%)
T2	206 (32%)	597 (76%)
T3	110 (17%)	138 (18%)
T4	3 (0.5%)	4 (0.5%)
Unknown	83 (13%)	19 (2.4%)

Table 2.

AUCs with standard errors for base models and kallikrein panel models for high grade cancer in men with PSA 10-25 and in men with positive DRE, by cohort.

	PSA 10 – 25 ng/ml			Positive DRE
Cohort	Base model	Kallikrein panel	Difference (95% CI)	Base model	Kallikrein panel	Difference (95% CI)
Göteborg Unscreened7	0.765 (SE 0.088)	0.835 (SE 0.056)	0.070 (−0.043, 0.184)	0.719 (SE 0.062)	0.762 (SE 0.055)	0.043 (−0.051, 0.136)
Göteborg Previously Screened*6	Excluded	Excluded	Excluded	0.671 (SE 0.075)	0.747 (SE 0.070)	0.076 (−0.028, 0.181)
OPKO*4	0.627 (SE 0.057)	0.871 (SE 0.037)	0.245 (0.133, 0.356)	0.733 (SE 0.035)	0.855 (SE 0.026)	0.122 (0.066, 0.177)
ProtecT12	0.652 (SE 0.026)	0.778 (SE 0.021)	0.126 (0.074, 0.178)	Excluded	Excluded	Excluded
Rotterdam*5, 8	0.760 (SE 0.034)	0.881 (SE 0.023)	0.121 (0.068, 0.174)	0.751 (SE 0.022)	0.832 (SE 0.019)	0.081 (0.049, 0.113)
Rotterdam Repeat Biopsy*10	0.615 (SE 0.115)	0.882 (SE 0.044)	0.267 (0.099, 0.436)	0.676 (SE 0.083)	0.833 (SE 0.059)	0.157 (0.028, 0.285)
Tarn*9	0.795 (SE 0.119)	0.955 (SE 0.052)	0.159 (−0.082, 0.400)	0.589 (SE 0.067)	0.763 (SE 0.054)	0.175 (0.075, 0.274)
UPCA*13	0.702 (SE 0.046)	0.791 (SE 0.037)	0.089 (0.011, 0.166)	0.703 (SE 0.040)	0.776 (SE 0.034)	0.073 (0.009, 0.137)
Fixed effects meta-analytic Estimate (95% CI)	0.690 (0.657, 0.724)	0.838 (0.814, 0.861)	0.128 (0.098, 0.159)	0.724 (0.694, 0.754)	0.820 (0.795, 0.844)	0.092 (0.069, 0.115)
Random effects meta-analytic Estimate (95% CI)	0.696 (0.646, 0.745)	0.851 (0.803, 0.898)	0.133 (0.092, 0.174)	0.720 (0.686, 0.753)	0.815 (0.786, 0.845)	0.094 (0.068, 0.119)
Heterogeneity p value	p=0.15	p=0.003	p=0.2	p=0.4	p=0.3	p=0.4

^*Independent validation.

For men with PSA 10 - 25 ng/mL, base model AUCs ranged from 0.62 to 0.80, with an overall fixed-effects estimate of 0.69 (95% CI 0.66, 0.72). AUCs for the kallikrein panel were 0.78 to 0.96 with an overall estimate of 0.84 (95% CI 0.81, 0.86). The meta-analytic estimate for the increase in AUC associated with the panel is 0.128 (95% C.I. 0.098, 0.159). The smallest increment in AUC for any study was 0.07. Hence the kallikrein panel clearly retains higher discriminative accuracy in these cohorts of men with PSA of 10-25 ng/mL compared to the base model.

Among men with a positive DRE, the discrimination of the base models was slightly higher than among men with PSA of 10-25 ng/mL, although kallikrein model discrimination was similar. Base model discrimination across cohorts was between 0.59 and 0.75, with an overall fixed-effects estimate of 0.72 (95% CI 0.69, 0.75). Discrimination of the kallikrein model ranged from 0.75 to 0.86. The overall meta-analytic estimate for these models was 0.82 (95% CI 0.80, 0.84) with an estimate for the increase in AUC of 0.092 (95% C.I. 0.069, 0.115). Again, it is clear that the kallikrein panel retains higher discriminatory accuracy than the base model in men with positive DRE.

Heterogeneity statistics are reported in table 2. It can be seen that although there is evidence that model discrimination varies between studies, there is no significant heterogeneity for the increment in discrimination afforded by the panel.

To assess the clinical impact of using the kallikrein panel outside the diagnostic grey zone, we calculated the number of men who would be advised against biopsy on the basis of low risk from the kallikrein panel, and of those, what proportion were found to have high grade cancer at biopsy (Table 3). These analyses were only conducted in the OPKO and UPCA cohorts, as these were independent validation studies of the contemporary 4Kscore model. Figures 1 and 2 show the net benefit in men with either PSA 10-25 ng/ml or positive DRE across a wide range of threshold probabilities, with the kallikrein panel having a higher net benefit than biopsying all men at threshold probabilities greater than a 5% risk of high-grade cancer.

Table 3.

Prevalence of high grade cancer in low risk patients with PSA between 10 and 25 ng/ml, by cohort. Cohorts were included if they were independent validations of the contemporary 4Kscore model.

Cohort	Low Risk Men	High Grade Cancer Among Low Risk Men
PSA 10 – 25 ng / mL (n=255)
<7.5% risk threshold
OPKO (N=102)	22 (22%)	1 (4.5%)
UPCA (N=153)	12 (8%)	0 (0.0%)
Total	34 (13%)	1 (2.9%)
<10% risk threshold
OPKO (N=102)	24 (24%)	1 (4.2%)
UPCA (N=153)	16 (10%)	0 (0.0%)
Total	40 (16%)	1 (2.5%)
Positive DRE (n=449)
<7.5% risk threshold
OPKO (N=240)	61 (25%)	0 (0.0%)
UPCA(N=209)	6 (3%)	1 (16.7%)
Total	67 (15%)	1 (1.5%)
<10% risk threshold
OPKO (N=240)	81 (34%)	1 (1.2%)
UPCA (N=209)	12 (6%)	1 (8.3%)
Total	93 (21%)	2 (2.2%)
PSA 10 – 25 ng / mL or positive DRE (n=615)
Total: <7.5% threshold
OPKO (N=321)	81 (25%)	1 (1.2%)
UPCA (N=294)	17 (6%)	1 (5.9%)
Total	98 (16%)	2 (2.0%)
Total: <10% threshold
OPKO (N=321)	103 (32%)	2 (1.9%)
UPCA (N=294)	26 (9%)	1 (3.8%)
Total	129 (21%)	3 (2.3%)

Figure 1.

Decision curve analysis in independent cohorts using the contemporary statistical model to predict high-grade disease in men with PSA 10-25 ng/ml or positive DREs. Red line: Biopsy in no men. Blue line: Biopsy in all men. Green Line: Biopsy according to 4Kscore.

Figure 2.

Net biopsies avoided in independent cohorts using the contemporary statistical model to predict high-grade disease in men with PSA 10-25 ng/ml or positive DREs.

Several sensitivity analyses were conducted. A sensitivity analysis for the outcome of any grade cancer found that discrimination was similar for any grade and high grade cancers among men with PSA 10-25. The overall fixed-effects estimate was 0.66 (95% CI 0.63, 0.69) for base models and 0.82 (95% CI 0.79, 0.84) for kallikrein models (Table 4). Heterogeneity was again significant for the absolute levels of discrimination of the kallikrein and base models, but not significant for the increment in AUC. Models without DRE were analyzed separately from models with DRE, with no significant differences found. Plasma cohorts and serum cohorts were compared separately, with similar results. Findings were also consistent when the Rotterdam cohort was analyzed as per our prior papers, that is, unscreened men⁸ separately from those with prior screening⁵. Results were also virtually unchanged in the analysis restricted to independent validation sets. For men with PSA 10-25 ng/ml, discrimination was similar for both the base model (0.72 vs 0.69 in all cohorts) and the kallikrein model (0.87 vs 0.84 in all cohorts). Among men with positive DRE in these six cohorts, AUCs were similar for both the base models (0.72 for both) and the kallikrein models (0.82 for both).

Table 4.

AUCs with standard errors for base models and kallikrein panel models for any grade cancer in men with PSA 10-25, by cohort.

	PSA 10 – 25 ng/ml
Cohort	Base model	Kallikrein panel	Difference (95% CI)
Göteborg Unscreened7	0.725 (SE 0.068)	0.876 (SE 0.044)	0.151 (0.025, 0.277)
Göteborg Previously Screened*6	0.667 (SE 0.333)	0.972 (SE 0.039)	0.306 (−0.236, 0.847)
OPKO*4	0.578 (SE 0.059)	0.833 (SE 0.040)	0.255 (0.145, 0.365)
ProtecT12	0.628 (SE 0.025)	0.741 (SE 0.023)	0.113 (0.060, 0.166)
Rotterdam*5, 8	0.746 (SE 0.031)	0.843 (SE 0.026)	0.097 (0.044, 0.150)
Rotterdam Repeat Biopsy*10	0.543 (SE 0.077)	0.776 (SE 0.061)	0.232 (0.107, 0.357)
Tarn*9	0.714 (SE 0.155)	0.946 (SE 0.053)	0.232 (−0.086, 0.550)
UPCA*13	0.596 (SE 0.046)	0.677 (SE 0.043)	0.081 (0.005, 0.158)
Fixed effects meta-analytic Estimate (95% CI)	0.655 (0.623, 0.687)	0.815 (0.791, 0.840)	0.124 (0.094, 0.155)
Random effects meta-analytic Estimate (95% CI)	0.647 (0.587, 0.706)	0.832 (0.765, 0.899)	0.139 (0.093, 0.184)
Heterogeneity p value	p=0.022	p<0.0001	p=0.11

^*Independent validation.

There were 1,588 patients included in the sensitivity analysis restricted to men with positive DRE and PSA < 10 ng/ml. Discrimination was slightly lower for both the base and kallikrein panel models when compared to all men with positive DRE. Base model discrimination ranged from 0.57 to 0.69. The overall fixed-effects estimate was 0.66 (95% CI 0.62, 0.69). Kallikrein model discrimination across the seven cohorts was between 0.70 and 0.82, with an overall estimate of 0.78 (95% CI 0.75, 0.81). Similar to the main analyses, net benefit for the model was higher than the net benefit for biopsying all at thresholds of both 7.5% (0.177 vs 0.165) and 10% (0.163 vs 0.142).

Discussion

We found that the four kallikrein panel contributed importantly enhanced discrimination in men outside the diagnostic grey zone. For men with PSA ≥10 and ≤25 ng / mL, or with positive DRE, the panel of four kallikrein markers was associated with an increase in AUC of 0.09 to 0.13 compared to base models that included PSA but not the additional kallikrein markers. We also demonstrated that use of the panel had clinical net benefit, with the number of missed high-grade cancers being too small to offset larger decreases in biopsy rates. The proportion of men outside the diagnostic grey zone to whom the panel could be applied is non trivial. In the US validation study⁴, for instance, about 1 in 4 men had PSA between 10 – 25 ng/mL and fully 1 in 3 had either a PSA 10 – 25 ng / mL or a positive DRE.

These results have clear clinical implications. It is normally assumed that reflex tests such as the four kallikrein panel should only be used in the diagnostic grey zone of PSA 3 – 10 ng / mL and negative DRE¹⁴. We found that use of the kallikrein panel outside the diagnostic grey zone will reduce number of biopsies without delaying the diagnosis of an undue number of high-grade cancers. Our findings reflect the study of Lazzeri et al., who found that the Prostate Health Index retained discriminative accuracy in men with PSA ≥10 ng / mL (AUC of 0.81). In contrast to our results, however, the Prostate Health Index was not favored in a decision analysis.

That said, there remains the question of whether urologists would feel comfortable recommending against biopsy in a man with a high PSA or abnormal DRE. For instance, take the following three actual patients: a 65 year old man with prior negative biopsy, negative DRE and a PSA of 20 ng/mL; a 51 year old without prior biopsy, negative DRE and a PSA of 13 ng/ml; a 66-year old, prior negative biopsy, positive DRE and PSA of 11 ng/ml. The risks from the panel for these three men were 3.0%, 5.9% and 6.1% and none had cancer. But one might reasonably hypothesize that most urologists would insist on biopsy for a PSA of 20 ng/ml, a PSA of 13 ng/ml in a young man or a patient with both a positive DRE and a PSA > 10 ng/ml.

Konety et al. conducted an “impact study” on the 4Kscore involving 611 patients seen by 35 academic and community urologists who ordered a 4Kscore as part of their routine clinical practice¹⁸. Use of the 4Kscore led to a 65% reduction in biopsy rates, with the probability of biopsy being strongly dependent on the 4Kscore result. Although PSA levels were not reported, it is of note that rates of positive DRE were very low (6%), suggesting that men with positive DRE were recommended for biopsy without further reflex testing. A subsequent impact study focusing on higher risk patients is warranted.

Conclusions

An important proportion number of men presenting for biopsy falls outside the diagnostic grey zone, having a positive DRE or PSA 10 – 25 ng / mL. The four kallikrein panel had good discrimination in these men, and use of the panel reduced biopsy rates in this group of men by over 20%. The use of the panel in men with positive DRE or PSA 10 – 25 is justified.

List of Abbreviations

PSA

prostate-specific antigen

DRE

digital rectal exam

ERSPC

European Randomized trial of Screening for Prostate Cancer

AUC

area-under-the-curve