Validation of a bone scan positivity risk table in non-metastatic castration-resistant prostate cancer

Abstract

Objectives

To test the external validity of a previously developed risk table, designed to predict the probability of a positive bone scan among men with non-metastatic (M0) castration-resistant prostate cancer (CRPC), in a separate cohort.

Patients and Methods

We retrospectively analysed 429 bone scans of 281 patients with CRPC, with no known previous metastases, treated at three Veterans Affairs Medical Centers. We assessed the predictors of a positive scan using generalized estimating equations. Area under the curve (AUC), calibration plots and decision-curve analysis were used to assess the performance of our previous model to predict a positive scan in the current data.

Results

A total of 113 scans (26%) were positive. On multivariable analysis, the only significant predictors of a positive scan were log-transformed prostate-specific antigen (PSA): hazard ratio (HR) 2.13; 95% confidence interval (CI) 1.71–2.66 (P < 0.001) and log-transformed PSA doubling time (PSADT): HR 0.53; 95% CI 0.41–0.68 (P < 0.001). Among men with a PSA level <5 ng/mL, the rate of positive scans was 5%. The previously developed risk table had an AUC of 0.735 to predict positive bone scan with excellent calibration, and provided additional net benefit in the decision-curve analysis.

Conclusion

We have validated our previously developed table to predict the risk of a positive bone scan among men with M0/Mx CRPC. Use of this risk table may allow better tailoring of patients’ scanning to identify metastases early, while minimizing over-imaging. Regardless of PSADT, positive bone scans were rare in men with a PSA <5 ng/mL.

Introduction

Despite early detection and aggressive treatments, prostate cancer remains the second most lethal cancer in men in the USA [1]. Most men present with early-stage disease, potentially amenable to curative therapy [2]; however, despite localized treatment, many men have a rising PSA recurrence [3]. For men with rising PSA-only recurrence, once local salvage options, if applicable, are exhausted, no therapy has been shown to improve outcomes and often androgen deprivation therapy (ADT) is prescribed before metastases. Alternatively, men present with locally advanced, non-metastatic (M0) disease and receive primary ADT. Thus, many men receive ADT before developing metastases. When men treated with ADT without metastases develop PSA progression whilst receiving ADT, the tumour is considered castration-resistant prostate cancer (CRPC) but, in many cases, remains undetectable by imaging and is considered non-metastatic (i.e. M0 CRPC). These men represent a clinical conundrum in that no therapy has been shown to improve survival and current AUA CRPC guidelines recommend observation [4]. Once metastases are detected, however, multiple life-prolonging options exist. The hypothesis (still as yet untested) that earlier treatment with life-prolonging therapy is better than later, means that metastases should be detected as soon as possible to allow earlier initiation of life-prolonging therapies.

The most common method of detecting prostate cancer metastases is still a bone scan; however, most bone scans are negative [5–7]. This also applies to men with M0 CRPC [8]. Given that bone scans are costly and create potential anxiety, it would be ideal to select only high-risk patients for imaging. To date, limited data exist for identifying who men are at greatest risk of a positive bone scan in this population. To address this gap, we recently analysed data from 312 M0/Mx CRPC men from two Veterans Affairs (VA) hospitals [9]. We found that both PSA and PSA doubling time (PSADT) were the only significant predictors of whether a given bone scan would show metastases [9]. We developed a table, the Moreira table, to predict the risk of a positive bone scan based on PSA and PSADT at the time of bone scan imaging, which had an area under the curve (AUC) of 0.773 (Table 1). In the present study, we tested the external validity of the Moreira table to predict bone scan positivity using data from a separate cohort of men diagnosed with M0/Mx CRPC within the VA system.

Table 1.

Predicted risk of positive scan by PSA and PSA doubling time group.

PSADT	PSA
	<5 ng/mL	5–14.9 ng/mL	15–49.9 ng/mL	≥50 ng/mL
≥15 months	6 (4–8)	11 (9–14)	22 (18–28)	47 (40–54)
9–14.9 months	6 (4–10)	12 (10–14)	24 (22–26)	49 (46–52)
3–8.9 months	8 (5–14)	16 (13–18)	30 (27–33)	57 (53–60)
<3 months	12 (8–19)	22 (19–25)	40 (37–42)	67 (64–69)

Cells represent the average estimate (95% CIs in parenthesis). Reproduced with permission from Moreira et al. [9].

Methods

Study Design

After receiving approval from the institutional review board, we reviewed patients at three VA Medical Centers (Augusta, GA, San Francisco, CA and West Los Angeles, CA, USA). Using an automated query, we identified 1 609 men who received at least one dose of ADT (either gonadotropin-releasing hormone agonist, antagonist or bilateral orchiectomy) and subsequently had a PSA increase >2 ng/mL and >25% higher than the post-ADT nadir (Fig. 1). From these patients, we manually screened medical records to select patients who had documented CRPC defined by the PC Working Group 2: ≥25% PSA increase and an absolute increase ≥2 ng/mL from the post-ADT nadir while receiving continuous ADT [10]. We excluded patients with documented metastatic disease at or before CRPC diagnosis, leaving 569 patients with M0/Mx (non-metastatic) CRPC. Because of the availability of electronic medical records and changing treatment practices, we restricted analyses to those diagnosed with M0/Mx CRPC in the year 2000 or later (n = 542). Our cohort was limited to the 281 patients who had at least one bone scan after M0/Mx CRPC diagnosis. One patient was missing race and was excluded from the multivariable analysis.

Fig. 1.

Patient Consort diagram. ADT, androgen deprivation therapy; CRPC, castration-resistant prostate cancer; PSA, prostate-specific antigen; M0/Mx, non-metastatic.

Data were collected on patient demographic, clinical and pathological characteristics. The number and interval of bone scans as well as primary and secondary prostate cancer treatments were at the discretion of the patient and treating physician. Technetium99-bone scans were read by nuclear medicine radiologists. Radiologists were not blinded to patients’ demographics or laboratory, radiological or pathological results. Bone scans were coded by trained personnel as positive or negative, based on the radiology report (equivocal scans, because they usually do not prompt a change in management, were considered negative unless confirmed positive by a biopsy or another imaging test). Patients were followed up to their first imaging test that was positive for prostate cancer metastases, bone scan or otherwise. Once a patient was documented as having metastases, no further bone scans were evaluated. No patient received bone imaging with sodium fluoride postitron emission tomography/CT scan.

Statistical Analysis

We calculated PSADT as the natural log of two, divided by the slope of the linear regression of the natural log of PSA over time in months [11]. We included all PSA values from CRPC or 2 years before the scan (whichever was closer to the scan) up until the time of the scan. Patients were required to have ≥2 PSA measurements over at least 3 months. Patients with PSADT >120 months or declining PSA levels were assigned 120 months for ease of analysis.

Patient characteristics at baseline (time of CRPC diagnosis) were summarized using frequency and percent for categorical variables and median and 25th and 75th percentile for continuous variables. Characteristics were compared between positive and negative bone scans. P values were calculated using generalized estimating equations, as described below, to account for multiple bone scans among patients.

Because of repeated measures (some patients had more than one bone scan), we used generalized estimating equations with a logit link and exchangeable working correlation (i.e. observations within a subject are assumed to be equally correlated) to examine the association between predictors and bone scan positivity. We fit univariable models with the following predictors: age (continuous); year of scan (continuous); race (black vs non-black); treatment centre; biopsy Gleason score (2–6 vs 7 vs 8–10 vs unknown/no biopsy), primary localized treatment (none vs radical prostatectomy ± radiation vs radiation alone), time from ADT to CRPC (continuous), PSA at CRPC (continuous; log-transformed), time from CRPC to scan (continuous), pre-scan PSA (continuous; log-transformed), and pre-scan PSADT (continuous, log-transformed). We then fit a multivariable model with all predictors, except we ran separate models with pre-scan PSA and PSADT because these two variables were strongly correlated (Spearman r = −0.59, P < 0.001).

We then measured the performance of the Moreira risk table to predict bone scan positivity among men with M0/Mx CRPC in our cohort [9]. We tabulated the frequency of positive bone scans within PSA (<5, 5–14.9, 15–49.9, ≥50 ng/mL) and PSADT (≥15, 9–14.9, 3–8.9, <3 months) groups using our previously identified thresholds. The predictive accuracy of the model developed by Moreira et al. applied to our dataset was assessed using AUC [9]. A calibration plot was created to show the performance of the risk table. Bar plots were created to show how bone scan positivity changed across PSA and PSADT categories. Decision-curve analysis was used to evaluate the clinical net benefit of ordering a bone scan based on the Moreira table vs ordering scans for all men vs ordering scans for no one [12]. All statistical analyses were two-tailed and performed using STATA version 12 (StataCorp, College Station, TX, USA).

Results

There were 429 bone scans performed among 281 patients. The median (interquartile range [IQR]) age at CRPC diagnosis was 77 (71–83) and the median (IQR) year of diagnosis was 2005 (2002–2009 [Table 2]). The median (IQR) follow-up after CRPC diagnosis was 3.7 (2.2–5.6) years. While most patients (68%) only had one bone scan after M0/Mx CRPC diagnosis, 12% had ≥3 bone scans. Among 113 patients who had a positive bone scan, 62% had metastases detected on their first scan and 17 (15%) had ≥3 bone scans before metastases were detected.

Table 2.

Baseline patient characteristics (N=281).

Variables
Median (Q1, Q3) number of bone scans	1 (1, 2)
Median (Q1, Q3) age at CRPC, years	77 (71, 83)
Median (Q1, Q3) year of CRPC diagnosis	2005 (2002, 2009)
Race, n (%)
Non-black	162 (58)
Black	118 (42)
Treatment centre, n (%)
1	69 (25)
2	59 (21)
3	153 (54)
Biopsy Gleason score, n (%)
2–6	45 (16)
7	46 (16)
8–10	84 (30)
Unknown/No biopsy	106 (38)
Primary localized treatment, n (%)
None	149 (53)
Radical prostatectomy ± radiation	45 (16)
Radiation alone	87 (31)
Median (Q1, Q3) time from ADT to CRPC, months	47 (22, 77)
Median (Q1, Q3) PSA at CRPC, ng/mL	4.7 (3.1, 8.4)
Median (Q1, Q3) total follow-up, months	41 (24–59)

ADT, androgen deprivation therapy; CRPC, castration-resistant prostate cancer; Q1, 25th percentile; Q3, 75th percentile.

Table 3 shows baseline bone scan characteristics stratified by bone scan positivity. Note that patients were counted multiple times in this table if they had multiple bone scans. There were 316 (74%) negative and 113 (26%) positive bone scans. Positive bone scans were associated with older age (77 vs 75 years, OR 1.03, P = 0.022), higher PSA level at CRPC (5.5 vs 4.3 ng/mL, OR 1.36, P = 0.015), higher pre-scan PSA level (31.0 vs 10.0 ng/mL, OR 2.00, P < 0.001), and shorter PSADT (6.1 vs 11.0 months, OR 0.55, P < 0.001), compared with negative scans (Tables 2 and 3). Treatment centre was significantly associated with bone scan positivity (P = 0.031). There were no associations between bone scan positivity and year of bone scan, race, biopsy Gleason, primary localized treatment, time from ADT to CRPC, or time from CRPC to scan (all P > 0.1).

Table 3.

Baseline bone scan characteristics by scan positivity.

Variables	Negative Bone Scan	Positive Bone Scan	P*
	N=316 (74%)	N=113 (26%)
Median (Q1, Q3) age at CRPC, years	75 (69, 82)	77 (70, 83)	0.022
Median (Q1, Q3) year of scan	2006 (2003, 2011)	2007 (2004, 2010)	0.461
Race, n (%)
Non-black	178 (57)	65 (58)
Black	137 (43)	48 (42)	0.902
Treatment centre, n (%)
1	70 (22)	38 (34)	0.031
2	71 (23)	26 (23)
3	175 (55)	49 (43)
Biopsy Gleason score, n (%)
2–6	51 (16)	16 (14)	0.792
7	58 (18)	17 (15)
8–10	89 (28)	35 (31)
Unknown/no biopsy	118 (37)	45 (40)
Primary localized treatment, n (%)
None	156 (49)	55 (49)	0.839
Radical prostatectomy ± radiation	58 (18)	18 (16)
Radiation alone	102 (32)	40 (35)
Median (Q1, Q3) time from ADT to CRPC, months	44 (22, 77)	44 (19, 67)	0.539
Median (Q1, Q3) PSA at CRPC, ng/mL	4.3 (3.1, 7.5)	5.5 (3.4, 8.9)	0.015
Median (Q1, Q3) time from CRPC to scan, months	14 (5, 30)	17 (6, 33)	0.140
Median (Q1, Q3) pre-scan PSA, ng/mL	10.0 (4.6, 25.5)	31.0 (16.1, 104.2)	<0.001
Median (Q1, Q3) pre-scan PSADT, months	11.0 (5.7, 40.6)	6.1 (3.8, 10.8)	<0.001

ADT, androgen deprivation therapy; CRPC castration-resistant prostate cancer; PSADT, PSA doubling time; PSAV, PSA velocity; Q1, 25th percentile; Q3, 75th percentile.

^*P value calculated using generalized estimating equations model.

On multivariable analysis, only higher pre-scan PSA level (OR 2.13, P < 0.001) and shorter PSADT (OR 0.53, P < 0.001) were significantly predictive of bone scan positivity (Table 4). Indeed, the observed frequencies of a positive scan in general increased with increasing PSA level and shorter PSADT (Table 5 and Fig. 2). It was notable that bone scan positivity in men with PSA <5 ng/mL was 5% (3/60) and did not vary by PSADT, although numbers in the short PSADT were small. Overall, the Moreira table had excellent calibration within our independent data (Fig. 3A), except if the estimated probability of a positive bone scan was >60%, when the Moreira tables underestimated actual risk. The Moreira risk tables had an AUC of 0.735 (Fig. 3B). Decision-curve analysis showed the Moreira risk estimates were superior to scanning everyone or scanning no one, with net benefit across a wide range of risk (Fig. 4)

Table 4.

Predictors of bone scan positivity.

Variables	Univariable results		Multivariable results*
	OR (95% CI)	P	OR (95% CI)	P
Age at CRPC (years)	1.03 (1.00–1.05)	0.022	1.01 (0.98–1.04)	0.515
Year of scan (years)	1.02 (0.97–1.07)	0.461	1.04 (0.97–1.11)	0.292
Race
Non-black	Reference	–	Reference	–
Black	0.97 (0.64–1.49)	0.902	0.99 (0.60–1.65)	0.982
Treatment centre
1	reference	0.031	reference	0.080
2	0.63 (0.35–1.12)		0.92 (0.44–1.92)
3	0.52 (0.32–0.85)		0.54 (0.30–0.97)
Biopsy Gleason score
2–6	Reference	0.792	Reference	0.756
7	0.91 (0.42–1.95)		1.28 (0.53–3.13)
8–10	1.24 (0.63–2.42)		1.45 (0.62–3.35)
Unknown/no biopsy	1.15 (0.60–2.19)		1.50 (0.70–3.21)
Primary localized treatment, n (%)
None	Reference	0.839	Reference	0.679
Radical prostatectomy ± radiation	0.87 (0.48–1.58)		1.37 (0.64–2.94)
Radiation alone	1.05 (0.66–1.68)		1.19 (0.66–2.17)
Time from ADT to CRPC (months)	1.00 (0.99–1.00)	0.539	1.00 (0.99–1.00)	0.558
PSA at CRPC (ng/mL)†	1.36 (1.06–1.73)	0.015	0.83 (0.60–1.14)	0.248
Time from CRPC to scan (months)	1.01 (1.00–1.02)	0.140	1.00 (0.98–1.01)	0.605
Pre-scan PSA (ng/mL)†	2.00 (1.67–2.41)	<0.001	2.13 (1.71–2.66)	<0.001‡
Pre-scan PSADT (months)†	0.55 (0.44–0.69)	<0.001	0.53 (0.41–0.68)	<0.001‡

ADT, androgen deprivation therapy; BCR, biochemical recurrence; CI, confidence interval; PSA, prostate-specific antigen; PSADT, PSA doubling time; PSAV, PSA velocity.

^*P value calculated using generalized estimating equations model;

^†Log-transformed variable was used in this analysis.

^‡Fit in separate multivariable models.

Table 5.

Observed frequencies of positive scan by PSA and PSA doubling time group.

PSADT (months)	PSA
	<5 ng/mL % (n/N)	5–14.9 ng/mL % (n/N)	15–49.9 ng/mL % (n/N)	≥50 ng/mL % (n/N)
≥15	6 (3/49)	12 (5/39)	33 (7/21)	29 (2/7)
9–14.9	0 (0/6)	14 (3/21)	39 (9/23)	29 (2/7)
3–8.9	0 (0/5)	26 (9/34)	25 (14/55)	50 (25/50)
<3	0	20 (1/5)	36 (4/11)	87 (13/15)

Cells represent the observed percentage of positive scans in that group.

Fig. 2.

(a) Bone scan positivity by pre-scan prostate-specific antigen (PSA) group. (b) Bone scan positivity by pre-scan PSA doubling time (PSADT) group.

Fig. 3.

(a) Calibration curve. (b) Receiver-operating characteristic (ROC) curve.

Fig. 4.

Decision-curve analysis.

Discussion

Among men with M0 CRPC, detecting metastases is a key event. It allows the delivery of multiple potential life-prolonging therapies not currently indicated for men without metastases; however, few tools are available to select which patients need imaging. We previously developed a risk table (Moreira table) with an AUC of 0.773 to predict positive bone scan in men with M0/Mx CRPC [9]. In this validation study, we analysed data from 281 men with M0/Mx CRPC undergoing bone scan imaging, none of whom were included in our previous study. On multivariable analysis, the only two significant predictors of a positive scan were the same two we found in our previous study: PSA and PSADT. Our previously developed Moreira table had an AUC of 0.735 in this validation set with very good calibration except in men with extremely high-risk prostate cancer. Decision-curve analysis showed net benefit to using the tables across a wide range of risk. In summary, we have developed and, in the present study, validated a risk table for predicting a positive bone scan in men with M0/Mx CRPC. Use of the Moreira table can reduce unnecessary scans and increase appropriate imaging to identify metastases as early as possible.

The development of metastases is a watershed moment in the life of a man with prostate cancer. Six different life-prolonging therapies have been approved by the US Food and Drug Administration for treating metastatic CRPC [13]. Moreover, two bone-targeted therapies are approved to reduce the risk of skeletal-related events; however, no therapies are approved for men with M0 CRPC so the development of metastases enables the clinician to provide efficacious treatments. Metastasis is an unwelcome sign that the tumour has progressed, therefore, detecting metastases early to allow a change in treatment plan is crucial.

Unfortunately, only limited data exist on which patients are at increased risk of having a positive imaging study. The Prostate Cancer Radiographic Assessments for Detection of Advanced Recurrence (RADAR) Group has previously published a report aimed at standardizing the follow-up of M0 CRPC [14]. They recommend a first and second bone scan when the PSA reaches 2 and 5 ng/mL, respectively, followed by scans every doubling of PSA level; however, this approach has not been validated and, based on our results, PSADT should be taken into consideration when determining the best bone scan schedule. Previous secondary analyses of randomized clinical trials evaluating novel agents showed that PSA and PSADT correlated with risk of future metastasis among men with M0 CRPC [15,16]. To improve risk stratification in this patient population, we previously analysed 312 patients with M0/Mx CRPC who were all diagnosed with M0/Mx CRPC at two VA centres in 2000 or later [9]. In that previous study, we found that pre-scan PSA level (hazard ratio [HR] 1.85, P < 0.001) and PSADT (HR 0.73, P = 0.035) were both significantly predictive of a positive scan. Using these data, we developed a risk table to predict the risk of a positive scan, which had an AUC of 0.773.

In the present study, we sought to validate our previous study findings. Despite using a different cohort for the present study, the patient and disease characteristics were very similar between the current (validation cohort) and our previous study (training cohort; Table S1). Thus, we believe both datasets well represent men with M0/Mx CRPC within the VA. Overall, we found very similar results. As in our previous study, both PSA (HR 2.13, P < 0.001) and PSADT (HR 0.53, P < 0.001) were significantly predictive of a positive scan, with similar, but slightly stronger HRs in the validation cohort. The model performed very well with an AUC of 0.735 and excellent calibration except in the highest risk men, for whom the model underestimated risk. As imaging would probably be recommended for these men, whether true risk is 70% (model estimate) or 75% (actual), the clinical implication of such a slight underestimate at the extreme is minimal.

Finally, on decision-curve analysis, the tables showed net benefit across a wide range of risk. The Moreira table represents an independently validated model to predict the risk of a positive bone scan in men with M0/Mx CRPC. It is hoped use of this model may allow better tailoring of patient’s scanning to identify metastases early while minimizing over-imaging.

The exact risk threshold that should prompt imaging should be left to the discretion of the patient and treating physician; however, for men with a PSA <5 ng/mL, the risk of positive imaging in the nomogram training cohort was 6% and 5% in this validation dataset [9]. Indeed, the Moreira tables predict a <10% risk of positive imaging except for men with the shortest PSADT (<3 months). Collectively these data suggest most men with a PSA <5 ng/mL may safely forego bone scan imaging.

The present study was not without limitations. First, both our development and validation cohort were men within the VA. Further external validation in other populations is needed. Second, the frequency of bone scans was not standardized and was at the discretion of the treating physician. Third, outcome was positive imaging on bone scan. We did not perform bone biopsies to confirm metastases. As bone scan imaging can have both false-negatives and false-positives, further exploration of the Moreira table is needed if different imaging techniques are used with varying sensitivity and specificity vs bone scans (e.g. sodium fluoride positron emission tomography scans). Fourth, we did not include results from cross-sectional imaging (e.g. CT). In a follow-up study, we found detecting soft tissue metastases using CT is challenging, with standard PSA and PSA kinetics providing limited information [17]. Future studies are needed to better define predictors of soft tissue metastases. Our cohort included men without known metastases. This included both men with previous imaging documenting lack of metastases but also other patients who had not undergone previous imaging. How this may have affected the results is unknown. Finally, the clinical relevance of detecting metastases earlier untested. While this allows earlier intervention with life-prolonging therapies, the degree to which this improves ultimate outcomes requires further study.

Despite these limitations, the present study has key strengths. It was an independent validation study using identical inclusion and exclusion criteria. It included a reasonably large sample size, although future larger studies are needed. We accounted for the repeated measure nature of repeat bone scans over time. Finally, we have for the first time, validated a model to predict metastases in men with M0/Mx CRPC, remains a common, but understudied patient population.

In summary, we validated our previously developed Moreira table to predict the risk of metastases among men with M0/Mx CRPC. In this validation cohort, the model worked well with excellent calibration, and decision-curve analysis showed net benefit across a wide range of risk. Using this model may allow better tailoring of patient’s scanning to identify metastases early while minimizing over-imaging.

Abbreviations:

CRPC

castration-resistant prostate cancer

AUC

area under the curve

HR

hazard ratio

PSADT

PSA doubling time

ADT

androgen deprivation therapy

VA

Veterans Affairs

IQR

interquartile range