Toward better use of bone scans among men with early-stage prostate cancer

Abstract

Objective

To evaluate the performance of published guidelines compared to current practice for radiographic staging of men with newly-diagnosed prostate cancer.

Materials and Methods

Using data from the Michigan Urological Surgery Improvement Collaborative (MUSIC) clinical registry, we identified 1,509 men diagnosed with prostate cancer from March 2012 through June 2013. Clinical data included age, prostate-specific antigen (PSA), Gleason score (GS), clinical T-stage, number of biopsy cores and bone scan (BS) results. We then fit a multivariable logistic regression model to examine the association between clinical variables and the occurrence of bone metastases. Because some patients did not undergo BS, we used established methods to correct for verification bias and estimate the diagnostic accuracy of published guidelines.

Results

Among 416 men who received a BS, 48 (11.5%) had evidence of bone metastases. Patients with bone metastases were older, with higher PSA and GS (all p <0.05). In multivariable analyses, PSA (p <0.001) and GS (p =0.004) were the only independent predictors of positive BS. Guidelines from the American Urology Association (AUA) and the National Comprehensive Cancer Network (NCCN) demonstrated similar performance in detecting bone metastases in our population, with fewer negative studies than the European Association of Urology (EAU) guideline. Applying the AUA recommendations (i.e., image when PSA >20 or GS ≥8) to current clinical practice, we estimate that <1% of positive studies would be missed, while the number of negative studies would be reduced by 38%.

Conclusions

Based on current practice patterns, more uniform application of existing guidelines would ensure that BS is performed for almost all men with bone metastases, while avoiding many negative imaging studies.

INTRODUCTION

Optimal treatment of men with newly-diagnosed prostate cancer depends on the stage of disease at diagnosis. An important aspect of clinical staging is the detection of metastases, including spread to the bone. Accordingly, performance of a radionuclide bone scan (BS) is pivotal to the diagnostic evaluation and treatment planning for some men with prostate cancer. At the same time, however, these studies are expensive and time-consuming, and the overall yield (i.e., likelihood of detecting metastases) is quite low for men with low- or intermediate-risk cancers. For these and other reasons, many express concern about the well-established and persistent variation in the use of staging BSs, including potentially unnecessary testing in many men at low risk for metastatic disease and the absence of testing for some men with higher-risk cancers. Underscoring the significance of this issue, the American Urology Association (AUA) recently identified the avoidance of BSs in men with low-risk prostate cancer as its number one priority for the national Choosing Wisely® program.¹

Nonetheless, while existing clinical guidelines are clear about omitting BSs in men with low-risk cancers,^2–5 there is no consensus regarding the optimal use of imaging for men with higher-risk, but still clinically-localized tumors. The net effect is that imaging practice patterns continue to vary widely, implying immediate opportunities to improve value in this area of prostate cancer care. Many believe that an important next step in this process is to move away from recommendations based on the risk of recurrence after treatment (e.g., D’Amico risk groups), and toward the identification and implementation of imaging criteria that most accurately forecast a positive study that would actually change clinical decision-making.

In this context, we sought to identify predictors of a positive BS in a population-based sample of men with newly-diagnosed prostate cancer from the diverse academic and community practices in the Michigan Urological Surgery Improvement Collaborative (MUSIC).⁶ We examined the association between routine clinical variables (e.g., PSA, clinical T stage) and the occurrence of bone metastases. Because not all men with newly-diagnosed prostate cancer underwent staging BS, we used an established method to correct for verification bias to evaluate the accuracy of published imaging guidelines for detection of bone metastases in this real-world patient population. We also estimated the percentage of patients with positive studies that would be missed, the total percentage of negative studies, and the change in total number of BSs that can be expected from successful implementation of each clinical guideline compared to current practice.

MATERIALS AND METHODS

Study population and clinical variables

Established in 2011 with funding from Blue Cross Blue Shield of Michigan (BCBSM),⁷ MUSIC is a consortium of 32 practices from throughout Michigan (including more than 75% of urologists in the state) that aims to improve the quality and cost-efficiency of care provided to men with prostate cancer. Each practice involved in MUSIC obtained an exemption or approval for participation from a local institutional review board.

All participating practices employ trained clinical abstractors to review the medical record and enter standardized data elements into a web-based clinical registry. Included in the registry are all men seen in participating practices for prostate biopsy or newly-diagnosed prostate cancer. The registry contains detailed clinical and demographic information, including patient age, serum PSA at diagnosis, clinical T stage, biopsy GS, total number of biopsy cores, number of positive cores, and the receipt and results of staging BS ordered by the treating urologist. This analysis included 1,519 patients with newly-diagnosed prostate cancer seen at 19 practices in Michigan from March 2012 through June 2013.

Primary outcome

The primary outcome variable for this analysis was the occurrence of a positive BS. The final classification of a study as positive or negative was determined by the local data abstractor, treating urologist, and clinical champion in each practice, according to established criteria for the MUSIC collaborative. For a sample of patients, BS results were also validated by members of the MUSIC Coordinating Center during regular on-site data audits performed at each participating practice.

Statistical analyses

As a first step, we compared clinical and pathological characteristics of patients with or without BS. Differences between these two groups of patients in medians for quantitative variables, and differences in distributions for categorical variables, were compared using Mann-Whitney’s U-test, and Chi-square test, respectively. We next performed univariate and multivariate analyses to examine the association between a positive BS and several routinely available clinical variables in the sample of patients who received staging BS. The variables included in the models were the age at diagnosis, a natural logarithm of PSA+1 (Ln(PSA+1)), biopsy GS (≤3+4 vs. 4+3 vs. 8–10), clinical stage (T1 vs. T2 vs. T3/4), and the percentage of positive biopsy cores (defined as the number of cores containing cancer over total number of cores sampled).). The selection of these variables was based on both previously published studies and clinical experience.. All statistical testing was two-sided with a significance level of 0.05, and was performed using computerized software (SAS v 9.3).

Guideline assessment and correction for verification bias

Next, we evaluated sensitivity and specificity of the European Association of Urology (EAU), American Urological Association (AUA) and the National Comprehensive Cancer Network (NNCN) guidelines,^2,3,5 each of which recommend staging BS only in certain patient subgroups. The EAU guideline recommends staging BS in patients with GS ≥8, or locally advanced disease or PSA >10 ng/ml.⁵ According to the AUA guidelines, BS is recommended for patients with poorly differentiated tumors or PSA >20 ng/ml.² According to the NCCN guidelines, staging BS should be performed in all patients with GS ≥8, or cT3/4, or cT1 and PSA >20 ng/ml, or cT2 and PSA >10ng/ml.³ Recently, Briganti et al. developed a risk stratification tool using the classification and regression tree (CART) technique to identify patients requiring staging BS at diagnosis.⁴ Based on this analysis, BS should be performed in patients with biopsy GS ≥8, or PSA >10 ng/ml and cT2/3. A table summarizing the guidelines is provided as supplementary materials.

A key consideration in this step is that patients who did not undergo a staging BS at diagnosis have unverified disease status because the presence or absence of bone metastases is not known with certainty. In order to address this, and obtain more accurate estimates of sensitivity and specificity, we used the method of Begg and Greenes to correct for verification bias.⁸ To apply this method, we estimated the probability of a positive BS for all patients as a function of clinical variables using the multivariate logistic regression model as presented in Table 3. For each guideline, the predicted probabilities were summed separately for those patients who were recommended and not recommended for staging BS, yielding the estimated number of patients with positive or negative BS result. Sensitivity and specificity for the entire sample were estimated using the equations defined in Begg and Greenes. These estimates are unbiased if the presence of metastatic disease is conditionally independent of whether or not a patient underwent BS.

Table 3.

Performance characteristics of the EAU, AUA, and NCCN guidelines after correction for verification bias

Clinical Guidelines	Sensitivity, %	Specificity, %
EAU5	84.5	75.9
AUA2	81.3	82.0
NCCN3	82.3	80.9
Briganti's CART4	79.4	83.3

RESULTS

Table 1 presents clinical characteristics of the 1,509 patients with newly-diagnosed prostate cancer. Among this group, 416 (27.6%) underwent staging BS. Patients who received staging BS had higher mean PSA values as well as higher percentages of positive cores compared to patients without BS (all p ≤0.001). Moreover, patients with BS were significantly older and showed a higher biopsy GS as well as higher rate of locally advanced prostate cancer compared to patients without BS (all p ≤0.001). Among the patients that received a BS, 48 (11.5%) had a positive study with evidence for bone metastases.

Table 1.

Patient characteristics

Variables	All patients without BS (n=1,103)	All patients with BS (n=416)	p value
Age, (years)			0.02
Mean (median)	64.2 (64.4)	68.2 (67.7)
Range	40.4 – 95.8	41.8 – 90.5
Clinical Stage, No. (%)			<0.0001
T1	881 (79.9)	216 (51.9)
T2	214 (19.4)	173 (41.6)
T3/4	8 (0.7)	27 (6.5)
PSA, ng/mL			0.003
Mean (median)	8.0 (5.2)	61.8 (7.7)
Range	0.2 – 620.8	0.4 – 6873.4
Ln(PSA+1)			<0.0001
Mean (median)	1.9 (1.8)	2.5 (2.2)
Range	0.2 – 6.4	0.3 – 8.8
PSA, ng/mL, No. (%)			-
≤10	1018 (92.3)	247 (59.4)
10.1–20	58 (5.3)	81 (19.5)
20.1–50	10 (0.9)	45 (10.8)
50.1–100	12 (1.1)	20 (4.8)
>100	5 (0.5)	23 (5.5)
Biopsy Gleason sum, No. (%)			<0.0001
≤6	488 (44.2)	33 (7.9)
3+4	439 (39.8)	105 (25.2)
4+3	137 (12.4)	58 (13.9)
8–10	39 (3.6)	220 (52.9)
Biopsy cores taken, No.			0.50
Mean (median)	12.5 (12.0)	12.9 (12.0)
Range	4 – 82	1 – 78
Positive cores, No.			0.0004
Mean (median)	3.2 (3.0)	6.3 (6.0)
Range	0 – 20	1 – 16
Positive cores, %			<0.0001
Mean (median)	26.4 (21.1)	51.2 (50.0)
Range	0 – 100	3.1 – 100

Table 2 summarizes results from univariate and multivariate analyses evaluating the relationship between clinical parameters and BS findings. There was a wide range of serum PSA values, (0.4 – 6873.4 ng/mL, coefficient of variation 651.2), and due to the dispersion in PSA levels, we used the natural logarithm transformation. In univariate logistic regression analyses, all variables were significant predictors of bone metastases (all p ≤0.01). In multivariable analyses, only serum PSA and biopsy GS were significant predictors of a positive BS (both p-values ≤0.004) (Table 2). Illustrating this point, the adjusted odds of a positive BS for patients with a biopsy GS 4+3=7 are 3.30 (95% CI: 0.55 – 19.89) times as great as for patients with GS 3+4=7 or GS =6, while for patients with biopsy GS 8–10, the odds of a positive BS are 9.53 (95% CI: 2.14 – 42.38) times the odds for patients in the reference group.

Table 2.

Univariable and multivariable logistic regression models predicting the presence of bone metastases at diagnosis

	Univariable logistic regression model		Multivariable logistic regression model
Factors	OR (95% CI)	p value	OR (95% CI)	p value	Overall p value
Age, (years)	1.04 (1.01 – 1.08)	0.01	1.03 (0.99 – 1.06)	0.14	(0.14)
Ln(PSA+1)	2.25 (1.76 – 2.88)	<0.0001	2.00 (1.51 – 2.64)	<0.0001	(<0.0001)
Biopsy Gleason sum
≤3+4	Reference		Reference		(0.004)
4+3	5.04 (0.90 – 28.31)	0.07	3.30 (0.55 – 19.89)	0.19
8–10	16.05 (3.82 – 67.45)	0.0002	9.53 (2.14 – 42.38)	0.003
Clinical T stage					(0.43)
T1	Reference		Reference
T2	2.64 (1.31 – 5.33)	0.007	1.61 (0.72 – 3.57)	0.25
T3/4	9.19 (3.51 – 24.03)	<0.0001	1.91 (0.57 – 6.43)	0.30
Positive Cores, %	13.32 (4.26 – 41.72)	<0.0001	1.70 (0.42 – 6.90)	0.46	(0.46)

OR = odds ratio; PSA = prostate specific antigen; CI = confidence interval

Guideline assessment and correction for verification bias

The verification-bias adjusted sensitivity and specificity of several existing guidelines are presented in Table 3. The EAU guideline had the highest sensitivity and the lowest specificity. Briganti’s CART had the lowest sensitivity but the highest specificity. The performance of the AUA and NCCN guidelines were relatively similar to the performance of Briganti’s CART in terms of sensitivity and specificity, with a maximum difference of approximately 3%.

We also used the multivariate logistic regression model to evaluate the performance of the guidelines with respect to the estimated number of positive BSs missed and the number of negative BSs. The results from this analysis are summarized in Table 4. The guidelines all had less than 1% of positive studies missed; however, the EAU guidelines resulted in a much higher number of negative studies and the highest total number of studies performed. The AUA and NCCN guidelines and Briganti’s CART all had significantly fewer studies performed as compared to the EAU guideline. Under the AUA guidelines, the average number of negative studies and the total number of studies would be reduced by 38% and 6% respectively, compared to current practice.

Table 4.

Performance characteristics of existing clinical guidelines

	Patients with BS (n=416)			Patients without BS (n=1103)			Entire Population (n=1519)
Guidelines	No. of patients to be scanned, (%)	No. of positive BS missed, (%)	No. of negative BS, (%)	No. of patients to be scanned, (%)	Expected no. of positive BS missed, (%)	Expected no. of negative BS, (%)	No. of patients to be scanned, (%)	Expected no. of positive BS missed, (%)	Expected no. of negative BS, (%)
EAU5	288 (69.2)	1 (0.1)	127 (30.5)	117 (10.6)	8 (0.7)	108 (9.8)	405 (26.7)	10 (0.7)	350 (23.0)
AUA2	255 (61.3)	1 (0.1)	160 (38.5)	59 (5.3)	9 (0.8)	51 (4.6)	314 (20.7)	12 (0.8)	261 (17.2)
NCCN3	265 (63.7)	1 (0.1)	150 (36.1)	67 (6.1)	9 (0.8)	59 (5.3)	332 (21.9)	12 (0.8)	278 (18.3)
Briganti’s CART4	244 (58.7)	5 (1.2)	167 (40.1)	50 (4.5)	10 (0.9)	44 (4.0)	292 (19.2)	13 (0.9)	243 (16.0)

COMMENT

We used contemporary data from a large group of community and academic urology practices to investigate the association between routinely available clinical variables and the likelihood of a positive BS among men with newly-diagnosed prostate cancer. We found that serum PSA and biopsy GS were the principal predictors of a positive study among patients who received BS. Furthermore, after accounting for the fact that not all patients underwent staging BS, we demonstrated that the AUA and NCCN guidelines and Briganti’s CART model all performed reasonably well in terms of sensitivity and specificity. The EAU guideline resulted in higher sensitivity but also substantially lower specificity.

Our work is consistent with previous investigations demonstrating that higher PSA levels and biopsy GS are associated with increased risk for a positive BS.^9–21 In contrast to our findings, some studies have also identified clinical stage as an important predictor of bone metastases at diagnosis.^4,9,10,22 In one recent study of 851 consecutive patients with imaging, Briganti et al found that PSA, biopsy GS, and clinical stage were all significant predictors of a positive BS and should therefore be used as criteria for selecting patients for such studies.⁴ Based on the findings of these studies, several organizations (EAU, AUA, and NCCN) updated their recommendations indicating the need for staging BS only for newly-diagnosed prostate cancer patients with certain unfavorable characteristics. However, despite the availability of these guidelines, there is still controversy over the referral criteria, and no consensus exits about the most accurate and cost-effective strategy. Accordingly, widespread implementation of these guidelines could improve care delivery by eliminating a large number of BSs that do not reliably contribute useful clinical information, while adding significant costs to the healthcare system.

Since not all patients in the registry received a BS, our findings are susceptible to verification bias. We partially mitigated the impact of this by using a bias-correction procedure for estimating sensitivity and specificity developed by Begg and Greenes.⁸ Since the underlying assumptions behind these formulas cannot be proven, the values of adjusted sensitivity and specificity must be viewed as estimates. Second, the results of this study depend on the characteristics of the patient population in practices in Michigan, and these may differ from those observed in other geographic regions. Another limitation is the possibility for correlation among clinical practices in MUSIC regarding imaging patterns for staging BS. We addressed this issue by performing sensitivity analyses that implemented generalized estimating equations to account for potentially correlated data, and we noted no substantive changes in our principal findings. Therefore, we assumed that the possible correlation among clinical practices in MUSIC is not strong, and we fit our final logistic regression model.

These limitations notwithstanding, our study has several strengths, as well as important clinical and policy implications. This is the first analysis to evaluate the performance of existing clinical guidelines for staging bone scans in a population-based sample of patients seen in diverse academic and community practices. In addition, we provide specific estimates around the impact of specific guideline implementation (relative to existing practice patterns) with respect to the number of positive scans missed, the number of negative scans, and the total number of scans performed in a population. Such estimates may prove quite useful for clinicians, specialty societies, and other stakeholders seeking a satisfactory tradeoff between the benefits and harms of using BSs for staging of patients newly-diagnosed with prostate cancer.

Illustrating this point, our data indicate that adherence with recommendations to image with a BS only when PSA >20 or GS ≥8 would lead to an estimated decrease in the overall utilization of staging BSs by 6.6% compared to current imaging practices in Michigan. If these criteria were implemented across all MUSIC practices, we estimate that fewer than 1% of patients with bone metastases would not be imaged, and that a large proportion of negative studies that are now being ordered could be safely omitted. Given the consistency of our empirical findings with recommendations from the AUA, many urologists in MUSIC have coalesced around PSA >20 or GS ≥8 as criteria for ordering staging bone scans in patients with newly diagnosed prostate cancer. Moreover, we are taking purposeful steps to now implement these criteria statewide through the use of data feedback, reminder cards, and other established quality improvement strategies.

CONCLUSIONS

In this analysis of patients seen in diverse community and academic practices in Michigan, we identified serum PSA and biopsy GS as significant predictors for the presence of bone metastases in newly- diagnosed, untreated prostate cancer patients. Our results also suggest that implementing recommendations where a staging BS is performed only in patients with PSA >20 ng/ml or GS ≥8 would simultaneously result in fewer positive studies missed, fewer negative studies, and fewer BSs overall.