Optimization of PSA Screening Policies: a Comparison of the Patient and Societal Perspectives

Abstract

Objective

To estimate the benefit of PSA-based screening for prostate cancer from the patient and societal perspectives.

Method

A partially observable Markov decision process (POMDP) model was used to optimize PSA screening decisions. We considered age-specific prostate cancer incidence rates and the mortality rates from prostate cancer and competing causes. Our model trades off the potential benefit of early detection with the cost of screening and loss of patient quality of life due to screening and treatment. PSA testing and biopsy decisions are made based on the patient’s probability of having prostate cancer. Probabilities are inferred based on the patient’s complete PSA history using Bayesian updating.

Data Sources

The results of all PSA tests and biopsies done in Olmsted County, Minnesota from 1993 to 2005 (11,872 men and 50,589 PSA test results).

Perspective and Outcome Measures

Patients’ perspective: maximize expected quality-adjusted life years (QALYs); societal perspective: maximize the expected monetary value based on societal willingness to pay for QALYs and the cost of PSA testing, prostate biopsies and treatment.

Results

From the patient perspective the optimal policy recommends stopping PSA testing and biopsy at age 76. From the societal perspective the stopping age is 71. The expected incremental benefit of optimal screening over the traditional guideline of annual PSA screening with threshold 4.0 ng/mL for biopsy is estimated to be 0.165 QALYs per person from the patient perspective, and 0.161 QALYs per person from the societal perspective. PSA screening based on traditional guidelines is found to be worse than no screening at all.

Conclusions

PSA testing done with traditional guidelines underperforms and therefore underestimates the potential benefit of screening. Optimal screening guidelines differ significantly depending on the perspective of the decision maker.

Introduction

PSA testing is controversial due to poor sensitivity and specificity of the test (1) and the concern of over-diagnosis (2, 3). Patients with higher than normal PSA values have a greater risk of prostate cancer. However, a patient’s PSA varies in a continuous range, and higher than normal levels may occur for a variety of other reasons. The imperfect sensitivity of PSA tests can result in harm to patients. For instance, if a patient’s PSA test result is incorrectly classified as suspicious, he will receive an unnecessary biopsy. In addition to providing imperfect information (sensitivity of 80% (4)) biopsies are painful and carry the possibility of side-effects such as cramping, bleeding, and infection. Furthermore, some patients detected with prostate cancer who receive treatment will ultimately die from other causes. The decision whether and when to perform a PSA test, and whether to refer a patient for biopsy must trade off the potential benefits from early detection, the side effects of biopsy and subsequent treatment, and the costs of PSA tests, biopsies and treatments because of the over-diagnosis.

Many family physicians and urologists in the U.S. use PSA tests to screen their patients, commonly initiating annual screening at age 50 (5). However, recently some have suggested that PSA screening should not be done routinely since it results in unnecessary biopsies, potential harm, and increased treatment costs (6, 7, 8, 9, 10). In 2009 the U.S. Preventive Services Task Force (USPSTF) recommended a guideline for prostate cancer screening, stating that people older than 75 years should not be screened (11). The guideline made no specific recommendation for people younger than 75 years, citing insufficient evidence. Another guideline from the American Urological Association (8) recommends PSA screening starting at age 40, and then determining future screening intervals based on previous results. However, no specific direction on how to use previous results is provided.

The debate on the controversy of PSA screening continues between two large clinical trials reported in 2009. The U.S. Prostate, Lung, Colorectal, and Ovarian (PLCO) Cancer Screening Trial (12) concludes that screening does not reduce mortality. On the other hand, the European Randomized Study of Screening for Prostate Cancer (ERSPC) trial (10) finds that the PSA based screening reduced prostate cancer mortality by approximately 20%. However, the two trials used two different PSA screening frequencies and thresholds for biopsy. Furthermore, concerns have been expressed about bias in the U.S. PLCO trial (13, 14).

We use a population-based model to investigate prostate cancer screening policies. We consider the patient’s perspective, measured in expected quality-adjusted life years (QALYs), and the societal perspective, measured as the difference in rewards for QALYs and the total cost of PSA tests, biopsies and treatment over the patient’s lifetime. We present empirical results for the optimal age dependent prostate cancer screening policy based on a population from Olmsted County, Minnesota. The optimal screening policy is compared to a commonly used PSA screening policy to estimate the potential benefit gain from considering all PSA tests in the patient’s history. Finally, the implied stopping time for PSA screening is contrasted for the patient and societal perspectives.

Methods

We formulate a partially observable Markov decision process (POMDP) model for the prostate cancer screening problem. The true health states in our model are not directly observable, but can be probabilistically inferred from the patient’s PSA history. Our POMDP model is described as follows. One criterion in our model, from the patient’s perspective, is to maximize the expected QALYs. QALYs are estimated by decrementing a normal life year based on: (a) occurrence of biopsy, (b) treatment upon detection of cancer, and (c) long-term complications resulting from treatment. The optimal policy obtained from solving the POMDP trades off the long-term benefits from early detection of prostate cancer with the short term negative impact of biopsy and long-term side effects of treatment. The second criterion in our model, from the societal perspective, is to maximize the expected monetary value, using the societal willingness to pay to translate the expected QALYs into a monetary value and subtracting the costs of PSA tests, biopsies, and treatments.

In our model patients progress through the following unobservable health states before cancer is diagnosed: no prostate cancer, organ confined prostate cancer, extracapsular prostate cancer, lymph node positive prostate cancer, metastasis and death. The patient’s health state is inferred probabilistically based on PSA test observations. Observations are defined by a set of clinically relevant ranges {[0, 1), [1, 2.5), [2.5, 4), [4, 7), [7, 10), and [10, ∞)}. At each annual decision epoch there is a two-stage decision problem: the first stage is whether to PSA test or not; the second stage is whether to biopsy or not. If a patient receives a positive biopsy result, he is assumed to be treated by prostatectomy. If a patient receives a negative biopsy result then he awaits PSA screening in the next decision epoch. Figure 1 is an illustration of the prostate cancer screening decision process.

Figure 1.

Recurring screening decision process for prostate cancer screening, in which decision epoches (t, t+1, ⋯) occur annually. Squares represent decisions; circles represent test observations; the triangle denotes the expected outcome upon detection of prostate cancer.

Following is a description of the essential elements of the POMDP model (additional details are in the appendix):

Time Horizon: PSA screening is performed at each of a set of annual decision epochs starting at age 40, t ∈ {40, 41, 42, ⋯}.

Decisions: a_t ∈ {B, DB, DP}, denotes the decision to perform a biopsy (B), defer biopsy and obtain a new PSA test result in epoch t + 1 (DB), or defer the biopsy decision and PSA testing in decision epoch t + 1 (DP). Combinations of these three actions over the decision horizon determine the PSA test and biopsy schedule. For instance, a₄₀ = DB, a₄₁ = DP, a₄₂ = DB and a₄₃ = B imply PSA testing at age 41 and 43, and followed by biopsy at age 43. Note that decisions are made sequentially and based on the probability of prostate cancer at each decision epoch.

States: At each decision epoch a patient is in one of several health states including no cancer (NC), organ confined (OC) cancer detected, extraprostatic (EP) cancer detected, lymph node-positive (LN) cancer detected, metastases (mets) detected, and death from prostate cancer and all other causes (D). Prior to detection, OC, EP, LN and mets are not differentiable. We use an aggregate state C to denote prostate cancer present but not detected. Note that we assume states NC and C are not directly observable without biopsy. The underlying health states of Markov model are illustrated in Figure 2.

Figure 2.

Transitions among health states of the prostate cancer Markov model. Note that death from other causes is possible for any health state in our model, but omitted for simplicity.

Observations: At each decision epoch the patient is observed in one of a set of observable states including a particular PSA interval, or cancer detected and treated (T) or death (D). The observable states are indexed by ℓ_t ∈ M = {1, 2, 3, …, m, T, D}.

Transition Probabilities: There are two kinds of transition probabilities in the model. p_t(s_t+1|s_t, a_t) denote the state transition probability from health state s_t to s_t+1 at epoch t given action a_t. q_t(ℓ_t|s_t) denote the probability of observing PSA state ℓ_t ∈ M given the patient is in health state s_t ∈ S; its matrix form is also known as the information matrix.

Belief States: The belief state, π_t = (π_t(NC), π_t(C), π_t(T), π_t(D)), defines the probability the patient is in one of the four health states at epoch t. Note that, for a patient without a positive biopsy result, his belief state can be represented as π_t = (1 − π_t(C), π_t(C), 0, 0). We use π_t(C), the probability of having prostate cancer, to denote the belief state in the remainder of the article.

Rewards: r_t(s_t, a_t) is the reward of living for a year given the patient is in health state s_t and decision a_t. The expected reward of living for a year is the average over possible health states: r_t(π_t, a_t) = ∑_st∈S r_t(s_t, a_t)π_t(s_t). The reward defines the decision makers perspective. For patient’s it is measured in QALYs. For the societal perspective it is measured (in dollars) as the difference in (a) the product of QALYs and a willingness to pay factor and (b) the cost of PSA tests, biopsy, and treatment.

We assume that a patient has at most one biopsy prior to detection of prostate cancer. This is a reasonable assumption since multiple biopsies prior to detection are seldom done due to the invasive nature of the procedure and the slow progression of prostate cancer. This assumption is validated by our dataset in which 81.4% of the patients have only one biopsy. It is also consistent with findings in (15) in the context of regular annual screening.

The detailed transition probabilities, the reward function and the optimality equations for the POMDP are provided in the appendix. The optimal screening policy is obtained by solving the optimality equations to estimate the PSA screening and biopsy which maximize the expected rewards over a patient’s lifetime. POMDPs are computationally challenging to solve. We use a finite fixed-grid method (16) to solve the optimality equations and obtain the optimal policy. Theoretical properties and methodological details of the solution methodology for a related model on biopsy referral can be found in (15).

The optimal policies obtained from our POMDP model differ in several aspects from traditional policies which are based on a predefined PSA screening frequency and PSA threshold for biopsy referral. First, our policy uses the probability of being in state C. The probability is estimated from all prior PSA observations using Bayesian updating as the patient ages (see the section on Bayesian updating in the appendix for details). There is no fixed PSA test frequency; rather, PSA testing and biopsy occur according to a policy, defined by testing according to probability threshold, that maximizes long term expected rewards for the patient.

Data description

We obtained the results of all PSA tests done in Olmsted County, Minnesota from 1983 to 2005. There are a total of 11,872 men underwent PSA testing during this timeframe with a total of 50,589 PSA test results. The medical records linkage system of the Rochester Epidemiology Project (17) was then used to identify all patients that underwent a prostate biopsy or that had a pathologic diagnosis of prostate cancer during this same period of time. All health care providers in Olmsted County participate in the records linkage system, and more than 95% of Olmsted County residents receive their medical care in Olmsted County, implying that missed prostate biopsies and prostate cancer diagnoses are unlikely. We merged the PSA data with the clinical data to obtain a comprehensive longitudinal dataset of PSA screening occurring in a fixed geographic population of men not subject to major referral biases. Since we focus on the screening policy for early detection we do not consider PSA records after cancer treatment. We use it to estimate prostate cancer probabilities conditional on PSA level for a general population. Details of clinical and demographic characteristics of the screened cohort can be found in Table 1.

Table 1.

Clinical and demographic characteristics of the screened cohort from Olmsted County, Minnesota.

Population size	11,872
Age: Mean(SD1)	63.0(12.7)
Race
Caucasian	96%
Other	4%
PSA Group (ng/mL)
0–1	5, 428
1–2.5	3, 657
2.5–4	1, 230
4–7	868
7–10	302
≥ 10	387
Outcomes
Prostate biopsy	908
Prostate cancer diagnosis	628

Base case parameter estimation

Following is a summary of the model parameter estimates. First, we use our dataset to estimate the probability of observing different PSA values conditional on the patients’ health states. This is represented by the information matrix defined earlier in the Methods section. (Note that, Thompson et al. (18) estimated the prostate cancer probability given PSA test results, which is related but not the same as what we need.) Since it is possible that some men with prostate cancer were never subjected to biopsy and therefore never diagnosed, the information matrix, Q_t(ℓ_t|s_t), is subject to bias. We used the methods proposed by Begg and Greenes (19) to correct for this bias. We use biopsy as the confirmative test; thus, we assume that patients who have positive biopsies are true cancer patients and those who have negative biopsy are true no cancer patients. We first separate the patients into different groups according to their PSA values ([0, 1), [1, 2.5), [2.5, 4), [4, 7), [7, 10) and ≥ 10) and ages ([40, 50), [50, 60), [60, 70), [70, 80) and ≥ 80). Within each group, we assume patients without a confirmative test (biopsy) have the same probability of prostate cancer as patients who have had a confirmative test. The probability of having prostate cancer based on patients with confirmative tests is used to infer the cancer state of patients without confirmative tests. The resulting information matrix is

$$Q_t({\ell }_t|s_t)=\left(\begin{array}{cccccccc}\hfill 0.471\hfill & \hfill 0.337\hfill & \hfill 0.101\hfill & \hfill 0.059\hfill & \hfill 0.015\hfill & \hfill 0.017\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0.138\hfill & \hfill 0.229\hfill & \hfill 0.192\hfill & \hfill 0.226\hfill & \hfill 0.103\hfill & \hfill 0.112\hfill & \hfill 0\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 0\hfill & \hfill 1\hfill \end{array}\right),\forall t.$$

The rows of Q_t(ℓ_t|s_t) correspond to states NC, C, T, and D, respectively; the columns correspond to PSA intervals [0, 1), [1, 2.5), [2.5, 4), [4, 7), [7, 10), [10, ∞), T, and D, respectively. Q_t(ℓ_t|s_t) is fixed for all the ages in this empirical study since our numerical experiments showed the differences in Q_t(ℓ_t|s_t) with respect to age do not significantly influence the optimal policy.

The prostate cancer incidence rate, w_t (shown in Table 2), is calculated from the prevalence of prostate cancer at autopsy from the autopsy review study (20) assuming the annual prostate cancer incidence rate is fixed for each ten-year age interval.

Table 2.

The age-specific values of the prostate cancer incidence rate, w_t.

t	40 – 50	50 – 60	60 – 70	70 – 80	≥ 80
wt	2.32 × 10−3	4.70 × 10−3	7.02 × 10−3	6.17 × 10−3	1.17 × 10−2

In the absence of screening, we assume that patients are not detected with prostate cancer unless metastasis is discovered, in which case, radical prostatectomy is no longer a good treatment option. Therefore, we assume patients in state C cannot transition to state T in the absence of screening. Patients in state C have higher death rate from prostate cancer. The prostate cancer death rate in the absence of screening, e_t, is approximated using the death rate of prostate cancer patients under conservative management (21).

We assume patients detected with prostate cancer are treated by prostatectomy. This is one of the most common forms of treatment at present (6, 22). Patients that enter the treatment state, T, receive expected rewards based on an aggregation of the four prostate cancer stages. In order to estimate the annual prostate cancer death rate excluding death from other causes for patients in state T, b_t, we estimated survival rates for the four prostate cancer stages using the Mayo Clinic Radical Prostatectomy Registry (MCRPR) and the Surveillance, Epidemiology and End Results (SEER) data (23) (see Figure 3 for details of data sources and values). In estimating the MCRPR transition rates after radical prostatectomy, all patients having undergone radical prostatectomy at the Mayo Clinic in the PSA era (1990–2005) were included. The patient population of 13,313 men was stratified into three TNM 2002-based stage groupings (Stage II = organ confined, Stage III = extra-prostatic, Stage IV = lymph node positive). The probability of a radical prostatectomy patient being in one of the three stage groups was calculated from the pathologic evaluation of the prostatectomy specimens obtained from each patient. The cumulative incidence (probability) of prostate cancer metastases and prostate cancer-specific mortality was calculated for each stage group using the competing risks estimator with non-prostate cancer death representing the competing risk.

Figure 3.

The detailed classification inside state T, where CD means the time at which cancer is detected; parameters in this figure are from the Mayo Clinic Radical Prostatectomy Registry and SEER (23).

We use the four cancer stages to calculate the total annual prostate cancer death rate for state T. The calculation is based on the mean death rate across the four prostate cancer stages at the time of detection. We estimated the annual death rates of organ confined, extraprostatic, and lymph nodes positive patients from the 5, 10, 15 and 20 year death rates in Figure 3. (Letting s_t denote the t year death rate and solving four equations (1 − s₁)^t = 1 − s_t for t = 5, 10, 15, 20 for organ confined, extraprostatic and lymph nodes positive patients.) We estimated annual death rate for metastases from the 5 year death rates for patients’ age < 65 and ≥ 65 from the SEER data. The aggregated death rate of T was then computed using the weighted average (weights correspond to the probabilities of being in different cancer stages upon detection in Figure 3) of the death rate of the 4 different cancer stages in Figure 3. Based on our estimates the annual prostate cancer death rate in state T is b_t = 0.00672 for t < 65 and b_t = 0.00923 for t ≥ 65 (note that the SEER data differentiates between patients younger and older than 65 years).

In our base case, we use a decrement of μ = 0.05 in the year of biopsy to estimate the disutility of biopsy. To our knowledge no estimates of utility decrement exist yet for prostate biopsy; however this estimate is consistent with similar choices of parameters for a recent breast cancer study (24) and a bladder cancer biopsy study (25). In our base case we assume that the utility decrement in years after treatment via prostatectomy is 0.145, which is the midpoint of two extremes reported in Bremner et al. (26): (a) the most severe (metastases), 0.24, and (b) minor symptoms (mild sexual disfunction), 0.05. We assume a patient in state C, who has not been detected with prostate cancer, has no reduction in quality of life.

The mortality rate from other causes, d_t, is age specific and based on the general mortality rate from Heron (27) minus the prostate cancer mortality rate from the National Cancer Institute (23). Note that the National Cancer Institute (23) reports a single prostate cancer mortality rate for ages greater than 95 and Heron (27) reports a single all cause mortality rate for ages greater than 95. Therefore, we assume that d_t is fixed after the age of 95. All other parameters and their sources, are provided in Table 3.

Table 3.

Parameters, their sources, and specific values used in our base case analysis.

Parameter	Description	Source	Value
wt	Annual prostate cancer incidence rate	(23)	Age Specific
dt	Annual death rate from other causes	(23, 27)	Age Specific
bt	Annual prostate cancer death rate excluding death from other causes for patients in state T	MCRPR	Age Specific
et	Annual prostate cancer death rate excluding death from other causes for patients in state C	(21)	0.033
f	Biopsy detection rate for prostate cancer patients	(4)	0.8
μ	One-time utility decrement associated with biopsy	(24)	0.05
ε	Annual utility decrement of living in state T	(26)	0.145
β	Societal willingness to pay in U.S. dollars/QALY	(28)	50, 000
cp	Cost of a PSA test in U.S. dollars	(29)	37
cb	Cost of a biopsy in U.S. dollars	(29)	393
ct	Cost of prostate cancer treatment in U.S. dollars	(22)	17, 226

Results

Optimal screening policies

Base case results are provided in Figure 4. Figure 4(a) illustrates the optimal prostate cancer screening policy from the patient’s perspective (maximize expected QALYs). PSA tests occur annually until the probability threshold for biopsy is 1. A threshold of 1 is interpreted as electing not to perform biopsy (and subsequent treatment) even if the patient is known to have prostate cancer with certainty. As the patient ages the thresholds for PSA testing and biopsy increase. Thus, PSA testing and biopsy is more selective at older ages. The stopping time for PSA screening is observed to be age 76.

Figure 4.

Optimal prostate cancer screening policies from the patient and societal perspectives. Lines denote thresholds for PSA testing and biopsy. If the patient’s age and probability of having prostate cancer are in area B they are referred for biopsy; DB means defer biopsy referral until obtaining the PSA test result in the next decision epoch; DP means defer biopsy referral and the PSA test in the next decision epoch.

Figure 4(b) illustrates the optimal prostate cancer screening policy from the societal perspective (maximize the difference in expected rewards for QALYs and expected costs). The two criteria are weighted using a societal willingness to pay of β = 50, 000 dollars/QALY (28). The stopping time of PSA screening is observed to be age 71. In contrast to the patient perspective, PSA testing is more selective at older ages. The threshold for biopsy is also higher than for the patient perspective.

Benefits of prostate cancer screening

We use a simulation model based on our partially observable Markov model to compare the optimal policy to a traditional guideline of annual PSA screening with PSA threshold 4.0 ng/mL for biopsy. A sample size of 3,000,000 is used in the simulations to obtain the expected QALYs associated with the traditional guideline. From the patient perspective, we estimate the benefit of prostate cancer screening based on the expected future QALYs for a 40 year old patient starting with no prostate cancer. We compare the optimal policy versus (a) no screening and (b) the traditional guideline.

For our base case, the optimal screening policy has an average benefit of 0.131 QALYs compared to no screening, and 0.165 QALYs compared to the traditional guideline. Thus, the number needed to screen (NNS) to gain one additional QALY is 6. Note that the benefit is greater relative to the traditional guideline, implying the traditional guideline is worse than no screening at all. From the societal perspective, for our base case, with a societal willingness to pay of β = 50, 000 U.S. dollars per QALY, males benefit an average of 0.110 QALYs compared to no screening, and 0.161 QALYs compared to the traditional guideline.

Sensitivity Analysis

We provide the results of two sensitivity analyses. The first, presented in Table 4, evaluated sensitivity to μ and ε, the factors affecting QALYs. The second, presented in Table 5, evaluates sensitivity to the economic factors: costs and willingness to pay.

Table 4.

Sensitivity analysis of the expected benefits of PSA screening for 40-year-old healthy men for different ε and μ. The optimal policy is compared to the case of no screening, and the case of annual PSA tests with 4.0 ng/mL threshold for biopsy from both patient and societal perspectives. The latter perspective is based on a societal willingness to pay of β = 50, 000. Since the expected QALYs of the traditional guideline is obtained by simulation, 95% confidence interval is presented in the parentheses. The base case is represented in bold.

Patient perspective
ε	μ	Benefit over no PSA screening (QALYs/person)	Benefit over traditional guideline (QALYs/person)	Screening stopping age
0.05	0.01	0.289	0.292 (0.277, 0.307)	> 84
	0.05	0.271	0.312 (0.297, 0.327)	> 84
	0.1	0.253	0.340 (0.325, 0.355)	> 84
0.145	0.01	0.148	0.151 (0.136, 0.166)	77
	0.05	0.131	0.165 (0.150, 0.180)	76
	0.1	0.116	0.204 (0.189, 0.219)	75
0.24	0.01	0.044	0.047 (0.031, 0.062)	61
	0.05	0.032	0.073 (0.058, 0.088)	60
	0.1	0.025	0.113 (0.098, 0.128)	60
Societal perspective
ε	μ	Benefit over no PSA screening (QALYs/person)	Benefit over traditional guideline (QALYs/person)	Screening stopping age
0.05	0.01	0.267	0.270 (0.255, 0.286)	84
	0.05	0.244	0.285 (0.270, 0.300)	83
	0.1	0.224	0.312 (0.297, 0.327)	82
0.145	0.01	0.131	0.134 (0.119, 0.149)	71
	0.05	0.110	0.161 (0.145, 0.176)	71
	0.1	0.095	0.192 (0.177, 0.207)	70
0.24	0.01	0.034	0.044 (0.029, 0.060)	58
	0.05	0.022	0.063 (0.048, 0.078)	57
	0.1	0.016	0.104 (0.089, 0.119)	57

Table 5.

Sensitivity analysis of the total costs of the optimal policy and the traditional guideline for 40-year-old healthy men from the societal perspective for varying β and costs. Other parameters including ε and μ are set according to the base case.

β	costs	Cost of the optimal policy (U.S. $/person)	Cost of the traditional guideline (U.S. $/person)	Screening stopping age
250,000	−20%	957	764	74
	base case	1185	956	74
	+20%	1411	1147	73
100,000	−20%	904	764	73
	base case	1104	956	73
	+20%	1274	1147	72
50,000	−20%	768	764	71
	base case	882	956	71
	+20%	989	1147	69
25,000	−20%	583	764	68
	base case	642	956	67
	+20%	658	1147	65

From Table 4 the benefit of screening is very sensitive to μ and ε. The benefits of prostate cancer screening are most significant for cases in which μ and ε, are minimized. From the patient perspective the NNS ranges from 3.46 (μ = 0.01 and ε = 0.05) to 40 (μ = 0.1 and ε = 0.24). From the societal perspective the NNS ranges for 3.75 (μ = 0.01 and ε = 0.05) to 62.5 (μ = 0.1 and ε = 0.24). The stopping time of PSA screening is also presented in Table 4. Note that since it is rare to screen people older than 84 for prostate cancer, the prostate cancer incidence rate for patients older than 84 is underestimated. Therefore we denote later stopping times as > 84. For the patient perspective stopping time ranges from age 60 to 84 or older; from the societal perspective stopping time ranges from 57 to 84.

Table 5 presents results based on varying societal willingness to pay, and costs of PSA tests, biopsy and treatment (it focuses on the societal perspective since it is the most affected by changes in cost and willingness to pay). Costs were most sensitive to willingness to pay, ranging from 583 U.S. dollars to 1411 U.S. dollars. Stopping times varied by as much as 2 years with respect to cost, and by as much as 7 years with respect to willingness to pay.

Discussion

The introduction of PSA screening for prostate cancer is responsible, in whole or in part, for a number of positive effects including, prostate cancer diagnoses are made 5–10 years earlier than previously possible (30), a reduction in the incidence of advanced stage prostate cancer (31, 32), and a reduction in prostate cancer mortality (10). However, PSA screening also has some downsides including, the cost of screening (29), the possibility of false positive tests leading to unnecessary prostate biopsy and anxiety (33), and the overdiagnosis and overtreatment of prostate cancer (2, 3). It is widespread interest in determining whether better designed PSA screening protocols can reduce the downsides of PSA screening while continuing to bolster its benefits. Because of the difficulties of using randomized clinical trials, models such as ours are necessary for analyzing and comparing alternative PSA screening policies (14).

Ross et al. (34) report on a simulation model to compare several alternative strategies (e.g. no screening, and screening intervals of 1, 2, and 5 yrs) based on performance measures including the number of PSA tests per 1000 men and prostate cancer deaths prevented. Like our study, their model is based on a Markov process for progression of the disease. They consider two competing criteria: prostate cancer deaths prevented per 1000 men and number of PSA tests per 1000 men. Etzioni et al. (31) also use a simulation model to determine if declines in advanced stage prostate cancer can be attributed to PSA screening. They conclude that PSA screening has contributed in part to declines in incidence and resulting mortality of prostate cancer.

Markov decision processes (MDP) have been applied to several complex medical decision paradigms including liver transplantation (35), diabetes management (36), and HIV treatment (37). POMDP models, such as ours, are a special case of MDPs in which there is imperfect information about the patient’s health state, can also been applied to medical decision making and we explore this possibility in the current study. Recently, Alagoz et al. (38) provided a tutorial on the construction and evaluation of MDPs and POMDPs in medical decision making. Hu et al. (39) consider the problem of choosing an appropriate drug infusion plan for the administration of anesthesia. Hauskrecht and Fraser (40) applied a POMDP formulation to the problem of treating patients with ischemic heart disease, and Maillart et al. (41) use a partially observable Markov chain to study breast cancer screening policies using mammography.

Our study differs from the above referenced work in several respects. To our knowledge, ours is the first optimization study of prostate cancer screening that explicitly treats the imperfect nature of PSA tests and prostate biopsies over the course of a patient’s lifetime. In contrast to simulation models for evaluating PSA screening policies (34) we study an optimization model which seeks to maximize an explicit objective. We evaluate and compare two objectives for prostate cancer screening from both the patient and societal perspectives. We calibrate our model with a large population-based dataset that includes all screening and treatment events. Other studies have used datasets derived from subsets of the total population, such as high risk patients or patients participating in clinical trials, and these studies are therefore subject to significant selection bias.

Several insights can be drawn from the results of our POMDP model. We found, that PSA testing and biopsy should be discontinued at older ages because competing mortality risks eventually dominate the risk of prostate cancer. For our baseline model, the optimal policy from the patient perspective suggests annual PSA screening until age 76, which is surprisingly close to the stopping time of age 75 recommended by the U.S. Preventive Services Task Force (11). The optimal policy from the societal perspective suggests more selective PSA screening based on the patients age and probability of having prostate cancer. It also suggests that all PSA screening should be stopped after age 71, independent of the probability of prostate cancer.

The PSA thresholds at which our model recommended performing prostate biopsy decreased with age from both the patient and societal perspectives, implying that younger patients should be screened more intensively than older patients. To gain perspective on the benefits at the population level, total annual benefits for the entire eligible U.S. population were estimated by multiplying the benefit per person by the 40-year-old male population estimated from the U.S. census (42). From the patient perspective the optimal policy yields an annual incremental benefit of 293,000 QALYs for the U.S. population. From the societal perspective the annual incremental benefit is 246,000 QALYs, based on a societal willingness to pay of 50,000 U.S. dollars per QALY.

Sensitivity analysis suggests the benefits of PSA screening from both patient and societal perspectives are highly dependent on the harm induced by prostate biopsy and prostate cancer treatment. This implies that society might benefit from future studies that better estimate how harmful prostate biopsy and cancer treatment really are. Furthermore, from the sensitivity analysis, the traditional PSA screening guideline is worse than no screening at all for all the parameter settings from both patient and societal perspectives.

Our study estimates the benefit of using a patient’s full PSA history and thus provides a more accurate estimate of the true information provided by PSA-based screening. Implementation of such policies based on a patient’s probability of prostate cancer would require access to a patient’s past PSA test results. Thus our study provides an example of benefits that could be derived from standardized and easy-to-access electronic medical records.

There are some limitations of our study. Our estimates of disutilities are based on a sparse literature on QALYs for prostate cancer, biopsy and treatment. Our assumptions reflect conservative estimates of the benefits of PSA screening. For instance, we do not discount quality of life for asymptomatic patients that become symptomatic. However, our assumptions were based on the best available data and we use sensitivity analysis to characterize variation with respect to these inputs. We do not consider other screening methods such as digital rectal examination (DRE) in our model. This is due in part to the lack of data concerning DRE in our population. Due to the demographic makeup of the population in Olmsted county, MN, our model was based on a largely Caucasian population. Future work based on a more diverse population could reveal insights about the role of race as a risk factor in prostate cancer screening decisions. Our model provides a foundation for these future studies.

Conclusions

Unlike traditional PSA-threshold based policies, our model produces policies that depend on the probability of prostate cancer and the patient’s complete history of PSA test results. Our results help to understand the gap between patient and societal perspectives on prostate cancer screening policy. Our results also demonstrate the relative sensitivity of policies to parameters that influence quality of life, and encourage directions of future research to better understand patient utility decrements for screening and treatment. Our results also shed light on the recent controversy over PSA testing by evaluating the benefits that may be derived from using the full range of PSA test results rather than a single observation.

Appendix

1 Model Parameters

The parameters used to construct the state transition probability matrices and the rewards in our model are defined in Table 3.

Transition probablities

We denote the transition probability matrix given the decision to biopsy, B, as P_t(s_t+1|s_t, B), consisting of elements p_t(s_t+1|s_t, B), ∀s_t+1 ∈ S, s_t ∈ S, where the non-zero elements are:

$$\begin{array}{c}{p}_t(\mathit{\text{NC}}|\mathit{\text{NC}},B)=(1-d_t)(1-w_t),\hfill \\ p_t(C|\mathit{\text{NC}},B)=(1-d_t)w_t,\hfill \\ p_t(D|\mathit{\text{NC}},B)=d_t,\hfill \\ p_t(C|C,B)=(1-f)(1-d_t)(1-e_t),\hfill \\ p_t(T|C,B)=f(1-b_t)(1-d_t),\hfill \\ p_t(D|C,B)=d_t+{\mathit{\text{fb}}}_t(1-d_t)+e_t(1-f)(1-d_t),\hfill \\ p_t(T|T,B)=(1-d_t)(1-b_t),\hfill \\ p_t(D|T,B)=d_t+b_t(1-d_t),\hfill \\ p_t(D|D,B)=1.\hfill \end{array}$$

We denote the transition probability matrix at epoch t given the decisions not to biopsy immediately, DB and DP, as P_t(s_t+1|s_t, DB) and P_t(s_t+1|s_t, DP) consisting of elements p_t(s_t+1|s_t, DB), ∀s_t+1 ∈ S, s_t ∈ S, and p_t(s_t+1|s_t, DP), ∀s_t+1 ∈ S, s_t ∈ S, where the non-zero elements are:

$$\begin{array}{c}{p}_t(\mathit{\text{NC}}|\mathit{\text{NC}},\mathit{\text{DP}})=p_t(\mathit{\text{NC}}|\mathit{\text{NC}},\mathit{\text{DB}})=(1-d_t)(1-w_t),\hfill \\ p_t(C|\mathit{\text{NC}},\mathit{\text{DP}})=p_t(C|\mathit{\text{NC}},\mathit{\text{DB}})=(1-d_t)w_t,\hfill \\ p_t(D|\mathit{\text{NC}},\mathit{\text{DP}})=p_t(D|\mathit{\text{NC}},\mathit{\text{DB}})=d_t,\hfill \\ p_t(C|C,\mathit{\text{DP}})=p_t(C|C,\mathit{\text{DB}})=(1-d_t)(1-e_t),\hfill \\ p_t(D|C,\mathit{\text{DP}})=p_t(D|C,\mathit{\text{DB}})=d_t+e_t(1-d_t),\hfill \\ p_t(T|T,\mathit{\text{DP}})=p_t(T|T,\mathit{\text{DB}})=(1-d_t)(1-b_t),\hfill \\ p_t(D|T,\mathit{\text{DP}})=p_t(D|T,\mathit{\text{DB}})=d_t+b_t(1-d_t),\hfill \\ p_t(D|D,\mathit{\text{DP}})=p_t(D|D,\mathit{\text{DB}})=1.\hfill \end{array}$$

Note that the last 3 elements of p_t(s_t+1|s_t, B), p_t(s_t+1|s_t, DB) and p_t(s_t+1|s_t, DP) are identical because they are independent of the decision. From this point forward, we write them as p_t(T|T), p_t(D|T), and p_t(D|D), respectively.

Reward function

Following are the annual rewards associated with the societal perspective (note that the patient perspective is a special case in which cost c_p = c_b = c_t = 0 and the willingness to pay β = ∞). The rewards given the decision to biopsy, B, are:

$$\begin{array}{c}{r}_t(\mathit{\text{NC}},B)=\beta (1-\mu )-c_b,\hfill \\ r_t(C,B)=\beta (1-\mu -\delta (1-f)-f_{\epsilon })-c_b-f·c_t,\hfill \\ r_t(T)=1-\epsilon ,\hfill \\ r_t(D)=0.\hfill \end{array}$$

We define the following rewards given the decision is DB:

$$\begin{array}{c}{r}_t(\mathit{\text{NC}},\mathit{\text{DB}})=\beta -c_p,\hfill \\ r_t(C,\mathit{\text{DB}})=\beta (1-\delta )-c_p,\hfill \\ r_t(T)=1-\epsilon ,\hfill \\ r_t(D)=0.\hfill \end{array}$$

The rewards given the decision is DP, are:

$$\begin{array}{c}{r}_t(\mathit{\text{NC}},\mathit{\text{DP}})=\beta ,\hfill \\ r_t(C,\mathit{\text{DP}})=\beta (1-\delta ),\hfill \\ r_t(T)=1-\epsilon ,\hfill \\ r_t(D)=0.\hfill \end{array}$$

The rewards defined above reflect the following assumptions. All the utility parameters, w_t, d_t, b_t, e_t, f, μ, δ and ε, have values in [0, 1]. A patient with prostate cancer that has not been detected may suffer side effects that degrade their quality of life by δ ≥ 0. A patient who has a biopsy has his quality of life reduced by μ in the year of biopsy. In other words μ reflects the reduction in quality of life due to the biopsy procedure including factors such as pain, anxiety, and short term procedure side effects such as infection. A patient who has a positive biopsy and is treated suffers a loss of ε QALYs for all future life years, i.e., ε reflects reduced quality of life due to side effect of treatment. Note that we are implicitly assuming that the utility decrements, μ, ε, δ, are based on population averages, and the decision maker is risk neutral. The reward in QALYs are then translated into monetary values by multiplying by β.

2 Bayesian updating

Bayesian updating is used to revise the patient’s belief state over time as PSA observations are obtained. Bayesian updates are defined by the following transformation of the belief state:

$${\pi }_{t+1}(s_{t+1})=\frac{q_{t+1}({\ell }_{t+1}|s_{t+1}){\displaystyle \sum _{s_t\in S}}{p}_t(s_{t+1}|s_t,a_t){\pi }_t(s_t)}{{\displaystyle \sum _{s_{t+1}\in S}}{q}_{t+1}({\ell }_{t+1}|s_{t+1}){\displaystyle \sum _{s_t\in S}}{p}_t(s_{t+1}|s_t,a_t){\pi }_t(s_t)},$$

where the posterior belief of being in state s_t+1, π_t+1(s_t+1), the component of the vector π_t+1, is a function of PSA observation ℓ_t+1, action a_t and the prior belief vector π_t. Thus, this equation provides a means to update the belief state of a patient based on their prior belief state and their most recent observed PSA interval. For instance, when the action is DP and the PSA test result falls into interval ℓ_t+1, the posterior belief of having prostate cancer is

$${\pi }_{t+1}(C)=\frac{q_{t+1}({\ell }_{t+1}|C){\displaystyle \sum _{s_t\in S}}{p}_t(C|s_t,a_t){\pi }_t(s_t)}{{\displaystyle \sum _{s_{t+1}\in S}}{q}_{t+1}({\ell }_{t+1}|s_{t+1}){\displaystyle \sum _{s_t\in S}}{p}_t(s_{t+1}|s_t,a_t){\pi }_t(s_t)}.$$

3 Optimality equation

Prostate cancer screening is a two-stage decision problem. The optimality equations can be written as

$${\upsilon }_t({\pi }_t(C))=\text{max }\{R_t({\pi }_t(C)),{\upsilon }_t({\pi }_t(C),\mathit{\text{DB}}),{\upsilon }_t({\pi }_t(C),\mathit{\text{DP}})\},\forall t,\forall {\pi }_t(C)\in [0,1],$$

where υ_t(π_t(C)) is the expected future reward for a patient age t with probability of prostate cancer, π_t(C); υ_t(π_t(C), DP) is the expected future reward given a_t = DP at decision epoch t, which can be written as:

$${\upsilon }_t({\pi }_t(C),\mathit{\text{DP}})=r_t({\pi }_t(C),\mathit{\text{DP}})+{\upsilon }_{t+1}({\pi }_{t+1})p_t({\pi }_{t+1}|{\pi }_t(C),\mathit{\text{DP}});$$

υ_t(π_t(C), DB) is the expected future reward given a_t = DB at decision epoch t, which can be written as:

$${\upsilon }_t({\pi }_t(C),\mathit{\text{DB}})=r_t({\pi }_t(C),\mathit{\text{DB}})+{\displaystyle \sum _{{\ell }_{t+1}\in M}}{\upsilon }_{t+1}({\pi }_{t+1})p_t({\ell }_{t+1}|{\pi }_t(C),\mathit{\text{DB}});$$

R_t(π_t(C)) is the expected future reward given a_t = B at decision epoch t, which can be written as:

$$R_t({\pi }_t(C))=(1-{\pi }_t(C)){\overline{R}}_t(\mathit{\text{NC}})+{\pi }_t(C)((1-f){\overline{R}}_t(C)+f{\overline{R}}_t(T))-\beta \mu -c_b,$$

where R̄_t(NC), R̄_t(C) and R̄_t(T) denote the expected future rewards under the policy of never referring the patient for PSA tests and biopsy after age t for states NC, C and T, respectively, which can be written as:

$$\begin{array}{c}{\overline{R}}_t(\mathit{\text{NC}})=r_t(\mathit{\text{NC}},\mathit{\text{DP}})+(p_t(\mathit{\text{NC}}|\mathit{\text{NC}},\mathit{\text{DP}}){\overline{R}}_{t+1}(\mathit{\text{NC}})+p_t(C|\mathit{\text{NC}},\mathit{\text{DP}}){\overline{R}}_{t+1}(C)),\hfill \\ {\overline{R}}_t(C)=r_t(C,\mathit{\text{DP}})+p_t(C|C,\mathit{\text{DP}}){\overline{R}}_{t+1}(C),\hfill \\ {\overline{R}}_t(T)=r_t(T)+p_t(T|T){\overline{R}}_{t+1}(T).\hfill \end{array}$$