Challenging the 10-year rule: The accuracy of patient life expectancy predictions by physicians in relation to prostate cancer management

Abstract

Introduction:

We assess physicians’ ability to accurately predict life expectancies. In prostate cancer this prediction is especially important as it affects screening decisions. No previous studies have examined accuracy in the context of real cases and concrete end points.

Methods:

Seven clinical scenarios were summarized from charts of deceased patients. We recruited 100 medical professionals to review these scenarios and estimate each patient’s life expectancy. Responses were analyzed with respect to the patients’ actual survival end points, then stratified based on the demographic information provided.

Results:

Respondent factors, such as sex, level of training, location of work or specialty, made no significant difference on prediction accuracy. Furthermore, respondents were typically pessimistic in their estimations with a negative linear trend between estimated life expectancy and actual survival. Overall, respondents were within 1 year of actual life expectancy only 15.9% of the time; on average, respondents were 67.4% inaccurate in relation to actual survival. If framed in terms of correctly identifying which patients would live more than or less than 10 years (dichotomous accuracy), physicians were correct 68.3% of the time.

Conclusions:

Physicians do poorly at predicting life expectancy and tend to underestimate how long patients have left to live. This overall inaccuracy raises the question of whether physicians should refine screening and treatment criteria, find a better proxy or dispose of the criteria altogether.

Introduction

Decisions on patient management are often predicated by the physician’s own clinical judgement. These decisions become increasingly difficult when factors, such as age of the patient, quality of life, comorbidities, tolerance to the treatment and long-term side-effects, become significant. In some conditions, such as prostate cancer, understanding the value of clinical judgement becomes crucial when deciding whether or not to screen.

In 2006, it was reported that life expectancy for males at age 65 is on average 19.9 years in Canada (ranging from 16.8–20.5 among the provinces and territories) and 17.0 years in the United States.^1,2 Prostate cancer is the most prevalent cancer among men and represents 38% of all cancers.³ The lifetime probability of developing prostate cancer is 13.6% (1 in 7.4 men); 25 500 new cases are diagnosed every year in Canada.³ Furthermore, the lifetime chance of dying from prostate cancer is 3.7%.³

In terms of screening, though there is significant controversy surrounding its value and need (Appendix A), those that do describe it often cite that patients should have at least a 10-year life expectancy.^4–7 Accuracy and precision regarding life expectancy are therefore important. In this regard, we wanted to see which physician groups do better at predicting life expectancy and we wanted to evaluate how well physicians predict life expectancy.

Methods

Respondents

Respondents were randomly selected within demographic groups of interest. Specifically, we restricted our respondents to attending physicians and residents in urology, internal medicine and family medicine, as well as medical students. We included respondents who would be dealing with prostate cancer screening. All three levels of training were included to be able to later analyze if the accuracy of life expectancy predictions improved with training and experience. To maintain a high response rate, surveys were distributed directly by the primary author. A goal of 100 respondents was set and met.

Questionnaire content

Demographic information was collected and included gender, level of training, years in practice (if any), subspecialty (if any), place of work (if any), medical school, residency school and program (if any), and fellowship school and program (if any). We also presented seven case summaries and asked respondents to provide their estimation of life expectancy based on the information provided. These cases were based on data of real deceased patients (Appendix B).

Statistical analysis

A priori sample size calculations were not undertaken, as this was an observational study that did not involve a specific intervention or effect size. Rather, a convenience sample consisting of the first 100 respondents from the attending physicians and residents in urology, internal medicine, and family medicine, as well as medical students, was obtained. A sample size of 50 is commonly considered the minimum number of patients required for testing associations;⁸ therefore the current sample size of twice that is believed to be sufficient to support the use of inferential statistics.

Data were collected in Microsoft Excel 2007 and statistical analyses were performed using SPSS version 17.0 (SPSS, Cary, NC). Predicted life expectancy was subtracted from the actual life expectancy for all respondents and all cases, so that underestimated life expectancies were represented by negative values. Following a descriptive analysis, independent samples t-tests were used to compare the actual scores for 2-level variables, such as sex, while one-way analysis of variance was used to compare multilevel variables, such as level of training, specialty and working environment. Chi-square tests were used to compare the aggregated values and the characteristics of the respondents. Responses for the 7 cases for all 100 respondents were also aggregated for a sample size of 700 to assess the differences between estimated and actual life expectancy, and the association of the difference scores to actual survival time (Spearman rank correlation). These data were also graphed to further illustrate the relationship between estimated life expectancy and survival time, to determine if physicians were more likely to underestimate life expectancy as survival time increased.

To calculate the dichotomous accuracy of life expectancy predictions (Fig. 1), the responses of the 7 cases of all 100 respondents were again aggregated to form a sample size of 700. Using Microsoft Excel 2007, each life expectancy prediction was simplified to either a true or false value to the question “Is this value greater or equal to 10 years.” For each case, these predictions were then dichotomously separated into whether they were correct when compared to the actual survival of the case study’s patient. The aggregate average of correct responses over the 700 responses was then calculated to give the dichotomous accuracy of 10 years. This same technique was then applied across all life expectancy cut-offs, simulating different life-expectancy cutoffs, from 0 years to 25 years at 0.5 year increments. A line of best fit was then graphed to better illustrate the trend of dichotomous accuracy across the spectrum of cut-off years.

Fig. 1.

The dichotomous accuracy of life expectancy predictions in relation to a spectrum of life expectancy cut-offs. As actual survival moves towards the 10 year mark, dichotomous accuracy decreases precipitously; however, as actual survival increases past this point, dichotomous accuracy slowly climbs back up to near 100%.

This analysis was taken a step further to calculate the non-dichotomous accuracy of life expectancy predictions by calculating the deviance of each prediction in relation to the actual survival of each case. These data were calculated as a both a percentage and number value with analyses in terms of both actual and absolute values. The mean and standard deviation for all cases was then calculated from these results. For interest, the sample size of 700 was also stratified by stage of training, and the best and worst predictions were identified from within the pool.

Results

Response

Surveys were distributed and collected personally; of the 127 surveys distributed, 100 surveys were collected. This translates to a response rate of 78.7%. Of those who responded, most were male urologists (47%). Some statistics, such as medical school, residency school and program, fellowship school and program, were too stratified for analysis and therefore left out (Table 1).

Table 1.

Descriptive statistics

Characteristics		n	Percent
Total		100
Gender	Male	75	75.0%
	Female	25	25.0%
Level of training	Medical student	17	17.0%
	Resident	25	25.0%
	Attending physician	58	58.0%
Workplace	Community	24	24.0%
	Academic centre	57	57.0%
	Not working	19	19.0%
Specialty	Family medicine	14	14.0%
	Internal medicine	15	15.0%
	Urology	50	50.0%
	Undeclared	21	21.0%

Comparative differences between groups

Analysis was done to compare outcomes between groupings. The four factors that were analyzed based on adequate numbers were sex, level of training, location of work and specialty. In terms of sex, we found that there was no significant difference in predictive accuracy between men and women (p = 0.27). Similarly, level of training, when broken down into medical student, resident, and attending physician (including all physicians who have completed residency), yielded no significant difference in predictive accuracy (p = 0.46). Location of work did not affect the ability to predict life expectancy (p = 0.48), and no particular specialty did significantly better (p = 0.24) (Table 2). However, post-hoc power analyses indicated that the study was not adequately powered to detect differences between these groups, with power ranging from a low of 8% for the comparison of medical student, resident and attending, to 31% for the comparison of men and women. These findings therefore need to be interpreted within the context of the limited study power.

Table 2.

Predictive accuracy between groups

Group		n	Mean accuracy*	95% CI		p value
				Lower	Upper
Sex	Male	75	−1.84	−0.46	1.63
	Female	25	−2.43	−0.46	1.63
						0.27
Level of training	Medical student	17	−2.29	−3.39	−1.19
	Resident	25	−1.51	−2.39	−0.62
	Attending	58	−2.11	−2.71	−1.50
						0.46
Place of work	Community	24	−2.43	−3.31	−1.54
	Academic centre	56	−1.76	−2.39	−1.14
	Not working	18	−2.16	−3.22	−1.10
						0.48
Residency program	Urology	50	−1.88	−2.52	−1.25
	Family medicine	14	−2.48	−3.79	−1.17
	Internal medicine	15	−1.05	−2.23	0.14
						0.24

^*Values represent the difference between the predicted life expectancy and the actual life expectancy, with negative values representing underestimated life expectancies. CI: confidence interval.

Accuracy of prediction overall

General trends in responses showed that medical professionals generally underestimate their patients’ life expectancy. Across all 700 responses to the clinical scenarios, the difference between the actual and estimated life expectancy ranged from −19.3 to 13.6, with a mean of −2.0 years (standard deviation [SD] 6.1), The coefficient relating actual survival time to estimated survival time was −0.57, (p < 0.001), suggesting that physicians are more likely to underestimate life expectancy as actual survival time increases.

To understand how well we do overall at estimating life expectancy in the context of screening guidelines, a dichotomous accuracy analysis was done, plotting the percentage of correct responses against actual life expectancy cut-offs (Fig. 1). For example, if we categorized answers as being ≥10 years or <10 years and compared them to the patients’ actual survival time, overall, medical professionals predicted correctly only 68.1% of the time; within the overall result, medical students scored 75.6%, residents 70.9% and attending physicians 64.8%. Interestingly, when graphed with a line of best fit, we found that the life expectancy cut-off at which medical professionals predict the least accurately was at exactly 10 years.

In investigating how accurate we are in our non-dichotomous life expectancy predictions versus actual survival, we considered each case response as an individual instance rather than as a case within a series (n=700). We found that, overall, if we considered only the absolute values of our error, we were 67.4% inaccurate on average compared to actual survival (SD=90.4%). Within that average, medical students were 77.5% inaccurate (SD=106.4%); residents were 68.6% inaccurate (SD=97.0%); attending physicians were 63.8% inaccurate (SD=81.9%). Overall, the range of inaccuracy per respondent ranged from 26.7% to 149.3% in the predictions of the actual survival over the 7 case scenarios, with both the best and worst predictors being attending physicians (Table 3). Another way of expressing this is in how many years off our predictions are from actual survival. We also stratified the predictions by group (Table 2). Overall, respondents underestimated life expectancy by an average of 2.0 years (SD=5.2).

Table 3.

Accuracy of life expectancy predictions vs. actual survival (calculated using absolute predictive error)

Group	Average	Standard deviation	Best predictor’s accuracy on average	Worst predictor’s accuracy on average
Medical student	77.5%	106.4	38.5	147.9%
Resident	68.6%	97.0	40.1	111.7%
Attending physician	63.8%	81.9	29.7	149.3%
Overall	67.4%	90.4	29.7	149.3%

Finally, though the analysis did not carry enough power of response, there was a positive trend between years in practice and non-dichotomous accuracy of life expectancy prediction (p = 0.122).

Discussion

Use for screening

The use of screening tests for prostate cancer has become controversial and, in response, several official guidelines have been updated (Appendix A). While most agencies either cite an inadequacy of information to choose a side or provide limited partial guidelines, agencies that do describe the screening test recommend that patients have at least a 10-year life expectancy for them to benefit from the screening; 80% of Canadian specialists concur with this cut-off.⁹

Similarly, there has also been controversy over whether patients over a certain age should be screened at all. This is because of their lesser likelihood to fulfill this “10-year rule” and benefit from treatment, even though it has been shown that survival after initial diagnosis is on average 15 to 20 years.⁴ Interestingly, Pearce and colleagues found that most urologists recommend screening patients up to 70 years old, while most radiation oncologists continue screening past 80.⁹ Contrarily, the U.S. Preventative Task Force currently advocates that patients over 75 years old should not be screened,¹⁰ though this recommendation might soon change to recommend against screening at all, regardless of age.¹¹ The result of this type of cap is that potential patients who might benefit from screening and treatment are arbitrarily excluded, despite the fact that their life expectancies might exceed 10 years.

Recently, two large long-term studies have also offered guidance on the use of screening. It is interesting to note, however, that both study groups strictly limited their patient population to men who fell well within the traditional screening ages of 50 and 75 (ages that were chosen to reflect the 10-year rule): 55 to 74 in the PLCO (Prostate, Lung, Colorectal and Ovarian) trial and 55 to 69 in the ERSPC (European Randomized Study of. Screening for Prostate Cancer) trial.^12,13 This is significant because it means that the way we measure the effectiveness of screening might be inherently faulty since we apply and study the effects of screening on a population that was chosen by fundamentally arbitrary mechanisms. Because of this general uncertainty, physicians are left with little guidance, save their personal opinions, as to how and if they should screen.

Life expectancy predictions

While the “10-year rule” is suspect, the question remains that if we were to include it in our screening protocol, how well can we apply it? Because this protocol can also affect treatment decisions, those who have to apply the “10-year Rule” include a wide range of physicians, such as primary care physicians, internists, urologists and radiation oncologists. Our results are based on a paper presentation of cases which have been shown by Kirwan and colleagues to correlate well with in-person assessments.¹⁴ They show that physicians are poor predictors of life expectancy, with no significant distinction between groups, such as sex, level of training, location of work or medical specialty. In addition, when combining our results with the literature on whether physicians overestimate or underestimate life expectancy, we find that the variability suggests no particular favour either way. The same applies to the accuracy relative to a physician’s number of years in practice.^15–18 Wilson and colleagues summarize this well by arguing that clinicians are not only inaccurate, but also imprecise and inconsistent with their predictions.¹⁶

Dichotomous accuracy

Physicians in modern and historic times have had to estimate how long patients have left to live. “Yes” and “No” answers to such questions as “Do I have 10 more years to live?” demonstrate the concept of dichotomous accuracy. Even states such as death have historically been a challenge to confirm. As patients’ actual survival moves away from 0 years, our analysis shows that physicians get progressively worse in predictive dichotomous accuracy. Interestingly, this decrease in the ability to estimate life expectancy only continues up to a point: 10 years. After this point, as actual survival continues to climb, the dichotomous accuracy slowly climbs back up to almost 100% accuracy. This might be due to the fact that life expectancies of patients at the ends of the spectrum, who are immediately sick or relatively healthy, are easier to assess as they are on more defined paths of well-being, a phenomenon that can be described as a “horizon effect;” however, moderately sick patients tend to occupy a grey zone of well-being and thus their life expectancy becomes a challenge to predict.

For this particular study, we questioned whether the 10-year mark is a valid standard for use as a screening criterion. In our study, physicians correctly estimated patients’ 10-year life/death status 68.14% of the time. Interestingly, the study by Walz and colleagues yielded a similar result at 10 years of 68% mean accuracy; physicians’ ability to predict patients’ 10-year life expectancy was “moderate at best.”¹⁸ While the judgement call of whether 68.1% accuracy is passable or not is subjective, we found that physicians’ ability to predict patients’ 10-year life expectancy was in fact the worst.

Other options

If clinical life expectancy prediction fails as a criterion, then what other options are we left with? One possible method is a mathematical model to predict life expectancy. More than 10 models have been proposed, including complex actuarial insurance models. Their precisions vary, from being highly synchronous with physician predicted life expectancy to yielding only modest success with none of them providing any additional benefit or significantly better predictions. Furthermore, in every case studied, these models were measured relative to hypothetical predictions rather than actual end points.^18–23

Another method proposed is age cut-offs. However, as discussed above, this similarly arbitrary cut-off not only categorizes many potential patients to inappropriate management, but has not been fully studied aside from retrospective studies. Currently, there is little tolerance of “age-ism” and so this would likely not be a viable alternative.

Limitations

One limitation of the study is the use of a convenience sample of the first 100 physicians and medical students who agreed to participate. While the study does have sufficient numbers to support the use of inferential statistics, it was under-powered to detect between-group differences such as the comparisons for men and women, level of training, specialty and working environment. Another limitation includes the case studies themselves; more information can influence a decision, such as if the patient looks ill or not. One must also balance this, however, with the reality that we never have perfect information.

Conclusions

Although physicians might be poor predictors of life expectancy, there may not be any significantly better alternatives at the moment. Our study implies that Canadian physicians are more likely to underestimate life expectancy. If this is true, then we are also missing a population of men who indeed have 10 years or more to live and who may benefit from screening. Further study in methods to improve our accuracy and determine the chances of death from competing causes is an understudied aspect of potential prostate cancer patients.

Appendix A.

Current screening protocol for all men for prostate cancer, abridged collection

Organization	Year	Guideline
World Health Organization24	2009	Cannot make recommendations before more trials are done.
American Urological Association4,5	2009	Should be offered to asymptomatic men 40 or over with an estimated life expectancy of at least 10 years.
Canadian Urological Association25	2011	PSA testing should be offered to all men 50 years of age or older with a life expectancy of at least 10 years.
American Cancer Society6	2010	Does not support routine PSA and DRE testing but recommends that it be discussed with and offered to men 50 year and at least a 10 year life expectancy.
Canadian Cancer Society26	2009	Discuss the risks and benefits of DRE and PSA for men over 50 (40 if at risk).
US Preventive Services Task Force9,10	2008	Not enough evidence for or against routine screening in men under 75. Recommends against screening in men over 75. *New 2011 Draft proposes no screening at all in asymptomatic men, regardless of age, race, or family history.
Canadian Task Force on the Preventative Health Care27	1994	Insufficient evidence to include or exclude DRE, so advises to continue whatever physicians’ current practice is. Does not support PSA testing.
National Cancer Institute (US)28	2009	No formal guidelines; awaiting more clinical trial evidence.

PSA: prostate-specific antigen; DRE: digital rectal examination.

Appendix B.

Patient life expectancy assessment survey questionnaire

1. Gender (Please circle one):	Male					Female
2. Occupation (Please circle one):
Attending physician	Resident					Medical student
	PGY1	PGY2	PGY3	PGY4	PGY5	Year 1	Year 2	Year 3	Year 4	Year 5
3. Number of years in practice:
4. Subspecialty, if any:
5. Where do you work primarily (Please circle one):
Community	Academic centre	Not working
6. Medical school:
7. Residency location:
8. Residency program (i.e., Internal, Urology, Family, etc):
9. Fellowship location:
10. Fellowship program:
For each of these scenarios, respondents were asked:
According to your clinical judgement, how many more years do you think this patient has to live? Number of Years left to live:
Clinical scenario 1				Clinical scenario 2
Patient is a 67 year old gentleman who was recently transferred from another hospital. His problem list includes ischemic and diabetic nephropathy, left nephrectomy for adenocarinoma of the kidney, angiodysplasia of the colon, abdominal aortic aneurysm repair with bypass and cholecystectomy, 70% stenosis of the left coronary artery, dyslipidemia, antibodies to Heparin with negative HITT assay, and GI hemorrhage due to ulcers. Medications include Norvasc 5mg PO OD, Metoprolol, 100mg PO BID, Pantoloc 40mg PO OD, Calcium 1g PO TID, Alprazolam 0.25 mg TID, Fenofibrate 1 capsule OD, Humulin N 26 units, Humulin R 14 units, Eprex 4000 units 3 times/week.				Patient is a 70 year old obese gentleman who presents with increasingly severe urinary tract obstructive symptoms. Past medical history includes two years of controlled type II diabetes, gout, and a hiatus hernia repair when he was 24 years old. Medications include Tolbutamide 500mg BID for his diabetes, Allopurinol, and Iron supplements.
Clinical scenario 3				Clinical scenario 4
Patient is a 54 year old gentleman who presents with recent chest pains lasting 45 minutes associated with diaphoresis, SOB, and presyncope. Patient was recently transferred from the psychiatric ward where he spent 11 days after being admitted with an overdose and suicidal tendencies. BP was elevated at 169/98, however the rest of the physical exam was unremarkable. Past medical history includes osteoporosis, osteoarthritis of the spine, coronary artery disease including stable angina and two past MIs 9 and 7 years ago, peripheral vascular disease, depression, and alcohol abuse for which he is practicing abstinence. Current medications include Cardizem 300mg BID, GTN (glyceryl trinitrate) patch 0.4mg x 12hr/day, enteric-coated Aspririn, and NSAIDs.				Patient is a 54 year old gentleman who presents with a gastric ulcer and a secondary lower GI bleed with a one month history of symptoms. The pain is dull, constant, present throughout, independent of meals, and often waking him at night. Recently, he has complained of increasing fatigue and weakness and noted 8 days of black stool with no visible blood. On physical exam, the liver was palpable 2 cm below the costal margin, and stool was positive for occult blood. The remainder was unremarkable. Past medical history includes alcohol abuse, hypertension, and chronic anxiety. Medications include Ranitidine 150 mg PO BID and Diltiazem SR 90 mg PO BID.
Clinical scenario 5				Clinical scenario 6
Patient is a 66 year old gentleman who presents with exertional dyspnea which he has had for the last 10 years but which has worsened significantly over the past 6 months. He was previously able to walk and carry wood without difficulty, but now becomes dyspneic from walking across the parking lot to the hospital. He has no symptoms of chest discomfort, no problems sleeping, and a normal stress test. In the past 6 months, he has had thrombosis of his left arm fistula formerly for dialysis, marked increase in gastroesophageal reflux symptoms, and marked right eye vision reduction which is under investigation. Past medical history includes LVH, CHF, an LAD angioplasty 6 years ago, a renal transplant 4 years ago, and severe hypertension. Medications include Cyclosporin, 100mg BID, Mycophenolate mofetil 500mg BID, Prednisone 5mg OD, Diltiazem 360mg OD, Atorvastatin 10mg OD, Cozaar 25mg OD, Nexium 40mg OD, Domperidone 10mg q6h, and Lasix 20mg OD.				Patient is a 69 year old gentleman who presents shortly after having had a total knee replacement following 10 years of osteoarthritic symptoms in his right knee. Physical exam was unremarkable. Past medical history includes 19 years of type II diabetes mellitus, appendectomy, and an operation for a bowel obstruction 54 years ago. Medications include Insulin, Tylenol 3, Coumadin, and Metamucil. Patient currently has Home Care physiotherapy and nursing support.
Clinical scenario 7
Patient is a 55 year old gentleman who presents with chest pain, diaphoresis, and shortness of breath after an exacerbation of asthma. These were ultimately relieved with sublingual Nitroglycerin and Ventolin. Other than use of accessory muscles on inspiration and a hyperinflated chest, physical exam was unremarkable. Cardiac workup was negative. There is no family history of hypertension, diabetes, or cardiovascular disease. The patient has a 20 pack-year smoking history but quit 6 years ago. Past medical history includes COPD with Reactive Airway Disease, Adult Respiratory Distress Syndrome, Aspiration pneumonitis, DVT. Medications include Theophylline 400mg PO BID, Coumadin 15mg PO OD, Nitroglycerin 0.3mg SL PRN, Salbutomol inhaler 2 puffs QID, Diltiazem 60mg PO QID.