How Accurate Are Physicians in the Prediction of Patient Survival in Advanced Lung Cancer?

The study compared actual with predicted survival estimates in advanced stage non-small cell lung cancer patients. Regardless of years of experience, physicians overestimated the survival duration of these patients.

Abstract

Background.

Because most cases of non-small cell lung cancer (NSCLC) are diagnosed at an advanced stage with a poor prognosis, patient inclusion in clinical trials is critical. Most trials require an estimated life expectancy >3 months, based on clinician estimates of patient survival probability, without providing formal guidelines. The aim of this study was to assess the accuracy of clinicians' predictions of survival in NSCLC patients (stages IIIB, and IV) and the possible impact of patient quality of life on survival estimation.

Methods.

At diagnosis, clinical, biological, and quality of life data (QLQ-C30 questionnaire) were recorded, and doctors “forecast” each patient's estimated survival. Concordance between predicted and actual survival was assessed with the intraclass correlation coefficient.

Results.

Eighty-five patients with a mean age of 62.2 years, 81.1% male, were included (squamous cell carcinoma, 33; adenocarcinoma, 42; large cell carcinoma, 8; neuroendocrine carcinoma, 2). The mean follow-up was 40 months and median survival time was 11.7 (range, 0.4–143.7) weeks. All clinicians (residents, registrars, and consultants) overestimated patient survival time, with a moderate concordance between predicted and actual survival time. A worse global health status was associated with a lower discrepancy between estimated and actual patient survival, and a worse role functioning was associated with a larger difference between estimated and actual patient survival.

Conclusion.

The absence of specific recommendations to estimate patient survival may introduce major selection in clinical studies. Further research should investigate whether the accuracy of patient survival estimates by clinicians would be improved by taking into account patient quality of life.

Introduction

Lung cancer is the leading cause of cancer death in developed countries, and its global incidence continues to rise [1]. The overall 5-year survival rate for lung cancer patients is about 15% [2]. Prognosis is directly related to tumor–node–metastasis (TNM) staging, and is extremely dismal when the diagnosis is made at an advanced stage (IIIB or IV) [2]. Indeed, surgery is the only treatment capable of providing real cure, with 5-year survival rates of 70% for stage I, 45% for stage II, and 10%–30% for stage IIIA patients. In contrast, when the diagnosis is made at stage IIIB or IV, the 5-year survival rates, are, respectively, 10% and <5%. This is a result of the fact that current chemotherapies and/or radiotherapy are only palliative treatments, although they can increase both survival and quality of life [3, 4].

Given this dearth of treatment options, all patients with stages IIIB and IV lung cancer should be included, when possible, in clinical trials. Interestingly, most, if not all, clinical trials require an estimated minimum patient life expectancy >3 months as an inclusion criterion, in order to see a benefit from treatment [5]. However, these trials often fail to provide any formal criteria to help make this survival estimate, and it is usually simply based on subjective clinician estimate [6–8]. The lack of guidelines or proposed specific criteria for the predictive estimate of patient survival is likely to introduce some imprecision in these protocols.

This study was conducted in order to evaluate the accuracy of clinicians' survival estimates and the possible impact of patient quality of life on survival evaluation. We evaluated the predictive accuracies of several doctors with different levels of experience in taking care of patients by comparing their survival predictions with actual patient survival times. Furthermore, several biological markers [9–14] have been related to patient survival. Because the perception of the disease is also known to affect mortality in many cancers [15], we assessed how this might have been taken into account by doctors to estimate their patients' survival.

Methods

Study Design

All patients meeting the inclusion criteria (histological type, TNM staging, World Health Organization [WHO] performance status [PS] score) and hospitalized in the chest disease department at the Nancy teaching hospital were included in a cohort study from September 2001 to May 2004. The eligibility criteria were stage IIIB or IV NSCLC. In this department, ambulatory patients are referred to senior physicians only, whereas hospitalized patients are referred to all physicians (residents, registrars, or consultants). Data were collected by a research nurse from medical records and from doctors for survival prediction. Patient follow-up lasted until September 2006, with death as the outcome. All patients signed an informed consent form.

Data Collected

In several studies, clinical and biological data like age [11], sex [9], histological typing, WHO PS score [16, 17], TNM staging [12, 18], weight (kg) [16], weight lost (% of usual weight) [16], blood hemoglobin (g/dl) [12], leukocytes (× 10⁹/l) [11], platelets (× 10⁹/l) [13], proteins (mg/l), albumin (mg/l) [19], calcium (mg/l) [12], and lactate dehydrogenase (LDH) (U/l) [12] were found to be prognostic factors in NSCLC. All these data were recorded. Quality of life was assessed using the QLQ-C30 questionnaire [20]. This questionnaire consists of 30 items, in six scales: global health status, physical functioning scale, role functioning scale, emotional functioning scale, cognitive functioning scale, and social functioning scale. Each scale was scored from 0.0 to 100.0; the lower the score, the better the quality of life. For each patient, the residents, registrars, and consultants in charge were required to fill in a form, in an independent and secret way, within a week of pathological diagnosis, prior to treatment decision, indicating their “forecast,” that is, estimate of survival of their patients. This estimate was put in a sealed envelope and transmitted to the data entry centre (Centre d'Investigation Clinique – Epidémiologie Clinique). Physicians were not aware of the results of formal quality-of-life assessments, because predictions were made independently of questionnaire collection, and quality of life scores were calculated in the statistician's office only at the end of the study.

Patients were then treated with chemotherapy, radiotherapy, and/or palliative care according to current international guidelines [3–5]. No treatment data were recorded.

In France, residents (“interns”) have just finished their medical school training and are directly in charge of patients; internships last for 4 years. They take care of the patient daily and are supervised by registrars (“chefs de clinique-assistants des hôpitaux”) who complete their specialist training within 2–4 years. The registrars help the residents in the care of patients but don't examine the patient every day. Finally, consultants (“praticiens hospitaliers”) are senior doctors responsible for each ward and are not involved in the daily clinical care. Five consultants, three registrars, and four residents contributed to this study. As a mean, consultants had ∼10 years of experience. Registrars had a mean clinical experience of 2 years plus their 4 years experience as a resident, and residents had a mean clinical experience of ∼2 years.

Follow-Up

A regular follow-up was organized for each patient including the date of death. The last patient was included in May 2004 and follow-up was stopped in September 2006. Thus, the minimum follow-up for each patient was 28 months. Patient survival was measured from the day of histopathological diagnosis to the date of death, which was obtained from death certificates.

Ethical Approval

Our study was approved by the regional “ethical committee on biomedical research involving human subjects” (Comité Consultatif de Protection des Personnes dans la Recherche Biomédicale de Lorraine) on August 2001.

Statistical Analysis

Characteristics of patients were described by the mean (± standard deviation) and percentage for continuous and categorical data, respectively, at study entry, and median actual survival times were estimated from Kaplan–Meier survival curves. A Cox regression model was used to analyze which variables, according to factors known in the literature, were associated with actual survival.

Patients with complete data at entry and follow-up were compared with those excluded (lack of follow-up, missing data) using the Kruskal-Wallis test for continuous variables and the Fisher exact and χ² tests for categorical data.

The median predicted survival time for each physician category was estimated from respective Kaplan–Meier survival curves.

Concordance between actual and predicted survival time was assessed by the intraclass correlation coefficient (ICC) and its 95% confidence interval CI derived from an analysis of variance model with a physician random effect. ICCs were interpreted according to Landis and Koch [21].

Statistical tests were interpreted at a type I error of α = 0.05.

All analyses were conducted using SAS 9.1 software (SAS Institute, inc., Cary, NC).

Results

Sample Initially Screened

In total, 139 patients referred to our department from September 2001 to May 2004 for advanced lung cancer were initially screened for this study. The main patient characteristics at time of diagnosis are detailed in Table 1. The male–female ratio was 114/25, mean age was 61.9 ± 11.5 years, and mean weight was 69.6 ± 15.2 kgs, at the time of diagnosis. Histological typing, according to WHO classification, was as follows: squamous cell carcinoma, n = 52 (37.4%); adenocarcinoma, n = 69 (49.7%); large cell carcinoma, n = 14 (10.0%); and neuroendocrine carcinoma, n = 4 (2.9%). Patients' WHO PS scores at the time of diagnosis were: 0, n = 35 (25.2%); 1, n = 47 (33.8%); 2, n = 29 (20.9%); 3, n = 19 (13.7%); and 4, n = 9 (6.4%). Patients TNM stages at diagnosis, in accordance with the international classification, were: II, n = 1 (0.7%); IIIB, n = 25 (18.0%); IV, n = 108 (77.7%); and not evaluated (because of a very bad WHO PS score), n = 5 (3.6%) (Table 1).

Table 1.

Main clinical characteristics of all patients included

^ap-value of the comparison between patients included in the study (n = 85) and patients excluded (n = 54) by a Kruskal-Wallis test.

^bp-value of the comparison between patients included in the study (n = 85) and patients excluded (n = 54) by a Fisher exact test.

Abbreviations: LDH, lactate dehydrogenase; PS, performance status; SD, standard deviation; TNM, tumor–node–metastasis; WHO, World Health Organization.

Sample

At the end of the follow-up period, eight patients were still alive and 54 had clinician's survival estimates (n = 54) or some clinical data (n = 59) missing. Only 85 patients were eventually included in the analysis. A comparison of included patients with excluded patients regarding sex, age, histological typing, WHO PS score, and TNM staging did not show any significant differences (Table 1).

Among these 85 patients (Table 1), 69 (81%) were men and 16 (19%) were women. Their mean age was 62.2 ± 11.6 years, their mean weight at the time of admission was 68.3 ± 13.9 kg, and their mean weight loss was 7.7 ± 6.8 kg. Histological typing according to WHO criteria was as follows: squamous cell carcinoma, n = 33; adenocarcinoma, n = 42; large cell carcinoma, n = 8; and neuroendocrine carcinoma, n = 2. Patient WHO PS scores were as follows: 0, n = 17; 1, n = 29; 2, n = 17; 3, n = 13; and 4, n = 9. TNM staging was as follows: II, n = 1; IIIB, n = 11; and IV, n = 69; TNM staging was not carried out in four patients because of their extremely poor WHO PS scores on admission. None of those four patients were included in a research trial, and all were treated according to current recommendations [3].

Means levels of biological parameters [7] were: blood hemoglobin, 11.7 ± 1.8 g/dl; platelet count, 397.8 ± 156.3 × 10⁹/l; serum protein level, 68.7 ± 7.1 mg/l; and serum calcium level, 92.3 ± 10.0 mg/l.

Quality of Life

The quality of life of each patient as assessed by the QLQ-C30 questionnaire was poor on most dimensions, particularly concerning the emotional, cognitive, and social scales (median scale values, 83.3, 66.6, and 66.6, respectively). The mean global health score was ∼50 (Table 2).

Table 2.

Patients' quality of life status

Patient Survival

The median and mean survival times were 11.7 weeks (range, 0.4–143.7 weeks) and 21.7 weeks (± 26.8 weeks), respectively. As expected from the literature, a low WHO PS score (p < .0001), low hemoglobin level (p = .012), low protein levels (p = .026), and high platelet count (p = .0006) were found to be prognostic factors for shorter survival.

Actual and Physicians' Estimates of Patient Survival

The actual survival times and estimates of survival times of the patients under their care by each category of physician (residents, registrars, and consultants) are shown in Table 3 and Figure 1. The level of concordance was moderate (ICC = 0.5 for residents, ICC = 0.4 for registrars, and ICC = 0.5 for consultants). Interestingly, all physicians overestimated survival, but residents (median, 21.5 weeks) and registrars (median, 21.4 weeks) were more accurate than consultants (median, 25.7 weeks).

Table 3.

Patient survival and survival predicted by physicians; intraclass coefficient (ICC)

Abbreviation: SD, standard deviation.

Figure 1.

Comparison of predicted versus actual patient survival: Kaplan–Meier survival curves. Survival curves were constructed using Kaplan–Meier estimation (n = 85). Cumulative survival is expressed as a percentage, and survival is expressed in weeks.

With respect to the usual requirement of probable life expectancy of at least 3 months (thereby permitting patient recruitment into a research protocol), 61 patients, 63 patients, and 69 patients, according to residents', registrars', and consultants' survival estimates, respectively, could have been eligible for inclusion in a trial. Moreover, about 60% of the patients whose physicians estimated a survival time >3 months actually survived >3 months. In fact, 51 patients survived >3 months, that is, 83.6% (51/61), 80.9% (51/63), and 73.9% (51/69) of their predicted values, respectively.

Quality-of-life scores had differential relations with the difference between actual and predicted survival among the three physician categories (Table 4). For consultants and registrars, no quality-of-life scale was related to departure of prediction from actual patient survival. For residents, interestingly, four scales of quality of life had no significant relation with departure of prediction from actual patient survival (emotional functioning scale, p = .084; cognitive functioning scale, p = 0.71; social functioning scale, p = .45; and physical functioning scale, p = .053), whereas two did. A worse global health status score was associated with a lower discrepancy between the estimated and actual patient survival time (p < .003), and a worse role functioning was associated with a larger difference between estimated and actual patient survival (p = .01).

Table 4.

Relation between quality of life scores and difference between estimated and actual patient survival

Discussion

In patients with advanced lung cancer, regardless of the duration of their clinical experience, physicians overestimated the median survival time at diagnosis by almost twice the actual survival later observed (observed to expected ratio, 1.8 to 2.2). Younger physicians, registrars, and, to an even greater extent, residents were more accurate. Interestingly, a relationship between departure of prediction from actual survival and patient quality of life (global status and role functioning QLQ-C30 scales) was observed in younger physicians, suggesting a better knowledge or higher perception of their patients' actual prognosis, compared with senior doctors. Classical prognostic factors, such as a poor WHO PS score, low hemoglobin level, low protein level, high platelet count, and high LDH level, were confirmed to be related to shorter survival, suggesting no particular selection bias in this sample.

The large majority of already published data on this issue relies on the first approach, and deals with chronic diseases and cancers. In general, and for quite obvious reasons, the accuracy of prediction for patient (cancer or other illnesses) survival is much better when the estimate is made near death, whereas prognostication over a longer term seems more uncertain, as confirmed by Brandt et al. [22], with a cohort of 515 terminally ill patients. In this respect, Christakis et al. [23] observed that physicians overestimated survival in terminal cancer or HIV patients (in 63% of cases, doctors overestimated the survival with a mean predicted-to-observed survival ratio of 5.3). However, Chow et al. [24] showed, in their literature review based on 12 articles, that clinical predictions tended to be inaccurate in the optimistic direction.

With respect to determinants of lung cancer patient survival, most papers, if not all, have evaluated the relationships among several clinical and biological factors, such as TNM staging, WHO PS score, symptoms (anorexia, dyspnea, loss of weight), platelet count, LDH level, hemoglobin level, and protein level, blood group, and oncogene expression by tumor cells, etc. This was shown, for example, by Moldway et al. [25] in adenocarcinoma patients, for whom positive Bcl-2 staining and A+B+H antigen tumor staining was associated with longer survival. A review of the literature by Chow et al. [24], based on 19 articles, on prognostic factors confirmed that WHO PS score is a prognostic factor for survival. With respect to quality of life, Herndon et al. [15], using univariate analysis, confirmed the usual prognostic factors related to poor survival, but also observed that lower quality of life on QLQ-C30 subscales related to pain, appetite loss, fatigue, lung carcinoma symptoms, overall quality of life, and physical functioning scale was related to shorter survival.

Only a few papers have dealt with the actual evaluation by physicians of each patient's likely survival. In this respect, Viganó et al. [26, 27], in a study of two cohorts of terminal cancer patients, suggested that physicians overestimated patient survival. In the first cohort, dyspnea, nausea, vomiting, liver metastases, and lung cancer were prognostic factors for survival. In the second cohort, the patients had worse outcomes than in the first cohort, and dyspnea, weakness, and breast, gastrointestinal, or urinary cancer were also prognostic for survival. The authors concluded that clinicians should focus on physical quality-of-life indicators to gather prognostic clues in these patients. Indeed, Coates et al. [28], in a trial of breast cancer patients, found that there was a strong prognostic significance of quality-of-life scores after disease relapse in patients with advanced breast cancer. Gripp et al. [29] carried out a study on 216 patients with terminal cancer and observed that PS, primary cancer, fatigue, dyspnea, use of strong analgesics, brain metastasis, leukocytosis, LDH level, blood level, and anxiety were related to survival, and that physicians' survival estimates were unreliable, especially in patients near death, a finding that diverges from findings observed in most published papers, as cited above. Moreover, they found that a strong doctor–patient relationship did not appear to improve the accuracy of the clinical prediction of survival. Llobera et al. [30] saw, in 200 terminal cancer patients, that a shorter survival time was related to anorexia, weight loss, and dyspnea.

Obviously, our study has some limitations. First, the study population was drawn from patients admitted to only one chest department specialized in the management of lung cancer, and this reflects referral to a specialized setting. The number of patients is limited, but the study was conducted with well-defined types of patients with lung cancer. Moreover, the fact that classical prognostic factors were observed in this work suggests that our patient sample was representative, and that the exclusion of some patients for technical reasons was sound. Second, the restriction to advanced stage lung cancer with a poor prognosis was the consequence of the question asked, and of the knowledge at the start of the study that evaluation of patients with shorter survival times is more accurate. Finally, quality of life during admission in these patients might have been influenced by the acute, severe, but mostly reversible, conditions that had led to the hospital admission, rather than the baseline condition solely related to cancer. The former can be a confounding factor and needs to be explored further.

The observation of a difference in accuracy of survival estimates among the three categories of physicians needs to be explained. Though all physicians overestimated patient survival, residents were more accurate. Only a few clinical trials have evaluated the prognostic value of quality-of-life scores. For example, Viganó et al. [27] found that clinicians overestimated patient survival. Gripp et al. [29] observed results similar to our findings. In their study, doctors in training and experienced doctors estimated patient survival, and a final estimation was decided by a consensus vote. In our study, there was a trend toward better accuracy for doctors in training than for experienced doctors. The residents seemed to take into account the patients' quality of life in their estimation of patient survival; this is likely to be a result, at least in part, of their closer relationship with patients, involving empathy. Residents are directly involved in the care of patients with lung cancer and have a better perception of their health than senior physicians. Another explanation, perhaps, is the fact that physicians have only a limited knowledge about the social or cognitive impact on health status.

Without a description and evaluation of more efficient means to prospectively evaluate each patient's survival, such prediction inaccuracy will remain a problem, and clinical trials will continue to include patients who are not correctly satisfying the expected conditions. Therefore, the full benefit of experimental treatment will not be assessed correctly.

Conclusions

In conclusion, this study shows that physicians overestimate survival in patients with advanced stage or metastatic NSCLC. Residents are more accurate in their estimation of patient survival than senior doctors, and this may be related to their perception of patient quality of life. In order to improve the evaluation of patient survival in NSCLC patients, further research should investigate whether the accuracy of patient survival estimates by clinicians would be improved by taking into account patient quality of life.

Author Contributions

Conception and design: Christelle Clément-Duchêne, Francis Guillemin, Yves Martinet

Collection and/or assembly of data: Christelle Clément-Duchêne, Charlotte Carnin

Data analysis and interpretation: Christelle Clément-Duchêne, Francis Guillemin

Manuscript writing: Christelle Clément-Duchêne, Francis Guillemin, Yves Martinet

Final approval of manuscript: Christelle Clément-Duchêne, Francis Guillemin, Yves Martinet