Survival Prediction for Terminally Ill Cancer Patients: Revision of the Palliative Prognostic Score with Incorporation of Delirium

The palliative prognostic score, used to predict survival in terminally ill cancer patents, was revised to include the evaluation of delirium. The revised score performed better while maintaining the simplicity of the original score.

Learning Objectives

After completing this course, the reader will be able to:

1. Describe the effect on the palliative prognostic score classifications when delirium was included as a variable.

2. Compare changes in overall survival times when delirium was added to the palliative prognostic score.

This article is available for continuing medical education credit at CME.TheOncologist.com

Abstract

Purpose.

An existing and validated palliative prognostic (PaP) score predicts survival in terminally ill cancer patients based on dyspnea, anorexia, Karnofsky performance status score, clinical prediction of survival, total WBC, and lymphocyte percentage. The PaP score assigns patients to three different risk groups according to a 30-day survival probability—group A, >70%; group B, 30%–70%; group C, <30%. The impact of delirium is known but was not incorporated into the PaP score.

Materials and Methods.

Our aim was to incorporate information on delirium into the PaP score based on a retrospective series of 361 terminally ill cancer patients. We followed the approach of “validation by calibration,” proposed by van Houwelingen and later adapted by Miceli for achieving score revision with inclusion of a new variable. The discriminating performance of the scores was estimated using the K statistic.

Results.

The prognostic contribution of delirium was confirmed as statistically significant (p < .001) and the variable was accordingly incorporated into the PaP score (D-PaP score). Following this revision, 30-day survival estimates in groups A, B, and C were 83%, 50%, and 9% for the D-PaP score and 87%, 51%, and 16% for the PaP score, respectively. The overall performance of the D-PaP score was better than that of the PaP score.

Conclusion.

The revision of the PaP score was carried out by modifying the cutoff values used for prognostic grouping without, however, affecting the partial scores of the original tool. The performance of the D-PaP score was better than that of the PaP score and its key feature of simplicity was maintained.

Introduction

Survival prediction is an important aspect of palliative care. Accurate prediction of survival is necessary for clinical, organizational, and ethical reasons, especially in order to prevent the administration of inappropriate therapies, to avoid inflicting harm or discomfort to vulnerable patients [1], and to plan specific care strategies. Although cancer can follow a variety of paths in the early course of the disease, involving a gradual decline over a period of months or years, prognostication in far advanced cancer is easier because there is usually an accelerated decline in the final weeks of life. The challenge facing clinicians caring for advanced cancer patients is to identify the starting point of the irreversible decline. Despite the predictable pattern of decline, clinical prediction of survival (CPS) is often incorrect, and physicians' estimates tend to give an optimistic view [2–4].

During the last decade, several prognostic tools have been developed to provide simple and more accurate estimates of survival in terminally ill cancer patients [5–8]. We previously developed the palliative prognostic (PaP) score for survival prediction in terminally ill cancer patients [5, 9–11]. The PaP score, repeatedly validated with independent prospective cohorts of patients [11–15], consists of a “weighted” scoring system obtained from a multivariate analysis that takes into account the following prognostic factors: Karnofsky performance status score, CPS, anorexia, dyspnea, total WBC, and lymphocyte percentage.

Although altered cognition is recognized as a common problem at the end of life and is associated with a worse prognosis [16, 17], neurological assessment of confusion/delirium (henceforth, delirium) was not included in our original PaP score. Caraceni and coworkers [18] reported that delirium and the PaP score are independent predictors of survival in advanced cancer patients, suggesting that the score could be improved by incorporating delirium, as defined by the confusion assessment method (CAM) algorithm [19].

The objective of the present paper was therefore to revise the PaP score by introducing delirium as an additional variable.

Materials and Methods

PaP Score and Case Series

The original PaP score was developed to predict overall survival using data from 519 consecutive terminally ill cancer patients [5, 9, 10]. The score was obtained from a Weibull multivariate regression model including Karnofsky performance status score, CPS, anorexia, dyspnea, total WBC, and lymphocyte percentage. These variables were chosen after a backward selection procedure from a set of 34 biological and clinical factors. The sum of the partial scores gave the PaP score (Table 1). Total scores are in the range of 0–17.5 and assign patients to three different risk groups according to a 30-day survival probability: group A, >70%; group B, 30%–70%; and group C, <30%. According to the PaP score, the median survival times were 64 days in group A (30-day survival probability, 82.0%), 32 days in group B (30-day survival probability, 52.7%), and 11 days in group C (30-day survival probability, 9.6%).

Table 1.

PaP score and D-PaP score and classification of patients into the three risk groups

Abbreviations: D-PaP, palliative prognostic plus delirium; PaP, palliative prognostic.

The PaP score was successfully tested using data from an independent validation cohort of 451 patients consecutively entered into hospice programs on the basis of the same inclusion and exclusion criteria [11]. The score was later validated using several independent case series [12–15]. The prognostic role of delirium was studied by Caraceni and coworkers [18] using a subset of the first validation cohort comprised of 361 patients with complete prognostic information [11]. Delirium was assessed according to the CAM algorithm [19]. This algorithm applies criteria from the Diagnostic and Statistical Manual of Mental Disorders to diagnose the presence of delirium on the basis of acute onset and a fluctuating course of symptoms, inattention, and either disorganized thinking or altered level of consciousness. The CAM has been fully validated as an instrument for diagnosing delirium and has very high specificity and sensitivity and good interrater reliability. In our case series, the treating physician applied the CAM at the same time as the PaP score variables were collected. A training session was held before the start of the study to explain how to use this instrument, and a neurologist was available throughout the study period for consultation.

Patients with delirium had a significantly different overall survival duration than nondelirious patients (p < .0001), and the presence of delirium maintained an independent correlation with survival in a multivariate model including the PaP score. The same series of 361 patients was used in the present study to revise the original PaP score (D-PaP score). This study was approved by the Medical Scientific Committee of the Istituto Scientifico Romagnolo per lo Studio e la Cura dei Tumori.

Statistical Methods

The study's main endpoint was the overall survival time, calculated as the time interval between the date of enrolment in the study and the date of death from any cause. Survival curves stratified by the PaP score were estimated using the Kaplan–Meier product-limit method [20]. To check the original PaP score and allow for its revision by integrating the prognostic effect of delirium, we adopted the approach of “validation by calibration” [21]. The score revision was performed by including the original score and the new variable of delirium in a multivariate model [22]. The discriminating ability of the three-group prognostic classification obtained by the PaP score and the D-PaP score was assessed using the K statistic [23–24]. Further details are provided in the Appendix. Statistical analyses were carried out using SAS statistical software (release 9.1; SAS Institute, Cary, NC) and R software (http://www.stat-project.com). p-values < .05 were considered to be statistically significant.

Results

The main patient characteristics are reported in Table 2. At the last follow-up in September 1996 (time of analysis of the original PaP score), 327 patients had died, with a median survival duration of 34 days (first to third quartiles, 18–73 days). Estimation of model parameters including the PaP score (model 1) and the D-PaP score (model 2) are shown in Table 3. Statistical analysis of individual parameter estimates showed that the scale parameter diverged from the null hypothesis value, which means that the baseline survival function failed to perfectly fit the available data. On the other hand, the prognostic factor effect can be considered coherent with that of the original series (see Discussion for further details).

Table 2.

Main clinical–biological characteristics of 361 patients

Table 3.

Parameters and standard errors estimated by the calibration (model 1) and revision (model 2) models

^a Likelihood ratio test for the null hypothesis α = 0, β = 1, and γ = 1 was rejected (p < .0001),

Model 1, PaP score only; model 2, PaP score plus delirium.

Abbreviations: PaP, palliative prognostic; SE, standard error.

With regard to the model used for score revision (model 2), significant results were obtained for delirium (p < .0001), suggesting that this variable provides important prognostic information (hazard ratio, 1.60; 95% confidence interval [CI], 1.22–1.99) in addition to that furnished by the original PaP score variables. The discriminating ability of the D-PaP score was 0.860 (95% CI, 0.817–0.880) compared with 0.853 (95% CI, 0.823–0.877) for the PaP score. D-PaP partial scores are shown in Table 1, together with the new cutoffs used for defining the three prognostic groups. For the D-PaP score, a partial score of two was added to the PaP score for patients with delirium.

The PaP and D-PaP score classifications were in agreement in 292 patients (80.9%). Of the 69 remaining patients, two switched from PaP score group A to D-PaP score group B, 28 switched from PaP score group B to D-PaP score group A, 15 switched from PaP score group B to D-PaP score group C, and 24 switched from PaP score group C to D-PaP score group B. Overall, 4.7% of the patients switched to a less favorable prognostic group, whereas 14.4% switched to a more favorable group. The numbers of patients with delirium were 14 (10.1%) and 37 (25.9%) in groups A and B, respectively, and sharply greater in group C at 48 (60.8%). Once again, as a function of the hazard ratio of delirium and of 30-day survival estimates in the three prognostic groups defined by the D-PaP score, it is estimated that survival differed in patients with or without delirium by 0.9% for group A, 4.4% for group B, and 6.2% for group C. Delirium thus seems to better classify patients in group C.

Figure 1 shows the survival curves for the three prognostic groups defined by the PaP score (Fig. 1A) and the D-PaP score (Fig. 1B). The 30-day survival probability estimates in groups A, B, and C were 87%, 51%, and 16% for the PaP score and 83%, 50%, and 9% for the D-PaP score, respectively (Table 4).

Figure 1.

Overall survival curves according to risk groups derived from (A) the original palliative prognostic score and (B) the revised plus delirium palliative prognostic score.

Table 4.

Overall survival in PaP and D-PaP score groups

Abbreviations: CI, confidence interval; OS, overall survival; PaP, palliative prognostic.

Discussion

We revised the previously developed PaP score currently used for terminally ill cancer patients in palliative care units by integrating it with information on the neurological assessment of delirium. It was seen that the D-Pap score had a better, albeit slight, discriminating ability than the PaP score and, more importantly, resulted in group reclassification in as many as 20% of the patients.

In a palliative oncology setting, prognostication and what it tells us can be used to keep a patient and his/her family informed, help clinicians in decision-making processes, identify the most appropriate health care setting, and obtain more accurate information for research purposes. Prognostic factors for the last 3 months of life have been extensively studied over the past few years [17, 25–27], and it has been observed that, although such factors in early phases are related to pathological findings, correct diagnosis, and appropriate therapy, they take on greater importance in end-of-life care [28].

A landmark paper on prognosis in palliative care was presented by Parkes in 1972 when the CPS was reported as a process by which physicians, on the basis of clinical experience, could estimate survival with a certain degree of accuracy [29]. Since then, numerous authors have studied the scope and limits of the CPS, identifying a number of other prognostic factors capable of providing superior prognostic capacity than the CPS or even replacing it. Variables with well-defined and independent prognostic capacity maintained on multivariate analysis have been reported: performance status, categorized and described using different tools such as the Karnofsky performance status score [17] and palliative performance scale [30]; several symptoms, in particular dyspnea, delirium, and signs/symptoms relating to nutritional status and cachexia–anorexia cancer syndrome; and biological factors (leukocytosis, lymphocytopenia, C-reactive protein) related directly or indirectly to nutritional or immune status [17].

Recently, many authors have attempted to build prognostic scores, integrating factors with independent prognostic ability and measuring the respective weight among the selected parameters [5–8]. A working group of the European Association for Palliative Care reported that the score that has undergone the most complete validation process in independent patient populations is the PaP score [17]. However, a number of priority issues (core topics) require further evaluation in relation to prognostication [31]: the validity of prognostic tools, feasibility of using prognostic criteria as entry criteria for research, method for evaluating the impact of a prognostic score in clinical practice, most suitable way of presenting survival data to patients, and most user-friendly validated tool. Although every prognostic score should ideally undergo complete and external validation before being introduced into clinical practice, this rarely happens. Both discrimination and calibration should be assessed for complete evaluation of predictive accuracy. Because a suitable validation process is difficult to implement, very few risk prediction scores are properly validated [32]. Unfortunately, all prognostic scores built to date suffer from some methodological shortcomings, ranging from difficulty identifying the real inception cohort to the selection of specific candidate parameters, perhaps excluding other potentially useful ones. An impact study should be performed to determine whether or not the use of the model obtains better results than those of the previous standard approach. Such studies can provide information on whether or not the use of a given score could influence the appropriateness of clinical behavior, thus improving the acceptability of the prognostic model by clinicians [33–35]. In particular, evaluation of the impact of a prognostic score appears to be important because a well-validated score could be used by physicians in a clinical setting to gauge when it is time to stop administering ineffective chemotherapy and move to palliative care.

When the PaP score was initially proposed, delirium had not yet been shown to have any definitive prognostic capacity and was therefore not inserted into the first pool of parameters evaluated in the training set. As the study progressed, data emerged indicating delirium as a candidate prognostic symptom, and we thus inserted it into the training set validation study, confirming its independent role in predicting survival [18]. We are aware that this study has a number of limitations, the most important being the very old case series used and the fact that the variable “delirium” was used in a palliative care context by a specialist trained in the use of the CAM algorithm. The CAM algorithm shows high accuracy [19], but it may not be a familiar tool for oncologists. For these reasons, a simpler version for detecting delirium is currently being developed.

Conclusion

In the present work, we built a new version of the PaP score called the D-PaP score, which integrates delirium as an additional predictor. Model validation and revision were carried out by modifying the cutoff values used for prognostic grouping without affecting the partial scores of the PaP score, as explained in Statistical Methods. This helps clinicians who are already familiar with the PaP score. In our case series, the D-PaP score showed a better, albeit slight, performance than the PaP score while maintaining its key feature of simplicity. We now intend to compare the clinical accuracy of the D-PaP score with that of other prognostic scores in a prospective cohort study.

Appendix

The “validation by calibration” approach, proposed by van Houwelingen [21], allows for assessing the validity of an established score by embedding the score into a “calibration model,” which is a Weibull survival regression model (model 1) estimating three parameters: intercept α, slope β, and scale γ. The score can be considered valid if the joint null hypothesis on α = 0, β = 1, and γ = 1 is not rejected. Testing on α and γ allows us to verify the correctness of the baseline survival function whereas testing on β/γ allows the stability of the prognostic variable effect to be validated. In the event of score revision by the inclusion of a new variable, as in the present study, the approach described by Miceli and coworkers [22] can be used, in which a new multivariate model is fitted including the original score and the new variable (model 2). The model parameter estimates are then used to calculate the partial score for the new variable and the revised cutoffs for defining the prognostic groups in such a way as to leave unchanged the partial scores for the variables originally contributing to the score.

The discriminating ability of the three-group prognostic classification obtained by the PaP score and the D-PaP score was assessed using the K statistic developed by Begg and coworkers [23], rescaled with respect to the maximum value attainable for each of the two prognostic classifications. This rescaled K can vary from 0.5 (lack of discrimination) to 1, attainable in the event of perfect survival discrimination. The corresponding 95% CI was obtained by bootstrap, applying the bias-corrected and accelerated method [24].

Author Contributions

Conception/Design: Emanuela Scarpi, Marco Maltoni, Oriana Nanni

Provision of study material or patients: Marco Maltoni, Augusto Caraceni

Collection and/or assembly of data: Emanuela Scarpi

Data analysis and interpretation: Emanuela Scarpi, Marco Maltoni, Oriana Nanni, Rosalba Miceli, Luigi Mariani

Manuscript writing: Emanuela Scarpi, Marco Maltoni, Oriana Nanni, Rosalba Miceli, Luigi Mariani, Augusto Caraceni

Final approval of manuscript: Emanuela Scarpi, Marco Maltoni, Oriana Nanni, Rosalba Miceli, Luigi Mariani, Augusto Caraceni, Dino Amadori