Abstract
Objective
To examine the prognostic value of different comorbidity coding schemes for predicting survival of newly diagnosed elderly cancer patients.
Materials and Methods
We analyzed data from 8,867 patients aged 65 years of age or older, newly diagnosed with cancer. Comorbidities present at the time of diagnosis were collected using the Adult Comorbidity Evaluation-27 index (ACE-27). We examined multiple scoring schemes based on the individual comorbidity ailments, and their severity rating. Harrell’s c index and Akaike Information Criterion (AIC) were used to evaluate the performance of the different comorbidity models.
Results
Comorbidity led to an increase in c index from 0.771 for the base model to 0.782 for a model that included indicator variables for every ailment. The prognostic value was however much higher for prostate and breast cancer patients. A simple model which considered linear scores from 0 to 3 per ailment, controlling for cancer type, was optimal according to AIC.
Conclusion
The presence of comorbidity impacts on the survival of elderly cancer patients, especially for less lethal cancers, such as prostate and breast cancers. Different ailments have different impacts on survival, necessitating the use of different weights per ailment in a simple summary score of the ACE-27.
Keywords: Comorbidity, comorbid ailment, elderly, cancer patients, prognostic, survival
INTRODUCTION
Elderly cancer patients often have other diseases, illnesses, and conditions at the time of their diagnosis. These diseases, illnesses, and conditions are referred to as comorbidities. Comorbidity is an independent predictor of survival and is important to the care of cancer patients. Patients with comorbidities have worse outcomes than patients with no such conditions.[1] Read et al.[2] demonstrated that the prognostic value of overall comorbidity is relative to the mortality burden of the index cancer. That is, comorbidity is prognostically most important in situations where the prognostic impact of the tumor is small.
The incidence of a new cancer diagnosis in the elderly population is reported as 25% between the ages of 65 and 74, 22% between the ages of 75 and 84, and 8% for individuals aged 85 and older.[3] Elderly patients tend to suffer from cancers that are not rapidly fatal, such as prostate, colon, uterus, and breast.[4–6] Elderly cancer patients also have more comorbid conditions and poorer physical and mental health than elderly peers with no cancer.[7;8] The presence and severity of some comorbidities increase with age, while other comorbidities decrease.[9] For instance, the prevalence and severity of dementia and congestive heart failure increased with age while HIV/AIDS and obesity decreased.
There are several comorbidity instruments and different methods of combining the prognostic impact of multiple diseases, illnesses, and conditions to create an overall score.[10] Some instruments require a detailed chart-based review, while others can be implemented with ICD-9 data contained within a hospital patient’s discharge summary. Independent of the measurement method, comorbidities have been significantly associated with outcomes in mortality and quality of life of elderly cancer patients.[11]
Acknowledging the importance of comorbidities, different groups have focused their efforts on comparing the different methods of assessing comorbidities with the goal of identifying a better assessment tool as relates to elderly cancer patients.[12] Piccirillo et al[1] modified the original Kaplan-Feinstein Index (KFI)[13] by adding several health conditions (e.g., diabetes, HIV/AIDS, dementia) not present on the original KFI, and created a comorbidity index for newly diagnosed cancer patients [1]. The new index is called the Adult Comorbidity Evaluation-27 (ACE-27) index. The ACE-27 is a validated instrument that captures cogent comorbidities for cancer patients and grades severity of the ailments at the time of cancer diagnosis.[14; 15]
Using modern methods of analysis, we revisited the issue of quantifying comorbidity accurately in elderly cancer patients to appropriately reflect prognosis. The goal of our study was to examine the impact of different comorbidity scoring schemes for newly diagnosed elderly cancer patients.
MATERIALS AND METHODS
Study population
The study population consisted of 9,253 patients aged 65 years or older, diagnosed with cancer between 2000 and 2007 at Barnes Jewish Hospital in Saint Louis (BJH), for whom detailed comorbidity data was collected using ACE-27[14] index. Comorbid health information was collected for each patient by cancer registrars trained to use the ACE-27 method as part of an on-going cancer education program sponsored by the National Cancer Institute.[15] Data elements, such as date of diagnosis, demographic information, cancer diagnosis, Surveillance, Epidemiology, and End Results (SEER) tumor staging, and survival, were collected as part of the cancer registrars’ routine job. The presence of one or more different comorbid ailments, present at time of cancer diagnosis, was coded for all patients using ACE-27. Of the 9,253 patients, information on race, gender, tumor staging, or survival was missing for 386 (4%) patients and these patients were thus excluded from analysis. The “zero-time” for the study was defined as the date of cancer diagnosis and the duration to the last follow-up was cited relative to this zero-time. The date of last follow-up was July 2010.
Classification of data
Patients were classified into one of three race categories: white, black, or other. Cancer diagnosis was recorded by cancer registrars based on the pathology report documenting the malignancy. The dataset included malignancies of the following body systems: lung, breast, prostate, colorectal, digestive, genitourinary, head and neck, gynecologic, skin (excluding basal cell), musculoskeletal, or other. Under the “other” category we included malignancies of the nervous, lymphatic, ophthalmologic, blood, other male genitalia systems together with mesotheliomas and thyroid tumors. The SEER stage uses clinical and pathological information in the chart, and categorizes tumors by the method of spread. SEER Summary Stage is a required data field for cancer registrars. Prior to data analysis, SEER summary staging was re-categorized into one of three main groups: in situ/local, regional, or distant.
Comorbid ailments and coding
Concomitant medical conditions, not related to the index cancer, were captured by cancer registrars through medical chart review using the ACE-27. The cogent comorbid conditions were grouped under each of the following systems: cardiovascular, respiratory, gastrointestinal, renal, endocrine, neurological, psychiatric, rheumatologic, immunological, malignancy prior to index malignancy or concomitant malignancy with index malignancy, substance abuse, and obesity. Cogent individual comorbid ailments, except for obesity, were classified according to the severity of organ decompensation into one of three severity categories: none, mild, moderate, or severe. Morbid obesity defined as BMI greater than or equal to 38, was scored as moderate. The ACE-27 form and related material is available on line at: http://oto2.wustl.edu/clinepi/comorbid.html
Comorbidity scoring schemes
Multiple comorbidity scoring schemes were examined based on the individual comorbidity ailments and their severity ratings. The characteristics of the eight different comorbidity scoring schemes and their relationship to each other are described in greater detail below and are presented in Table 3.
Table 3.
Comparison of Harrell’s c concordance statistic, AIC and Nagelkerke’s R square resulting from Cox-proportional hazard model analysis of models including different comorbidity coding schemes versus the model with no comorbidities.
| Comorbidity System |
Definition | Allows for non-linearity? |
Different Weights? |
Model Label |
Delta* df |
Delta c-index§ |
AIC€ by comorbidity |
Delta R square ¥ (%) |
|---|---|---|---|---|---|---|---|---|
| Kaplan- Feinstein |
Highest-ranked ailment, or 2 Grade 2 ailments equals Severe (Grade 3) |
No | No | A1 | 1 | 0.004 | 105 | 0.8 |
| Yes | No | A2 | 3 | 0.004 | 103 | 0.8 | ||
| Simple Sum | Sum of scores | No | No | B1 | 1 | 0.006 | 189 | 1.3 |
| Individual Comorbidity Scores |
Different weight for each comorbid ailment |
No | Yes | C1 | 26 | 0.009 | 283 | 2.3 |
| NA | Yes | C2 | 26 | 0.007 | 178 | 1.6 | ||
| Yes | Yes | C3 | 71 | 0.010 | 267 | 2.8 | ||
| Non-Linear Weighting |
Prognostic weights per grade of comorbidity |
Yes | No | D1 | 3 | 0.006 | 186 | 1.3 |
| Yes | Yes | D2 | 28 | 0.009 | 280 | 2.3 |
Delta refers to change in performance of comorbidity plus the baseline model to the baseline model alone.
Delta c-index is the increase in the c-index for each model as compared to the baseline model with c-index=0.772. The larger the value, the better the model discrimination is.
AIC by comorbidity is the magnitude of reduction (absolute value) of the AIC for each model as compared to the baseline model with AIC=79997.842. This is attributable only to comorbidity inclusion in the model. The larger the value of AIC by comorbidity, the larger the reduction in AIC is, and the better the model performance.
Delta R Square is the magnitude of change of Nagelkerke’s R square from each model relative to the baseline model with R square=38.8%. The larger the value of delta R square, the larger the improvement in the model’s performance in explaining the variability of outcome.
A1 and A2: Kaplan-Feinstein Score
The overall comorbidity severity score is defined according to the scoring algorithm for the original Kaplan-Feinstein Index.[(13)] In that scoring system, the possible overall scores are none (0), mild (1), moderate (2), or severe (3). The overall comorbidity score for an individual patient represents the severity grade of organ decompensation for the highest-ranked single comorbid ailment or, in the case in which there are two comorbid ailments in 2 different organ systems each rated as moderate organ decompensation, the overall score is severe. This comorbidity scoring scheme is currently the recommended scoring algorithm for the ACE-27. The overall ACE-27 score also ranges from none (0) to severe (3). This score is considered as a linear term (labeled as A1 in Table 3) with 1 degree of freedom (df), as well as a categorized version (labeled as A2) with 3df.
B1: Simple Sum
The simplest extension of the Kaplan-Feinstein Index score is to allow the maximum score to reflect the sum of each individual comorbid condition rather than restrain the maximum score to 3. Similar to the Kaplan-Feinstein Index score, the simple sum does not allow for nonlinearity, and assumes each comorbidity condition to have identical weights. The simple score based on this scheme (labeled as B1 in Table 3) ranges theoretically from 0 in the case of no comorbidity present, to 77 (= 25 × 3 + 2) in the cases of the highest severity score recorded on every single ailment.
C1, C2, C3: Individual Comorbidity Scores
A further extension of the coding scheme is to allow for different weights for each comorbid ailment. Scheme C1 considers the scores ranging from 0 to 3 per ailment. Scheme C2 dichotomizes each individual ailment, with comorbidity severity none (Grade 0) or mild (Grade 1) grouped together reflecting minor comorbidity and comorbidity severity groups moderate (2) and severe (3) grouped to reflect major comorbidity. Scheme C3 considers categorized scores per condition, with three indicator variables generated for every ailment, except obesity. For obesity, only one indicator variable was possible, namely, moderate vs. none.
D1, D2: Non-Linear Weighting
For non-linear weighting, three different scoring variables were used: G1, G2, and G3 to allow for non-linearity in the scores from 0 to 3 per condition. G1 is the sum of all comorbid ailments scored as mild in their severity, G2 is the sum of all moderate severity ailments, and G3 is the sum of all severe coded ailments. The first scoring scheme (D1) uses just G1, G2, and G3 with the purpose of non-linear weighting and equal weights for each comorbidity (3df). To create the second non-linear weighting scheme (D2), the individual comorbid scores were added to scheme D1 to create a new scheme which allowed for different weights per condition (29df).
Analytical Work
The base model included the age-group (5 categories), race (3 categories), gender (2 categories), cancer site (11 categories), and SEER stage (4 categories). All comorbidity schemes were considered in addition to this base model. Models were compared across all cancer sites and within the 4 most common cancer sites (lung, prostate, colorectal, and breast). The Cox proportional hazard model was used for univariate and multivariable analyses. The likelihood ratio chi-square statistic, Akaike Information Criterion (AIC), Nagelkerke’s R- square, and Harrell’s c index were used to evaluate the performance of the different comorbidity models. The AIC is a version of chi-square that is penalized by the number of independent variables in the model, thus allowing comparison of models of varying complexity.[16] AIC By Comorbidity is calculated as model chi-square minus 2*df, where model chi-square is the difference in -2 log likelihood of a model with and without the specific comorbidity scoring, and df is the degrees of freedom used by adding the specific comorbidity scoring. AIC By Comorbidity is the magnitude of reduction (absolute value) of the AIC for each model as compared to the baseline model without comorbidity. The larger the value of AIC By Comorbidity, the better the model performance is. AIC hence penalizes more complex models, similar to measures such as the Bayesian Information Criterion (BIC).[17] The c index is defined as the proportion of all usable patient pairs in which the predictions and outcomes are concordant.[18] For binary outcomes, c is identical to the area under the receiver operating characteristic curve. All presented c statistics were corrected for optimism through a bootstrapping procedure using 200 bootstrap samples, drawn with replacement from the original samples.[17] The mean decrease in c-index resulting from bootstrapping was negligible (p=0.001). We used SAS® version 9.2., STATA-SE version 10, R version 2.11.1. This study was approved by Washington University Human Research Protection Office.
RESULTS
Patient characteristics
The characteristics of 8,867 elderly patients (65 years of age or older at the time of cancer diagnosis) diagnosed with cancer at BJH are described in Table 1. Of the total 8,867 patients, 82% were between 65 and 79 years old at the time of cancer diagnosis, and 53% were males. The sample consisted of 83% Caucasians, 16% African-Americans, and 1% other races. The frequency distribution of the Kaplan-Feinstein Index overall comorbidity score was none 1,660 (19%), mild 3,705 (42%), moderate 1,960 (22%) and severe 1,542 (17%).
Table 1.
Descriptive Characteristics of 8,867 Patients Diagnosed With Cancer (2000–2007) at BJH with Comorbidities Assessed Using A Chart-Based Method (ACE-27)
| N | % | Hazard Ratio (HR) |
95% Hazard Ratio Confidence Limits |
|
|---|---|---|---|---|
| Age groups | ||||
| 65–69 | 2930 | 33 | Ref. | -- |
| 70–74 | 2390 | 27 | 1.31 | 1.21 to 1.41 |
| 75–79 | 1920 | 22 | 1.58 | 1.46 to 1.71 |
| 80–84 | 1100 | 12 | 1.85 | 1.69 to 2.03 |
| ≥85 | 527 | 6 | 2.66 | 2.39 to 2.97 |
| Gender | ||||
| Female | 4202 | 47 | Ref. | -- |
| Male | 4665 | 53 | 0.95 | 0.89 to 0.99 |
| Race | ||||
| Caucasian | 7394 | 83 | Ref. | -- |
| African-American | 1366 | 16 | 1.27 | 1.18 to 1.36 |
| Other | 107 | 1 | 0.86 | 0.64 to 1.15 |
| Comorbidity Score* | ||||
| None | 1660 | 19 | Ref. | -- |
| Mild | 3705 | 42 | 1.43 | 1.31 to 1.56 |
| Moderate | 1960 | 22 | 1.85 | 1.68 to 2.03 |
| Severe | 1542 | 17 | 2.05 | 1.86 to 2.26 |
| SEER Tumor staging | ||||
| In Situ/Local | 4150 | 47 | Ref. | -- |
| Regional | 2502 | 28 | 2.50 | 2.32 to 2.68 |
| Distant | 2215 | 25 | 5.89 | 5.49 to 6.32 |
| Main cancer diagnosis** | ||||
| Lung | 1447 | 16 | 2.48 | 2.32 to 2.64 |
| Breast | 783 | 9 | 0.31 | 0.27 to 0.36 |
| Prostate | 1293 | 15 | 0.19 | 0.17 to 0.22 |
| Colorectal | 910 | 10 | 0.83 | 0.75 to 0.91 |
| Digestive | 1203 | 14 | 2.87 | 2.68 to 3.08 |
| Urinary | 732 | 8 | 0.81 | 0.73 to 0.90 |
| Head and neck | 436 | 5 | 0.93 | 0.81 to 1.06 |
| Gynecological | 771 | 8 | 0.91 | 0.82 to 1.01 |
| Skin | 184 | 2 | 0.64 | 0.51 to 0.80 |
| Musculoskeletal | 163 | 2 | 1.12 | 0.92 to 1.37 |
| Other | 945 | 11 | 1.55 | 1.43 to 1.68 |
Calculated based on original Kaplan-Feinstein scoring method.
The reported survival HR for each site is relative to all the others together (e.g. lung cancer vs. non-lung cancer)
The most frequent cancer site reported was the lung (1,447, 16%), followed by prostate (1,293, 15%), digestive system (1,203, 14%), colorectal (910, 10%), and breast (783, 9%). The SEER tumor staging was distributed as follows: in situ and local (4,150, 47%), regional (2,502, 28%), and distant (2,215, 25%).
Outcome
Patient overall survival was the outcome of interest. At the time of last follow-up, 3,943 (44.5%) of the patients were alive and 4,924 (55.5%) were dead. The median follow-up time was 30 months and inter-quartile range was 10 to 61 months. The median overall survival was 44.9 months.
Bivariate analyses
Univariate Cox proportional hazards models were run with each of the main characteristics included in the base model and the hazard ratios generated are reported in Table 1. A steady increase in hazard ratio (HR) was noticed with every 5-year increase in age. There was no significant difference in survival for females compared to males. African-Americans were 1.27 [1.18 to 1.36] times more likely to die than Caucasians. As the morphological stage of the tumor increased, so did the mortality risk - when compared to patients with in situ/local tumors, patients with regional tumors had a HR of 2.50 [2.32 to 2.68], and 5.89 [5.49 to 6.32] for distant spread of tumor.
The hazard ratio was markedly different for tumors from different sites. When patients with specific types of cancers were compared to the rest of the study population, lung, digestive, musculoskeletal, and other rare cancers grouped together had an increased risk for mortality. For example, lung cancer patients had a HR of 2.48 [2.32 to 2.64] relative to all other patients; patients with digestive system cancer had a HR of 2.87 [2.68 to 3.08] and patients with less prevalent cancers grouped under the “other” category had a HR of 1.55 [1.43 to 1.68].
As severity of overall comorbidity, calculated based on Kaplan-Feinstein Index algorithm, increased so did the mortality risk. When compared to patients with no comorbidities, patients with mild comorbidity had a HR of 1.43 [1.31 to 1.56], moderate 1.85 [1.68 to 2.03], and severe 2.05 [1.86 to 2.26]
Multivariable analysis
We performed a Cox regression analysis of the complete dataset with all the cancer sites included, followed by separate analyses for subsets of data for each of the following cancer sites: breast, prostate, colorectal, and lung. Table 2 summarizes the impact of each of the comorbid conditions included in ACE-27 in prediction of overall survival after controlling for age group, gender, race, tumor stage, and tumor site (for the all cases model). Based on the likelihood ratio test, with the reported degrees of freedom, the base model (age group, gender, race, diagnosis, and SEER stage) and all comorbidity models were highly statistically significant (i.e., p values all less than 0.001). The interactions of comorbidity by type of cancer were statistically significant.
Table 2.
Comparison of Adjusted Hazard Ratios (HR) for the 26 Comorbid Ailments Included in the Adult Comorbidity Evaluation-27 Index Generated from Model C1
| All cases | Breast Cancer | Prostate Cancer | Colorectal Cancer | Lung Cancer | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Comorbidity Ailment |
HR | 95% HR Confidence Limits |
HR | 95% HR Confidence Limits |
HR | 95% HR Confidence Limits |
HR | 95% HR Confidence Limits |
HR | 95% HR Confidence Limits |
|||||
| MI | 1.02 | 0.97 | 1.08 | 0.83 | 0.58 | 1.20 | 1.22 | 0.96 | 1.56 | 1.12 | 0.96 | 1.31 | 0.98 | 0.89 | 1.08 |
| Angina | 1.06 | 0.99 | 1.13 | 1.24 | 0.78 | 1.99 | 1.11 | 0.83 | 1.49 | 1.02 | 0.84 | 1.25 | 1.14 | 0.99 | 1.32 |
| CHF | 1.21 | 1.12 | 1.31 | 0.85 | 0.61 | 1.19 | 1.44 | 0.99 | 2.08 | 1.63 | 1.33 | 2.00 | 1.08 | 0.91 | 1.27 |
| Arrhythmias | 1.07 | 1.02 | 1.12 | 1.49 | 1.16 | 1.92 | 1.57 | 1.29 | 1.91 | 1.06 | 0.91 | 1.22 | 1.05 | 0.96 | 1.15 |
| HTN | 0.99 | 0.94 | 1.05 | 0.82 | 0.60 | 1.12 | 1.00 | 0.75 | 1.34 | 0.96 | 0.79 | 1.17 | 0.96 | 0.85 | 1.08 |
| Venous | 1.01 | 0.97 | 1.05 | 0.84 | 0.66 | 1.07 | 0.98 | 0.79 | 1.22 | 1.09 | 0.97 | 1.23 | 1.01 | 0.92 | 1.10 |
| Arterial | 1.09 | 1.01 | 1.18 | 1.27 | 0.58 | 2.77 | 1.62 | 1.02 | 2.59 | 1.00 | 0.76 | 1.31 | 1.14 | 1.00 | 1.30 |
| Respiratory | 1.18 | 1.12 | 1.24 | 1.72 | 1.33 | 2.21 | 1.59 | 1.22 | 2.08 | 1.13 | 0.94 | 1.36 | 1.13 | 1.05 | 1.21 |
| Hepatic | 1.41 | 1.25 | 1.60 | 1.06 | 0.13 | 8.48 | 9.62 | 2.28 | 40.60 | 1.38 | 0.50 | 3.79 | 1.28 | 0.92 | 1.78 |
| Stomach | 0.97 | 0.88 | 1.07 | 1.00 | 0.45 | 2.27 | 0.42 | 0.17 | 1.05 | 0.86 | 0.61 | 1.21 | 0.90 | 0.73 | 1.12 |
| Pancreas | 1.47 | 1.08 | 2.00 | . | . | . | 18.57 | 2.60 | 132.89 | . | . | . | 1.16 | 0.61 | 2.20 |
| Renal | 1.33 | 1.21 | 1.45 | 3.09 | 2.05 | 4.66 | 2.35 | 1.51 | 3.65 | 1.77 | 1.30 | 2.41 | 1.13 | 0.93 | 1.38 |
| Diabetes | 1.13 | 1.08 | 1.18 | 1.31 | 1.01 | 1.71 | 1.49 | 1.14 | 1.96 | 1.21 | 1.05 | 1.40 | 1.09 | 0.98 | 1.21 |
| Stroke | 1.09 | 1.01 | 1.18 | 1.00 | 0.73 | 1.36 | 0.88 | 0.60 | 1.29 | 1.15 | 0.90 | 1.47 | 1.05 | 0.90 | 1.22 |
| Dementia | 1.57 | 1.38 | 1.78 | 2.39 | 1.54 | 3.70 | 1.65 | 0.67 | 4.06 | 1.64 | 1.23 | 2.18 | 1.43 | 1.04 | 1.96 |
| Paralysis | 1.15 | 0.72 | 1.83 | . | . | . | . | . | . | 2.33 | 1.20 | 4.55 | 0.93 | 0.48 | 1.80 |
| Neuromuscular | 1.28 | 1.07 | 1.54 | 1.91 | 1.08 | 3.38 | 16.52 | 3.04 | 89.64 | 1.31 | 0.74 | 2.32 | 1.17 | 0.73 | 1.88 |
| Psychiatric | 1.13 | 1.01 | 1.26 | 1.24 | 0.70 | 2.21 | 0.73 | 0.31 | 1.73 | 1.41 | 1.04 | 1.91 | 1.17 | 0.92 | 1.48 |
| Rheumatologic | 1.15 | 1.01 | 1.30 | 1.57 | 0.74 | 3.34 | 1.88 | 0.85 | 4.16 | 1.09 | 0.60 | 1.96 | 1.24 | 0.99 | 1.55 |
| AIDS | 1.76 | 0.83 | 3.70 | . | . | . | . | . | . | . | . | . | . | . | . |
| Solid tumor | 1.07 | 1.03 | 1.11 | 1.03 | 0.86 | 1.23 | 1.72 | 1.49 | 1.98 | 1.08 | 0.97 | 1.20 | 0.98 | 0.90 | 1.06 |
| Leukemia | 1.47 | 1.21 | 1.80 | 2.49 | 0.84 | 7.40 | 0.00 | 0.00 | . | 3.21 | 1.74 | 5.92 | 1.09 | 0.60 | 1.97 |
| Lymphoma | 1.09 | 0.88 | 1.35 | 2.39 | 1.36 | 4.20 | 2.60 | 0.95 | 7.14 | 1.27 | 0.56 | 2.88 | 1.05 | 0.80 | 1.39 |
| Alcohol | 1.19 | 1.02 | 1.38 | 0.00 | 0.00 | -- | 2.88 | 1.35 | 6.12 | 0.64 | 0.27 | 1.51 | 0.93 | 0.72 | 1.21 |
| Drugs | 2.16 | 1.44 | 3.25 | . | . | . | . | . | . | 3.07 | 1.56 | 6.02 | 1.57 | 0.88 | 2.80 |
| Obesity | 1.06 | 0.95 | 1.19 | 0.89 | 0.49 | 1.60 | 0.08 | 0.00 | 1.91 | 1.11 | 0.84 | 1.48 | 1.11 | 0.82 | 1.51 |
Discriminatory power of the created models was determined using Harrell’s c-index generated from STATA’s ‘estat concordance’ procedure. Overall, all our models, including the models containing all cases and the four models containing cancer-specific patients, were able to discriminate between high- and low-risk patients with c-indices ranging from 0.71 to 0.81. In the baseline model, without comorbidity, the c-index was 0.772, AIC was 79997.842 and Nagelkerke’s R square was 38.8%. In all cases, the models containing comorbidity performed better than the baseline model, which did not contain comorbidity. The best discriminating power for “All Cases” was observed in Model C3 (c=0.782), which showed an improvement of 0.010 over the model with no comorbidities (c=0.772) (Table 3). Model C3 was also best in all cancer-specific models (Breast c=0.796; Prostate c=0.808; Colorectal c=0.769; Lung c= 0.712) with improvements of 0.050, 0.086, 0.058, and 0.015 respectively over the model with no comorbidities for each site with c-indexes 0.745, 0.723, 0.711 and 0.697 (Table 4). The discriminating power of Model C3 was followed by Models C1 and D2.
Table 4.
Comparison of Harrell’s c concordance statistic, AIC and Nagelkerke’s R square resulting from Cox-proportional hazard model analysis of models including different comorbidity coding schemes versus the model with no comorbidities per each of the four main cancer sites.
| Model | Delta df |
Delta c-index |
AIC by comorbidity |
Delta R square (%) |
|---|---|---|---|---|
| Breast Cancer (n=1651)* | NA | 0.745 | 2363 | 18.5 |
| A1 | 1 | 0.009 | 3 | 0.5 |
| A2 | 3 | 0.012 | 1 | 0.7 |
| B1 | 1 | 0.019 | 15 | 1.7 |
| C1 | 22 | 0.043 | 34 | 7.8 |
| C2 | 20 | 0.040 | 34 | 7.4 |
| C3 | 51 | 0.050 | 12 | 11.1 |
| D1 | 3 | 0.019 | 12 | 1.8 |
| D2 | 24 | 0.042 | 31 | 7.9 |
| Prostate Cancer (n=2514)* | NA | 0.723 | 2943 | 14.4 |
| A1 | 1 | 0.044 | 29 | 2.1 |
| A2 | 3 | 0.047 | 26 | 2.1 |
| B1 | 1 | 0.060 | 57 | 3.8 |
| C1 | 23 | 0.083 | 85 | 8.3 |
| C2 | 20 | 0.077 | 44 | 5.4 |
| C3 | 53 | 0.086 | 60 | 10.4 |
| D1 | 3 | 0.057 | 56 | 4.0 |
| D2 | 25 | 0.081 | 85 | 8.5 |
| Colorectal Cancer (n=1699)* | NA | 0.711 | 5701 | 25.8 |
| A1 | 1 | 0.017 | 22 | 1.9 |
| A2 | 3 | 0.017 | 18 | 1.9 |
| B1 | 1 | 0.026 | 42 | 3.5 |
| C1 | 24 | 0.048 | 51 | 7.7 |
| C2 | 23 | 0.038 | 44 | 7.0 |
| C3 | 58 | 0.058 | 47 | 12.2 |
| D1 | 3 | 0.026 | 42 | 3.8 |
| D2 | 26 | 0.041 | 52 | 8.0 |
| Lung Cancer (n=2538)* | NA | 0.697 | 15024 | 27.7 |
| A1 | 1 | 0.004 | 16 | 0.9 |
| A2 | 3 | 0.005 | 14 | 1.0 |
| B1 | 1 | 0.006 | 22 | 1.2 |
| C1 | 25 | 0.010 | 2 | 2.6 |
| C2 | 25 | 0.007 | −11 | 1.9 |
| C3 | 64 | 0.015 | −27 | 4.9 |
| D1 | 3 | 0.006 | 19 | 1.3 |
| D2 | 27 | 0.010 | −1 | 2.6 |
The row with the label of the cancer site reports the respective values of the baseline model for each cancer site.
Delta c-index is the magnitude of increase in c-index for each model as compared to the baseline model. The larger the value, the better the model discrimination is.
AIC by comorbidity is the magnitude of reduction (absolute value) of the AIC for each model as compared to the baseline model. This is attributable only to comorbidity inclusion in the model. The larger the value of AIC by comorbidity, the larger the reduction in AIC is and the better the model performance.
Delta R Square is the magnitude of change of Nagelkerke’s R square from each model to the baseline model with. The larger the value of delta R square, the larger the improvement in the models performance in explaining the variability of outcome.
Model C3 was also the model with the greatest number of terms with 71 degrees of freedom more than the model with no comorbidities (df =19). However, there were models with smaller numbers of terms that had essentially the same predictive power as Model C3. For instance, based on the AIC criterion, Model C1 is the best for “All Cases”, “Breast”, and “Prostate” with a reduction in AIC as compared to the baseline model of 283, 34, and 85, respectively. Model D2 was the best for colorectal (delta AIC=52) and Model B1 was the best for lung (delta AIC=22).
Comparison of performance between models was assessed by the difference in Nagelkerke’s R square statistic calculated for each of the Cox proportional hazard models versus the model with no comorbidities. As shown in Table 3, Model C3 was again the best performer with an improvement in R square of 2.8% over the base model (R2=0.388), followed by Model D2 (2.3%) and Model C1 (2.3%) for all cases included, as well as for each of the specific sites. It should be noted, however, that Nagelkerke’s R square calculation does not adjust for the number of terms in the model.
DISCUSSION
This study showed the importance and the need for assessing the weight of different comorbid ailments and their severity. Overall, all the models containing comorbidity performed better than the baseline model, which did not contain comorbidity. The best comorbidity scoring method was in Model C1 where the degree of organ decompensation for each individual comorbid ailment was scored as 0=None, 1=Mild, 2=Moderate, or 3=Severe. The accuracy of the models did not increase as the complexity of the models increased. Furthermore, given the observed differential impact of comorbid ailments across different cancer types, it is best to develop weights for the different comorbid ailments within each cancer type. Interestingly, the model with the best AIC value for Lung (Model B1) different from the best model (Model C1) for all cases, breast, colorectal, and prostate. Lung cancer is a very aggressive type of cancer and comorbidities do not carry as much prediction of overall survival as they do for more latent cancers.
All of the models presented in this study were created by using different combinations and scoring algorithms of the comorbid elements included in the ACE-27 index. Each comorbid condition included in the ACE-27 index was chosen based on statistical and clinical significance.[14;19] The ACE-27 index includes serious comorbidities reported as frequent conditions among cancer patients and especially among those over 60 years of age.[20] While there are many studies that focus on comparing the prognostic accomplishments of different adaptations of the Charlson’s scoring algorithm,[12;21–24] there are few studies that investigate such comparisons or evaluations using the ACE-27 index. Fang et al. reported that the chart-based ACE-27 and Charlson’s comorbidity index differed in the distribution of patients, at baseline, according to levels of comorbidity severity.[25]
This study’s analytical approach, using the Cox proportional hazard model, was performed on a large sample of cancer patients diagnosed and/or treated at Barnes Jewish Hospital/Siteman Cancer Center, a comprehensive cancer center. A standard baseline model that included baseline demographic and tumor-specific characteristics, served as the starting point and the reference for comparisons. Acknowledging that the Kaplan-Feinstein scoring of the Adult Comorbidity Evaluation-27 index did not make maximal use of our data, we did not limit ourselves to just simple variation of the overall Kaplan-Feinstein score. Instead different calculation scores and schemes that went beyond the assumption of linearity were tested as well. Since models were distinguished by addition of different comorbidity calculations over the base model, differences in statistical performance and the level of improvement of the models were due to inclusion of this comorbidity calculation. All comparisons were made using the same set of patients. The sample size was more than 10 times as large as the number of variables included in our models, which protects against the possibility of over-fitting the data.[18] The comparison of our models was based on more than one criterion. The discriminate power of each model was assessed using Harrell’s c-index. The c-index improvement in our models ranged from 0.010 to 0.086. In a logistic regression model, an improvement of 0.010 in the c-index would be associated with an improvement in classification of approximately 0.85% of the patients, and an improvement of 0.086 in the c-index would be associated with an improvement in the classification of approximately 7.3% of patients.
In addition, we also used Nagelkerke’s R square and the AIC-index, which penalizes for the number of variables included in the model, in order to have a fair comparison of models with different levels of complexity.
This study has some limitations. Our study population includes only elderly patients with cancer. It is known that the prevalence of different comorbid conditions varies across the age spectrum.[9] Elderly patients suffer from more comorbid conditions than younger patients, and the incidence of more indolent cancers is higher in the elderly compared to the general population.[4–6;11;26;27] This may limit the generalizability or external validity of the results. Nevertheless, we feel the internal validity of the data is accurate for the following reasons: comorbid health information is routinely collected prospectively by trained cancer and thorough follow-up of all patients is performed regardless of type of cancer or number of comorbid ailments.
The number of comorbid ailments included in a comorbidity index will impact on the prognostic capabilities of the index. For example, Stukenborg et al.[28] reported in 2001 that the Elixhauser[29] comorbidity model provided better discrimination than the Deyo adaption of the Charlson index.[30] Although the dataset used for comparison was the same, and both methods were adjustments of the Charlson index, the number of comorbidities captured by the Elixhauser method was nearly twice the number of comorbidities captured by the Deyo method. In contrast, our comparisons are based on the same number of comorbid ailments and try to identify better weighting of these conditions to increase the predictive power. Wang et al.[31] used a time-dependent comorbidity factor and assessed this factor in addition to baseline comorbidity to conclude that longitudinal assessment of comorbidity is an important predictor of survival. Their study also found that the best model combined both time-independent baseline comorbidity and the time-dependent comorbidity assessments. The worst fitting model included baseline comorbidity only. Overall the best fit models used the “the rolling” comorbidity measures that assumed chronic conditions persisted rather than measures using only prior years recorded diagnoses. Our study was limited in using only the time-independent baseline comorbidity, but one can argue on our approach’s support since the change in comorbidities after the cancer diagnosis can be a complication of the cancer or its treatment.[32]
A simple model (C1), which uses severity grading for each ACE-27 comorbid ailment, captures sufficient information to provide valid prognostic predictions. Model C1 is simpler than the original Kaplan-Feinstein scheme of scoring ACE-27. The accuracy of scoring did not improve substantially with models more complex than Model C1.
Acknowledgments
Partial funding for this research was provided from National Institutes of Health, National Cancer Institute RO1 CA104797 (Jay F. Piccirillo)
Footnotes
Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errors may be discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.
Author Contributions
Dr. Kallogjeri contributed to the conception and design of the manuscript, data collection, analysis and interpretation of data, and manuscript writing.
Dr. Piccirillo contributed to the conception and design of the manuscript, data collection, analysis and interpretation of data, and manuscript writing.
Prof. Spitznagel contributed to the conception and design of the manuscript, analysis and interpretation of data, and manuscript writing.
Prof. Steyerberg contributed to the conception and design of the manuscript, analysis and interpretation of data, and manuscript writing.
All authors provided final approval of the manuscript.
Disclosure Statement
The other authors have no conflict of interest to disclose.
REFERENCES
- 1.Piccirillo JF, Feinstein AR. Clinical symptoms and comorbidity: significance for the prognostic classification of cancer. Cancer. 1996 Mar 1;77(5):834–842. [PubMed] [Google Scholar]
- 2.Read WL, Tierney RM, Page NC, Costas I, Govindan R, Spitznagel EL, et al. Differential prognostic impact of comorbidity. Journal of Clinical Oncology. 2004 Aug 1;22(15):3099–3103. doi: 10.1200/JCO.2004.08.040. [DOI] [PubMed] [Google Scholar]
- 3.Surveillance Epidemiology and End Results (SEER) program. SEER Stat Fact Sheets. 2009 [Google Scholar]
- 4.Satariano WA. Comorbidities and Cancer. In: Hunter CP, Johnson KA, Muss HB, editors. Cancer in the Elderly. 1st ed. New York: Marcel Dekker; 2000. pp. 477–499. [Google Scholar]
- 5.Albertsen PC, Fryback DG, Storer BE, Kolon TF, Fine J. The impact of co-morbidity on life expectancy among men with localized prostate cancer. Journal of Urology. 1996 Jul;156(1):127–132. [PubMed] [Google Scholar]
- 6.Yancik R, Wesley MN, Ries LG, Havlik RJ, Long S, Edwards BK, et al. Comorbidity and age as predictors of risk for early mortality of male and female colon carcinoma patients - A population-based study. Cancer. 1998 Jun 1;82(11):2123–2134. [PubMed] [Google Scholar]
- 7.Smith AW, Reeve BB, Bellizzi KM, Harlan LC, Klabunde CN, Amsellem M, et al. Cancer, comorbidities, and health-related quality of life of older adults. Health Care Financing Review. 2008;29(4):41–56. [PMC free article] [PubMed] [Google Scholar]
- 8.Bellizzi KM, Mustian KM, Palesh OG, Diefenbach M. Cancer survivorship and aging : moving the science forward. Cancer. 2008 Dec 15;113(12 Suppl):3530–3539. doi: 10.1002/cncr.23942. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9.Piccirillo JF, Vlahiotis A, Barrett LB, Flood KL, Spitznagel EL, Steyerberg EW. The changing prevalence of comorbidity across the age spectrum. Critical Reviews in Oncology-Hematology. 2008 Aug;67(2):124–132. doi: 10.1016/j.critrevonc.2008.01.013. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10.Charlson ME, Pompei P, Ales KL, MacKenzie CR. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chron Dis. 1987;40(5):373–383. doi: 10.1016/0021-9681(87)90171-8. [DOI] [PubMed] [Google Scholar]
- 11.Extermann M. Measurement and impact of comorbidity in older cancer patients. Critical Reviews in Oncology-Hematology. 2000 Sep;35(3):181–200. doi: 10.1016/s1040-8428(00)00090-1. [DOI] [PubMed] [Google Scholar]
- 12.Colinet B, Bertrand D, Lacombe S, Bozonnat MC, Daures JP, Pujol JL, et al. A new simplified comorbidity score as a prognostic factor in non-small-cell lung cancer patients: description and comparison with the Charlson's index. Br J Cancer. 2005 Nov 14;93(10):1098–1105. doi: 10.1038/sj.bjc.6602836. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Kaplan MH, Feinstein AR. The importance of classifying initial co-morbidity in evaluating the outcome of diabetes mellitus. J Chron Dis. 1974;27(7–8):387–404. doi: 10.1016/0021-9681(74)90017-4. [DOI] [PubMed] [Google Scholar]
- 14.Piccirillo JF, Tierney RM, Costas I, Grove L, Spitznagel EL., Jr Prognostic importance of comorbidity in a hospital-based cancer registry. JAMA. 2004 May 26;291(20):2441–2447. doi: 10.1001/jama.291.20.2441. [DOI] [PubMed] [Google Scholar]
- 15.Johnston AS, Piccirillo JF, Creech CM, Littenberg B, Jeffe D, Spitznagel ELJ. Validation of a comorbidity education program. J Reg Manag. 2001;28(3):125–131. [Google Scholar]
- 16.Akaike H. A new look at the statistical model identification. IEEE Transactions on Automatic Control. 1974;19(6):716–723. [Google Scholar]
- 17.Steyerberg EW. Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating. Springer New York: Statistics for Biology and Health; 2009. [Google Scholar]
- 18.Harrell FE, Lee KL, Mark DB. Tutorials in biostatistics. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996 Feb 28;15(4):361–387. doi: 10.1002/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4. [DOI] [PubMed] [Google Scholar]
- 19.Piccirillo JF, Creech CM, Zequeira R, Anderson S, Johnston AS. Inclusion of comorbidity into oncology data registries. J Reg Manag. 1999;26(2):66–70. [Google Scholar]
- 20.Coebergh JWML, Post PN, Razenberg PP. Serious co-morbidity among unselected cancer patients newly diagnosed in the southeastern part of The Netherlands in 1993–1996. J Clin Epidemiol. 1999 Dec;52(12):1131–1136. doi: 10.1016/s0895-4356(99)00098-0. [DOI] [PubMed] [Google Scholar]
- 21.Sundarajan V, Quan H, Halfon P, Fushimi K, Luthi JC, Burnand B, et al. Cross-national comparative performance of three versions of the ICD-10 Charlson index. Medical Care. 2007;45(12):1210–1215. doi: 10.1097/MLR.0b013e3181484347. [DOI] [PubMed] [Google Scholar]
- 22.Cleves MA, Sanchez N, Draheim M. Evaluation of two competing methods for calculating Charlson's comorbidity index when analyzing short-term mortality using administrative data. J Clin Epidemiol. 1997;50(8):903–908. doi: 10.1016/s0895-4356(97)00091-7. [DOI] [PubMed] [Google Scholar]
- 23.Ghali WA, Hall RE, Rosen AK, Ash AS, Moskowitz MA. Searching for an improved clinical comorbidity index for use with ICD-9-CM administrative data. J Clin Epidemiol. 1996;49(3):273–278. doi: 10.1016/0895-4356(95)00564-1. [DOI] [PubMed] [Google Scholar]
- 24.D'Hoore W, Bouckaert A, Tilquin C. Practical considerations on the use of the Charlson comorbidity index with administrative data bases. J Clin Epidemiol. 1996;49(12):1429–1433. doi: 10.1016/s0895-4356(96)00271-5. [DOI] [PubMed] [Google Scholar]
- 25.Fang A, Piccirillo JF. A comparison of the claims-based ICD-9-CM and the chart-based approaches to classifying comorbidity and complications by hospital-based cancer registrars. J Reg Manag. 2006;33(1):10–16. [Google Scholar]
- 26.Wells CK, Stoller JK, Feinstein AR, Horwitz RI. Comorbid and clinical determinants of prognosis in endometrial cancer. Arch Intern Med. 1984;144(10):2004–2009. [PubMed] [Google Scholar]
- 27.Satariano WA, Ragland DR. The effect of comorbidity on 3-year survival of women with primary breast cancer. Ann Intern Med. 1994;120(2):104–110. doi: 10.7326/0003-4819-120-2-199401150-00002. [DOI] [PubMed] [Google Scholar]
- 28.Stukenborg GJ, Wagner DP, Connors AFJ. Comparison of the performance of two comorbidity measures, with and without information from prior hospitalizations. Medical Care. 2001 Jul;39(7):727–739. doi: 10.1097/00005650-200107000-00009. [DOI] [PubMed] [Google Scholar]
- 29.Elixhauser A, Steiner C, Harris R, Coffey RM. Comorbidity measures for use with administrative data. Medical Care. 1998;36(1):8–27. doi: 10.1097/00005650-199801000-00004. [DOI] [PubMed] [Google Scholar]
- 30.Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICD-9-CM administrative databases. J Clin Epidemiol. 1992;45:613–619. doi: 10.1016/0895-4356(92)90133-8. [DOI] [PubMed] [Google Scholar]
- 31.Wang CY, Baldwin LM, Saver BG, Dobie SA, Green PK, Cai Y, et al. The contribution of longitudinal comorbidity measurements to survival analysis. Medical Care. 2009;47(7):813–820. doi: 10.1097/MLR.0b013e318197929c. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32.Yung KC, Piccirillo JF. The incidence and impact of comorbidity diagnosed after the onset of head and neck cancer. Archives of Otolaryngology -- Head & Neck Surgery. 2008 Oct;134(10):1045–1049. doi: 10.1001/archotol.134.10.1045. [DOI] [PubMed] [Google Scholar]