Abstract
Purpose
Older adults are at increased risk for chemotherapy toxicity, and standard oncology assessment measures cannot identify those at risk. A predictive model for chemotherapy toxicity was developed (N = 500) that consisted of geriatric assessment questions and other clinical variables. This study aims to externally validate this model in an independent cohort (N = 250).
Patients and Methods
Patients age ≥ 65 years with a solid tumor, fluent in English, and who were scheduled to receive a new chemotherapy regimen were recruited from eight institutions. Risk of chemotherapy toxicity was calculated (low, medium, or high risk) on the basis of the prediction model before the start of chemotherapy. Chemotherapy-related toxicity was captured (grade 3 [hospitalization indicated], grade 4 [life threatening], and grade 5 [treatment-related death]). Validation of the prediction model was performed by calculating the area under the receiver-operating characteristic curve.
Results
The study sample (N = 250) had a mean age of 73 years (range, 65 to 94 [standard deviation, 5.8]). More than one half of patients (58%) experienced grade ≥ 3 toxicity. Risk of toxicity increased with increasing risk score (36.7% low, 62.4% medium, 70.2% high risk; P < .001). The area under the curve of the receiver-operating characteristic curve was 0.65 (95% CI, 0.58 to 0.71), which was not statistically different from the development cohort (0.72; 95% CI, 0.68 to 0.77; P = .09). There was no association between Karnofsky Performance Status and chemotherapy toxicity (P = .25).
Conclusion
This study externally validated a chemotherapy toxicity predictive model for older adults with cancer. This predictive model should be considered when discussing the risks and benefits of chemotherapy with older adults.
INTRODUCTION
Approximately 60% of all cancers and 70% of cancer mortality occurs in individuals age ≥ 65 years, defining cancer as a disease of older adults.1 Chemotherapy remains a standard component of cancer treatment. The literature is replete with studies that demonstrate that older adults experience similar chemotherapy efficacy; however, these patients are also at increased risk for chemotherapy toxicity compared with younger adults.2-4 Furthermore, older adults are less likely to be offered chemotherapy because of concerns about their capacity to endure treatment.5,6 To make personalized treatment decisions and to anticipate serious adverse effects, it is important to identify those older adults who are at risk.
Conventional oncology performance status measures, such as the Karnofsky Performance Status (KPS)7 or the Eastern Cooperative Oncology Group performance status,8 are used in patients with cancer to predict treatment toxicity and survival, regardless of age9-11; however, it is not clear if they are valid predictors of toxicity, especially in older adults, as these tools were validated in younger adults and do not address the diversity in health status of the older cancer population. Instead, it has been widely recognized that incorporating geriatric assessment tools can identify areas of vulnerability beyond chronologic age by providing an evaluation of functional status, cognition, nutrition, social support, psychological state, comorbidity, and medications.12,13 A validated tool to assess chemotherapy toxicity risk in older adults with cancer is needed.14
To determine the utility of a geriatric assessment in anticipating chemotherapy toxicity in older adults with cancer, a prospective study of 500 older adults with cancer was previously conducted. A predictive model for chemotherapy toxicity was developed that consisted of 11 questions, including factors obtained in everyday oncology practice—patient age, number of chemotherapy drugs, dosing, and laboratory values—and factors not typically used in everyday oncology practice—geriatric assessment questions.2 The model identified older adults at risk for chemotherapy toxicity, whereas the commonly used measure of performance status in oncology practice, KPS, did not. The model was internally validated using a 10-fold validation process. The aim of this study is to externally validate the predictive model in an independent cohort of older adults. This study fills a gap in the literature by providing evidence of external validation, which is needed to demonstrate generalizability of results and support translation of research into practice.
PATIENTS AND METHODS
The chemotherapy toxicity prediction model was developed from data collected in a prospective longitudinal study of 500 patients age ≥ 65 years. Study methods and derivation of the toxicity model have been previously described.2 A prediction model that consisted of 11 prechemotherapy variables—geriatric assessment questions, laboratory values, tumor characteristics, planned treatment, and age—was developed (Table 1). In the prediction model, risk scores were assigned to each variable. Total risk score ranged from 0 (lowest toxicity risk) to 19 (highest toxicity risk). The cohort was divided into three categories on the basis of this risk score: low risk (0 to 5 points), medium risk (6 to 9 points), and high risk (10 to 19 points). The treating team was blind to the calculated risk score and category of the patient.
Table 1.
Prediction Model and Scoring Algorithm for Chemotherapy Toxicity
| Variable | Value/Response | Score |
|---|---|---|
| Age of patient | ≥ 72 years | 2 |
| < 72 years | 0 | |
| Cancer type | GI or GU cancer | 2 |
| Other cancer types | 0 | |
| Planned chemotherapy dose | Standard dose | 2 |
| Dose reduced upfront | 0 | |
| Planned No. of chemotherapy drugs | Polychemotherapy | 2 |
| Monochemotherapy | 0 | |
| Hemoglobin | < 11 g/dL (male), < 10 g/dL (female) | 3 |
| ≥ 11 g/dL (male), ≥ 10 g/dL (female) | 0 | |
| Creatinine clearance (Jeliffe, ideal weight) | < 34 mL/min | 3 |
| ≥ 34 mL/min | 0 | |
| How is your hearing (with a hearing aid, if needed)? | Fair, poor, or totally deaf | 2 |
| Excellent or good | 0 | |
| No. of falls in the past 6 months | ≥ 1 | 3 |
| None | 0 | |
| Can you take your own medicine? | With some help/unable | 1 |
| Without help | 0 | |
| Does your health limit you in walking one block? | Somewhat limited/limited a lot | 2 |
| Not limited at all | 0 | |
| During the past 4 weeks, how much of the time has your physical health or emotional problems interfered with your social activities (like visiting with friends, relatives, etc)? | Limited some of the time, most of the time, or all of the time | 1 |
| Limited none of the time or a little of the time | 0 |
NOTE. See Hurria et al.2
Abbreviation: GI, gastrointestinal; GU, genitourinary.
In this article, external validation of this predictive model is described. Between April 2008 and October 2012, 250 patients age ≥ 65 years were recruited at eight institutions—six participated in the development study; two new sites participated in the validation cohort—across the United States. Eligibility criteria were age ≥ 65 years, diagnosis of any type of solid tumor, scheduled to receive a new chemotherapy regimen, and fluent in English (because all geriatric assessment measures were not validated in other languages). Patients completed the informed consent process to participate. This study was approved by the institutional review board at each participating institution.
Study procedures for the validation cohort were identical to those used in the development cohort. Patients completed a prechemotherapy geriatric assessment that captured socio-demographics, tumor and treatment variables, laboratory test results, and geriatric assessment domains. Patients were observed through the chemotherapy course and grade 3 (hospitalization indicated), grade 4 (life-threatening), and grade 5 (treatment-related death) adverse events, as defined by the National Cancer Institute Common Terminology Criteria for Adverse Events, version 3.0, were captured by chart review.15 Two physicians reviewed toxicities and concurred that they were attributable to chemotherapy. Routine blood counts checked between chemotherapy cycles were not included as hematologic toxicity. Hematologic toxicity was only considered to be attributable to chemotherapy if blood counts were checked because the patient was symptomatic and sought medical attention or if low blood counts led to a delay or modification on the day of scheduled treatment.
Statistical Analyses
Baseline characteristics of patients in the development and validation cohorts were tabulated and compared by using χ2 statistics for categorical variables and by two-sampled t tests for continuous variables. National Cancer Institute Common Terminology Criteria for Adverse Events, version 3.0, was used to capture grade 3 to 5 hematologic and nonhematologic toxicity.15
A chemotherapy toxicity score was calculated for each individual patient in the validation cohort by using the 11 prechemotherapy variables that were included in the predictive model for chemotherapy toxicity derived from the development cohort. Patients were categorized as being at low, intermediate, or high risk of toxicity on the basis of specified cut points. Observed grade 3 to 5 toxicity rates between groups were compared by using a χ2 test of proportions. This distribution of toxicity over the different risk groups was compared with the ability of KPS to predict toxicity. For this, KPS scores were divided into three groups (90 to 100, 80, and ≤ 70) and differences were likewise assessed by using a χ2 test of proportions. We assessed the validity of the model by composing receiver-operating characteristic (ROC) curves and calculating the area under the curve (also known as C-statistic) for the prediction model and KPS scores.16 Areas under the curve were compared by using the Delong nonparameteric approach.17
For approximately 80% power to detect a substantial difference in model performance, the validation sample had to include a minimum of 100 patients with and 100 patients without the study endpoint.18 This assumed that approximately 50% of patients had experienced at least one grade 3 to 5 toxicity, which is consistent with data from the development cohort. We thus increased the sample size of this validation cohort to 250 patients to exceed 80% power.
All analyses were performed in SAS (Version 9.2; SAS Institute, Cary, NC). All tests were two-sided, and a P value of < .05 was considered statistically significant.
RESULTS
Patient, Tumor, and Treatment Characteristics
Table 2 includes patient characteristics for the development and validation cohorts. Mean age for participants in the development cohort was 73.1 years (standard deviation [SD], 6.2) and 73.0 years (SD, 5.8) in the validation cohort (P = .78); 56% of patients in the development cohort were female compared with 55% in the validation cohort (P = .79). Significant differences between the two cohorts were observed for cancer type and stage. In the development cohort, more patients had gynecologic cancers (17% in the development cohort v 7% in the validation cohort), whereas in the validation cohort, breast cancer was more prevalent (11% in the development cohort v 24% in the validation cohort; P < .001). In addition, the development cohort included more patients with metastatic disease compared with the validation cohort (P = .02). There were no significant differences between the two cohorts regarding the following treatment characteristics: receiving the standard dose of chemotherapy (per National Comprehensive Cancer Network guidelines), receiving polychemotherapy (a multidrug regimen), line of therapy (first line or greater than first line), and use of growth factors.
Table 2.
Patient and Treatment Characteristics: Development and Validation Cohorts
| Characteristic | Development Cohort | Validation Cohort | P |
|---|---|---|---|
| Patient | |||
| Age, years | .78 | ||
| 65-69 | 175 (35) | 86 (34) | |
| 70-74 | 127 (25) | 67 (27) | |
| 75-79 | 105 (21) | 60 (24) | |
| 80-84 | 73 (15) | 30 (12) | |
| ≥ 85 | 20 (4) | 7 (3) | |
| Mean (SD) | 73 (6) | 73 (6) | |
| Sex | .79 | ||
| Female | 281 (56) | 138 (55) | |
| Male | 219 (44) | 112 (45) | |
| Cancer type | < .01 | ||
| Breast | 57 (11) | 59 (24) | |
| Lung | 143 (29) | 64 (26) | |
| GI | 135 (27) | 68 (27) | |
| GYN | 87 (17) | 18 (7) | |
| GU | 50 (10) | 30 (12) | |
| Other | 28 (6) | 11 (4) | |
| Cancer stage | .02 | ||
| I | 23 (5) | 10 (4) | |
| II | 59 (12) | 40 (16) | |
| III | 111 (22) | 68 (27) | |
| IV | 307 (61) | 129 (52) | |
| Other | 0 (0) | 3 (1) | |
| Education | .41 | ||
| Less than high school | 18 (4) | 14 (6) | |
| High school graduate | 175 (35) | 82 (33) | |
| Associate/bachelor’s degree | 202 (40) | 109 (44) | |
| Advanced degree | 104 (21) | 45 (18) | |
| Missing | 1 (0) | 0 (0) | |
| Race | .01 | ||
| White | 426 (85) | 206 (82) | |
| Black | 42 (8) | 20 (8) | |
| Asian | 26 (5) | 10 (5) | |
| Other | 6 (1) | 14 (6) | |
| Treatment | |||
| Standard dose | .90 | ||
| No | 120 (24) | 61 (24) | |
| Yes | 380 (76) | 189 (76) | |
| No. of chemo drugs | .96 | ||
| Monochemotherapy | 149 (30) | 74 (30) | |
| Polychemotherapy | 351 (70) | 176 (70) | |
| Line of chemotherapy | .86 | ||
| First line | 355 (71) | 176 (70) | |
| > First line | 145 (29) | 74 (30) | |
| Growth factor use | .75 | ||
| No | 294 (59) | 144 (58) | |
| Yes | 206 (41) | 106 (42) |
NOTE. Data are given as No. (%) unless otherwise noted.
Abbreviations: SD, standard deviation; GI, gastrointestinal; GYN, gynecologic; GU, genitourinary.
Geriatric assessment variables and laboratory values are presented in Table 3. The development cohort reported a slightly higher level of social activity (development cohort: mean, 56.2 [SD, 22.8]; validation cohort: mean, 60.3 [SD, 21.8]; P = .02). All other geriatric assessment characteristics showed no significant differences between the two cohorts (Table 3). Mean physician-rated KPS was 84.7 (SD, 11.4) and 85.8 (SD, 13.0) for the development and validation cohorts, respectively (P = .50). Of patients in the development cohort, 56% had abnormal liver function tests compared with 26% in the validation cohort (P < .001). There were no statistically significant differences in other laboratory variables between the two cohorts.
Table 3.
Geriatric Assessment and Laboratory Values: Development and Validation Cohorts
| Variable | Development Cohort | Validation Cohort | P |
|---|---|---|---|
| Functional status | |||
| MOS-ADL, mean (SD) | 68.5 (26) | 64.9 (27) | .10 |
| IADL, mean (SD) | 12.9 (2) | 13 (2) | .67 |
| PKPS, mean (SD) | 85.6 (14) | 84.7 (14) | .50 |
| MDKPS, mean (SD) | 84.7 (11) | 85.8 (13) | .50 |
| Falls, No. (%) | |||
| 0 | 407 (82) | 196 (78) | .32 |
| ≥ 1 | 91 (18) | 54 (22) | |
| Mean (SD) | 0.3 (0.8) | 0.4 (2.3) | |
| Nutritional status | |||
| Body mass index, mean (SD) | 26.2 (5) | 26.7 (5) | .14 |
| Percent unintentional weight loss in last 6 months, No. (%) | |||
| ≤ 6% | 328 (66) | 178 (72) | |
| > 6% | 170 (34) | 70 (28) | |
| Mean (SD) | 4.7 (6) | 4.0 (6) | .13 |
| No. of comorbid conditions, No. (%) | |||
| 0 | 48 (10) | 16 (6) | .08 |
| 1 | 110 (22) | 40 (16) | |
| 2 | 121 (24) | 67 (27) | |
| ≥ 3 | 221 (44) | 126 (51) | |
| Cognition, No. (%) | |||
| Blessed Orientation–Memory Concentration test score ≥ 11 | 29 (5.8) | 16 (6.5) | .74 |
| Psychological state, mean (SD) | |||
| Hospital anxiety and depression scale | 8.3 (6) | 8.2 (6) | .84 |
| Social support, mean (SD) | |||
| MOS-social activity survey | 56.2 (23) | 60.3 (22) | .02 |
| MOS-social support survey | 84.9 (21) | 84.2 (21) | .79 |
| Laboratory value | |||
| WBC (k/μL), mean (SD) | 7.9 (4) | 7.5 (3) | .08 |
| Albumin (g/dL), mean (SD) | 3.8 (0.5) | 3.9 (0.5) | .87 |
| Hemoglobin (<11 g/dL [male], < 10 g/dL [female]), No. (%) | 62 (12) | 33 (13) | .83 |
| Creatinine clearance ([Jelliffe, ideal weight] < 34 mL/min), No. (%) | 44 (9) | 15 (6) | .14 |
| Abnormal liver function tests, No. (%) | 267 (56) | 66 (26) | < .01 |
Abbreviations: ADL, activities of daily living; IADL, instrumental activities of daily living; MDKPS, Karnofsky physician-rated performance status; MOS, Medical Outcomes Study; PKPS, Karnofksy self-reported performance status; SD, standard deviation; WBC, white blood cell count.
Chemotherapy Toxicity
More than one half of patients in the validation cohort (58%) experienced grade 3 to 5 toxicity compared with 53% of patients in the development cohort (Table 4). Of patients in the validation cohort, 34% experienced hematologic toxicity and 55% experienced nonhematologic toxicity. The most frequent hematologic toxicities were grade 3 absolute neutrophil count (12%) and grade 3 anemia (7%). The most frequent nonhematologic toxicities were grade 3 fatigue (20%) and grade 3 infections (10%). Of patients in both the development and validation cohorts, 2% died as a result of chemotherapy toxicity.
Table 4.
Toxicity in Development Cohort and Validation Cohorts
| Toxicity Type | Development Cohort | Validation Cohort | ||||||
|---|---|---|---|---|---|---|---|---|
| Grade 3 to 5 | Grade 3 | Grade 4 | Grade 5 | Grade 3 to 5 | Grade 3 | Grade 4 | Grade 5 | |
| Hematologic and Nonhematologic | 265 (53) | 197 (39) | 58 (12) | 10 (2) | 145 (58) | 132 (53) | 38 (15) | 6 (2) |
| Hematologic | 131 (26) | 90 (18) | 39 (8) | 2 (0) | 86 (34) | 58 (23) | 26 (10) | 2 (1) |
| ANC | 57 (11) | 40 (8) | 17 (3) | 0 (0) | 51 (20) | 31 (12) | 20 (8) | 0 (0) |
| WBC | 49 (10) | 41 (8) | 8 (2) | 0 (0) | 4 (2) | 4 (2) | 0 (0) | 0 (0) |
| Hemoglobin | 48 (10) | 45 (9) | 3 (1) | 0 (0) | 20 (8) | 18 (7) | 2 (1) | 0 (0) |
| Platelets | 25 (5) | 14 (3) | 11 (2) | 0 (0) | 6 (2) | 4 (2) | 2 (1) | 0 (0) |
| Infection with abnormal ANC | 10 (2) | 7 (1) | 1 (0) | 2 (0) | 7 (3) | 6 (2) | 1 (0) | 1 (0) |
| Nonhematologic | 217 (43) | 184 (37) | 25 (5) | 8 (2) | 132 (55) | 110 (44) | 17 (7) | 5 (2) |
| Fatigue | 81 (16) | 79 (16) | 2 (0) | 0 (0) | 53 (21) | 49 (20) | 2 (1) | 0 (0) |
| Infection with normal ANC | 48 (10) | 40 (8) | 5 (1) | 3 (1) | 27 (11) | 24 (10) | 2 (1) | 1 (0) |
| Dehydration | 43 (9) | 41 (8) | 2 (0) | 0 (0) | 12 (5) | 11 (4) | 1 (0) | 0 (0) |
| Thrombosis/embolism | 22 (4) | 17 (3) | 4 (1) | 1 (0) | 6 (2) | 2 (1) | 3 (2) | 0 (0) |
| Hyponatremia | 22 (4) | 22 (4) | 0 (0) | 0 (0) | 5 (2) | 5 (2) | 0 (0) | 0 (0) |
| Diarrhea | 22 (4) | 19 (4) | 3 (1) | 0 (0) | 9 (4) | 7 (3) | 2 (1) | 0 (0) |
| Hypokalemia | 15 (3) | 15 (3) | 0 (0) | 0 (0) | 1 (0) | 1 (0) | 0 (0) | 0 (0) |
| Dyspnea | 13 (3) | 5 (1) | 7 (0) | 1 (0) | 4 (2) | 3 (1) | 1 (0) | 0 (0) |
| Syncope | 13 (3) | 13 (3) | 0 (0) | 0 (0) | 4 (2) | 4 (2) | 0 (0) | 0 (0) |
| Neuropathy | 13 (3) | 13 (3) | 0 (0) | 0 (0) | 9 (4) | 9 (4) | 0 (0) | 0 (0) |
| Nausea | 12 (2) | 12 (2) | 0 (0) | 0 (0) | 11 (4) | 10 (4) | 1 (0) | 0 (0) |
NOTE. Data are given as No. (%).
Abbreviations: ANC, absolute neutrophil count; WBC, white blood cell count.
Model Validation
Risk score ranges from 0 to 19 points and was divided into three groups (low, 0 to 5 points; medium, 6 to 9 points; high, 10 to 19 points). Most patients (53%) were classified as intermediate risk, 24% of patients were classified as low risk, and 23% were classified as high risk. In the development cohort, risk of toxicity increased with increasing risk score (Fig 1). In the validation cohort, the increase in toxicity risk with increasing risk score was slightly attenuated but statistically significant (36.7% in the low-risk group, 62.4% in the medium-risk group, and 70.2% in the high-risk group; P < .001; Fig 1). Area under the ROC curve for the predictive model in the validation cohort (0.65 [95% CI, 0.58 to 0.71]) was not statistically different from the development cohort (0.72 [95% CI, 0.68 to 0.77]; P = .09; Appendix Fig A1, online only).2
Fig 1.
Risk strata versus toxicity percentage for the (A) development and (B) validation cohorts.
Physician-rated KPS was not predictive of chemotherapy toxicity in either the development cohort (P = .19) or the validation cohort (P = .25). Area under the ROC curve for KPS was 0.54 (95% CI, 0.48 to 0.61) for the validation cohort compared with 0.53 (95% CI, 0.48 to 0.57) for the development cohort.2 The C-statistic for KPS was 0.54 (95% CI, 0.48 to 0.61), which is statistically significantly smaller than the C-statistic for the predictive model (P = .02).
DISCUSSION
This study confirms that the risk of chemotherapy toxicity is high in older adults with cancer, with more than one half of all patients experiencing grade 3 to 5 toxicity. In both the development and validation cohorts, this model had a greater ability to discriminate toxicity risk in older adults than did the present standard oncologic assessment of performance status, the KPS score. Weighing the risks and benefits of chemotherapy in older adults is challenging as there are few older adults included in randomized clinical trials to inform this risk. Furthermore, those older adults who are included in clinical trials are physically fit and not representative of the general older population.19,20 Hence, this study fills a gap in knowledge by developing a validated tool to assess chemotherapy toxicity risk among older adults who receive chemotherapy in everyday practice to aid in clinical decision making.
The high risk of chemotherapy toxicity demonstrated in this study is consistent with previous studies2-4 and can be explained by several factors. First, physiologic reserves in all organ systems decrease with age, which may influence the capacity to endure treatment. Aging is associated with decreased bone marrow reserve and an increased risk of chemotherapy-related myelosuppression.21,22 In addition, clearance of chemotherapy can be influenced by impaired renal function, which is highly prevalent in older adults.23 Furthermore, practical issues may influence adherence to supportive care. For example, patients with a poor support system may be less able to seek medical attention when they experience adverse effects. Patients with impaired hearing or cognitive impairment may not have understood how to take their supportive care medications or when to seek attention if they develop a treatment-related adverse effect.
Therefore, identifying areas of vulnerability before the start of treatment is essential. A geriatric assessment can be used for this purpose13,24 and has been shown to be feasible in oncologic practices and clinical trials.25 In addition, several studies have suggested that findings from the geriatric assessment influence treatment decisions in 20% to 50% of patients.26 The International Society for Geriatric Oncology and the National Comprehensive Cancer Network therefore advises performing at least a screening geriatric assessment in all older adults with cancer.13,24
Other chemotherapy risk scores have been developed for older adults with cancer. For example, the Chemotherapy Risk Assessment Scale for High-Age Patients was also developed in a population of older adults with cancer and predicts grade 4 hematologic or grade 3 to 4 nonhematologic toxicity on the basis of clinical and geriatric assessment variables.3 This model was developed (N = 331; ROC, 0.65) and validated (N = 187; ROC, 0.64) in older adults with cancer. Variables predicting grade 4 hematologic toxicity were diastolic blood pressure, instrumental activity of daily living score, lactate dehydrogenase, and estimated toxicity of the chemotherapy regimen on the basis of the MAX2 score. Variables predicting grade 3 and 4 nonhematologic toxicity were Eastern Cooperative Oncology Group performance status, mini-mental health status, mini-nutritional assessment, and the MAX2 score. A main difference between these two predictive models is that we sought to identify specific questions that were predictive of chemotherapy toxicity risk, rather than full measures, to increase the ease of calculation and scoring.
This study has some important limitations. Although the model was able to discriminate toxicity risk better than KPS, it was less able to discriminate between moderate and high-risk toxicity than we expected on the basis of the development study. Therefore, the tool may be best used in distinguishing patients at a lower than average risk versus those who are at a higher than average risk for chemotherapy toxicity (Appendix Figs A2 and A3, online only, Appendix Tables A1 and A2, online only). In addition, the area under the ROC was slightly attenuated in the current study; however, this was not statistically different. A decrease of 0.07 points in the area under the ROC from development to validation has been described in the statistical literature as within an expected range.27 There were some differences in patient characteristics between the two cohorts that may explain part of the decrease in the area under the ROC. Furthermore, we only predicted grade 3 to 5 toxicity, but grade 2 toxicity may be equally important in this older population. Patients included in this cohort had solid tumors and did not receive biologics or high-dose chemotherapy; therefore, these results primarily apply to patients with solid tumors who receive chemotherapy. Finally, to use the model in other populations, such as in countries outside of the United States or in patients with different cancer types, additional external validation studies must be performed.
This study has some major strengths as well. This study fills a gap in knowledge by providing evidence of external validation of a prediction model that is needed to demonstrate generalizability of results and support translation of research into practice. Furthermore, this model was validated in a prospective cohort of patients seen in daily practice at eight institutions across the United States, which increases the generalizability of the data. The prediction model is easy to use, thus increasing the feasibility of incorporation in daily practice, and the model can provide patients and oncologists with additional information to be included in the decision making process. Discussing the risks of treatment is essential to enable patients to make a well-informed decision regarding treatment. Moreover, it may enable oncologists to anticipate toxicity in patients with a high risk and to take preventive measures to try to decrease this risk.
There are a number of future directions for this research. First, toxicity may depend on the specific treatment. Hence, there are several new ongoing, multicenter studies evaluating the efficacy of this tool in predicting toxicity among patients with specific tumor types and treatment regimens. To decrease the impact of chemotherapy toxicity in older adults with cancer, future studies should assess how the trajectory of toxicity can be changed. In particular, studies are underway to determine whether geriatric assessment targeted interventions can decrease chemotherapy toxicity and improve functional outcomes and quality of life in older adults with cancer (ClinicalTrials.gov: #NCT02054741). Furthermore, studies are needed to understand how geriatric assessment and toxicity prediction results should impact treatment regimen, dosing, and supportive care. Impact of treatment toxicity risk on doctor and patient treatment preferences and decisions must be evaluated. Finally, the role of this tool in nonchemotherapy regimens, that is, targeted therapy, and among patients with hematologic malignancies is an area of needed investigation.
In conclusion, this external validation study confirms that it is possible to predict chemotherapy toxicity in older adults with cancer by using a prediction model that consists of 11 questions, including five geriatric assessment questions and six items captured in routine daily practice—tumor type, treatment, and laboratory values. These data should be considered when the risks and benefits of chemotherapy are discussed in older adults. Future studies should assess possible interventions that can decrease the risk of toxicity in older adults with cancer.
Appendix
The following institutions participated in the development cohort:
City of Hope Comprehensive Cancer Center and Beckman Research Institute, Duarte, CA
University of Rochester Medical Center, Rochester, NY
Upstate Medical University and Syracuse VA Medical Center, Syracuse, NY
Wake Forest University, Winston Salem, NC
Yale University, New Haven, CT
Case Western Reserve University, Cleveland, OH
Memorial Sloan Kettering Cancer Center, New York, NY
The following institutions participated in the validation cohort:
City of Hope Comprehensive Cancer Center and Beckman Research Institute, Duarte, CA
University of Rochester Medical Center, Rochester, NY
Upstate Medical University and Syracuse VA Medical Center, Syracuse, NY
Wake Forest University, Winston Salem, NC
University of North Carolina at Chapel Hill, Lineberger Comprehensive Cancer Center, Chapel Hill, NC
Sidney Kimmel Cancer Center, Thomas Jefferson University, Philadelphia, PA
Memorial Sloan Kettering Cancer Center, New York, NY
Case Western Reserve University, Cleveland, OH
Fig A1.
Receiver-operating characteristic curves for development and validation cohorts.
Fig A2.
Low risk versus combining medium and high-risk scores for the development cohort.
Fig A3.
Low risk versus combining medium and high-risk scores for the validation cohort.
Table A1.
Summary Statistics for Development Cohort
| Risk Strata | Mean | Median | SD | Range |
|---|---|---|---|---|
| Low | 3.9 | 4 | 1.0 | 0-5 |
| High | 8.7 | 8 | 2.5 | 6-19 |
Abbreviation: SD, standard deviation.
Table A2.
Summary Statistics for Validation Cohort
| Risk Strata | Mean | Median | SD | Range |
|---|---|---|---|---|
| Low | 3.7 | 4 | 1.3 | 0-5 |
| High | 8.7 | 8 | 2.6 | 6-18 |
Abbreviation: SD, standard deviation.
Footnotes
Written on behalf of the Cancer and Aging Research Group.
Supported by the National Institutes of Health, National Institute on Aging Grant No. K23-AG026749-01 (A.H.), Paul Beeson Career Development Award in Aging Research, and American Society of Clinical Oncology, Association of Specialty Professors, Junior Development Award in Geriatric Oncology (A.H.).
Research reported in this publication included work performed in the Survey Research Core supported by the National Cancer Institute of the National Institutes of Health under award number P30CA33572. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Authors’ disclosures of potential conflicts of interest are found in the article online at www.jco.org. Author contributions are found at the end of this article.
AUTHOR CONTRIBUTIONS
Conception and design: Arti Hurria
Financial support: Arti Hurria
Administrative support: Arti Hurria
Provision of study materials or patients: Arti Hurria, Supriya Mohile, Ajeet Gajra, Heidi Klepin, Hyman Muss, Andrew Chapman, William P. Tew
Collection and assembly of data: Arti Hurria, Supriya Mohile, Ajeet Gajra, Heidi Klepin, Hyman Muss, Andrew Chapman, Vani Katheria, Caroline Doan, Laura Zavala, Abrahm Levi, Chie Akiba, William P. Tew
Data analysis and interpretation: Arti Hurria, Supriya Mohile, Ajeet Gajra, Heidi Klepin, Hyman Muss, Tao Feng, David Smith, Can-Lan Sun, Nienke De Glas, Harvey Jay Cohen, Vani Katheria, William P. Tew
Manuscript writing: All authors
Final approval of manuscript: All authors
AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST
Validation of a Prediction Tool for Chemotherapy Toxicity in Older Adults With Cancer
The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or jco.ascopubs.org/site/ifc.
Arti Hurria
Consulting or Advisory Role: GTx, Seattle Genetics, Boehringer Ingelheim, On Q Health, Sanofi
Research Funding: GlaxoSmithKline (Inst), Celgene (Inst), Novartis (Inst)
Supriya Mohile
Consulting or Advisory Role: Seattle Genetics
Ajeet Gajra
Consulting or Advisory Role: Celgene, Bayer AG
Research Funding: Celgene, Merck
Travel, Accommodations, Expenses: Celgene
Heidi Klepin
Consulting or Advisory Role: Celgene
Patents, Royalties, Other Intellectual Property: UpToDate
Travel, Accommodations, Expenses: Celgene, Genentech
Hyman Muss
No relationship to disclose
Andrew Chapman
No relationship to disclose
Tao Feng
No relationship to disclose
David Smith
No relationship to disclose
Can-Lan Sun
No relationship to disclose
Nienke De Glas
No relationship to disclose
Harvey Jay Cohen
No relationship to disclose
Vani Katheria
No relationship to disclose
Caroline Doan
No relationship to disclose
Laura Zavala
No relationship to disclose
Abrahm Levi
No relationship to disclose
Chie Akiba
No relationship to disclose
William P. Tew
No relationship to disclose
REFERENCES
- 1.Siegel RL, Miller KD, Jemal A. Cancer statistics, 2015. CA Cancer J Clin. 2015;65:5–29. doi: 10.3322/caac.21254. [DOI] [PubMed] [Google Scholar]
- 2.Hurria A, Togawa K, Mohile SG, et al. Predicting chemotherapy toxicity in older adults with cancer: A prospective multicenter study. J Clin Oncol. 2011;29:3457–3465. doi: 10.1200/JCO.2011.34.7625. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3.Extermann M, Boler I, Reich RR, et al. Predicting the risk of chemotherapy toxicity in older patients: The Chemotherapy Risk Assessment Scale for High-Age Patients (CRASH) score. Cancer. 2012;118:3377–3386. doi: 10.1002/cncr.26646. [DOI] [PubMed] [Google Scholar]
- 4.Muss HB, Berry DA, Cirrincione C, et al. Toxicity of older and younger patients treated with adjuvant chemotherapy for node-positive breast cancer: The Cancer and Leukemia Group B Experience. J Clin Oncol. 2007;25:3699–3704. doi: 10.1200/JCO.2007.10.9710. [DOI] [PubMed] [Google Scholar]
- 5.Hurria A, Wong FL, Villaluna D, et al. Role of age and health in treatment recommendations for older adults with breast cancer: The perspective of oncologists and primary care providers. J Clin Oncol. 2008;26:5386–5392. doi: 10.1200/JCO.2008.17.6891. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6.Kornblith AB, Kemeny M, Peterson BL, et al. Survey of oncologists’ perceptions of barriers to accrual of older patients with breast carcinoma to clinical trials. Cancer. 2002;95:989–996. doi: 10.1002/cncr.10792. [DOI] [PubMed] [Google Scholar]
- 7.Macleod CM. in Evaluation of Chemotherapeutic Agents. New York, NY: Columbia University Press; 1948. (ed): The clinical evaluation of chemotherapeutic agents in cancer; pp. 191–205. [Google Scholar]
- 8.Zubrod C, Schneiderman M, Frei E. Appraisal of methods for the study of chemotherapy of cancer in man: Comparative therapeutic trial of nitrogen mustard and triethylene thiophosphoramide. J Chronic Dis. 1960;11:7–33. [Google Scholar]
- 9.Bajorin DF, Dodd PM, Mazumdar M, et al. Long-term survival in metastatic transitional-cell carcinoma and prognostic factors predicting outcome of therapy. J Clin Oncol. 1999;17:3173–3181. doi: 10.1200/JCO.1999.17.10.3173. [DOI] [PubMed] [Google Scholar]
- 10.Motzer RJ, Bacik J, Schwartz LH, et al. Prognostic factors for survival in previously treated patients with metastatic renal cell carcinoma. J Clin Oncol. 2004;22:454–463. doi: 10.1200/JCO.2004.06.132. [DOI] [PubMed] [Google Scholar]
- 11.Albain KS, Crowley JJ, LeBlanc M, et al. Survival determinants in extensive-stage non-small-cell lung cancer: The Southwest Oncology Group experience. J Clin Oncol. 1991;9:1618–1626. doi: 10.1200/JCO.1991.9.9.1618. [DOI] [PubMed] [Google Scholar]
- 12.Chiang LY, Liu J, Flood KL, et al. Geriatric assessment as predictors of hospital readmission in older adults with cancer. J Geriatr Oncol. 2015;6:254–261. doi: 10.1016/j.jgo.2015.04.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 13.Wildiers H, Heeren P, Puts M, et al. International Society of Geriatric Oncology consensus on geriatric assessment in older patients with cancer. J Clin Oncol. 2014;32:2595–2603. doi: 10.1200/JCO.2013.54.8347. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14.Dale W, Mohile SG, Eldadah BA, et al. Biological, clinical, and psychosocial correlates at the interface of cancer and aging research. J Natl Cancer Inst. 2012;104:581–589. doi: 10.1093/jnci/djs145. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 15.Trotti A, Colevas AD, Setser A, et al. CTCAE v3.0: Development of a comprehensive grading system for the adverse effects of cancer treatment. Semin Radiat Oncol. 2003;13:176–181. doi: 10.1016/S1053-4296(03)00031-6. [DOI] [PubMed] [Google Scholar]
- 16.Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143:29–36. doi: 10.1148/radiology.143.1.7063747. [DOI] [PubMed] [Google Scholar]
- 17.DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics. 1988;44:837–845. [PubMed] [Google Scholar]
- 18.Vergouwe Y, Steyerberg EW, Eijkemans MJ, et al. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. J Clin Epidemiol. 2005;58:475–483. doi: 10.1016/j.jclinepi.2004.06.017. [DOI] [PubMed] [Google Scholar]
- 19.Hutchins LF, Unger JM, Crowley JJ, et al. Underrepresentation of patients 65 years of age or older in cancer-treatment trials. N Engl J Med. 1999;341:2061–2067. doi: 10.1056/NEJM199912303412706. [DOI] [PubMed] [Google Scholar]
- 20.Zulman DM, Sussman JB, Chen X, et al. Examining the evidence: A systematic review of the inclusion and analysis of older adults in randomized controlled trials. J Gen Intern Med. 2011;26:783–790. doi: 10.1007/s11606-010-1629-x. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21.Dees EC, O’Reilly S, Goodman SN, et al. A prospective pharmacologic evaluation of age-related toxicity of adjuvant chemotherapy in women with breast cancer. Cancer Invest. 2000;18:521–529. doi: 10.3109/07357900009012191. [DOI] [PubMed] [Google Scholar]
- 22.Gómez H, Hidalgo M, Casanova L, et al. Risk factors for treatment-related death in elderly patients with aggressive non-Hodgkin’s lymphoma: Results of a multivariate analysis. J Clin Oncol. 1998;16:2065–2069. doi: 10.1200/JCO.1998.16.6.2065. [DOI] [PubMed] [Google Scholar]
- 23.Lindeman RD, Tobin J, Shock NW. Longitudinal studies on the rate of decline in renal function with age. J Am Geriatr Soc. 1985;33:278–285. doi: 10.1111/j.1532-5415.1985.tb07117.x. [DOI] [PubMed] [Google Scholar]
- 24.Hurria A, Wildes T, Blair SL, et al. Senior adult oncology, version 2.2014: Clinical practice guidelines in oncology. J Natl Compr Canc Netw. 2014;12:82–126. doi: 10.6004/jnccn.2014.0009. [DOI] [PubMed] [Google Scholar]
- 25.Hurria A, Cirrincione CT, Muss HB, et al. Implementing a geriatric assessment in cooperative group clinical cancer trials: CALGB 360401. J Clin Oncol. 2011;29:1290–1296. doi: 10.1200/JCO.2010.30.6985. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 26.Hamaker ME, Schiphorst AH, ten Bokkel Huinink D, et al. The effect of a geriatric evaluation on treatment decisions for older cancer patients--A systematic review. Acta Oncol. 2014;53:289–296. doi: 10.3109/0284186X.2013.840741. [DOI] [PubMed] [Google Scholar]
- 27.Steyerberg EW, Bleeker SE, Moll HA, et al. Internal and external validation of predictive models: A simulation study of bias and precision in small samples. J Clin Epidemiol. 2003;56:441–447. doi: 10.1016/s0895-4356(03)00047-7. [DOI] [PubMed] [Google Scholar]