Development and Validation of a Risk Tool for Predicting Severe Toxicity in Older Adults Receiving Chemotherapy for Early-Stage Breast Cancer

Abstract

PURPOSE

Limited tools exist to predict the risk of chemotherapy toxicity in older adults with early-stage breast cancer.

METHODS

Patients of age ≥ 65 years with stage I-III breast cancer from 16 institutions treated with neoadjuvant or adjuvant chemotherapy were prospectively evaluated for geriatric and clinical features predictive of grade 3-5 chemotherapy toxicity. Logistic regression with best-subsets selection was used to identify and incorporate independent predictors of toxicity into a model with weighted variable scoring. Model performance was evaluated using area under the ROC curve (AUC) and goodness-of-fit statistics. The model was internally and externally validated.

RESULTS

In 473 patients (283 in development and 190 in validation cohort), 46% developed grade 3-5 chemotherapy toxicities. Eight independent predictors were identified (each assigned weighted points): anthracycline use (1 point), stage II or III (3 points), planned treatment duration > 3 months (4 points), abnormal liver function (3 points), low hemoglobin (3 points), falls (4 points), limited walking (3 points), and lack of social support (3 points). We calculated risk scores for each patient and defined three risk groups: low (0-5 points), intermediate (6-11 points), or high (≥ 12 points). In the development cohort, the rates of grade 3-5 chemotherapy toxicity for these three groups were 19%, 54%, and 87%, respectively (P < .01). In the validation cohort, the corresponding toxicity rates were 27%, 45%, and 76%. The AUC was 0.75 (95% CI, 0.70 to 0.81) in the development cohort and 0.69 (95% CI, 0.62 to 0.77) in the validation cohort. Risk groups were also associated with hospitalizations and reduced dose intensity (P < .01).

CONCLUSION

The Cancer and Aging Research Group-Breast Cancer (CARG-BC) score was developed and validated to predict grade 3-5 chemotherapy toxicity in older adults with early-stage breast cancer.

INTRODUCTION

Nearly half of the patients diagnosed with breast cancer are of age ≥ 65 years.¹ With the aging of the US population, the burden of breast cancer in older adults will continue to increase.^2,3 Although adjuvant chemotherapy improves survival in early-stage breast cancer, it is significantly underused in older patients.⁴ This underutilization may be because of the increased risk of chemotherapy toxicity in older adults^5-7 and the challenges with balancing the potential benefits against the potential risks for each individual patient.^8,9

CONTEXT

• Key Objective

• To develop and validate a model that can predict grade 3-5 chemotherapy toxicity in patients of age 65 years or older with early-stage breast cancer.

• Knowledge Generated

• The Cancer and Aging Research Group-Breast Cancer (CARG-BC) score, derived by combining eight clinical and geriatric variables, was developed to classify older patients with early-stage breast cancer into low, intermediate, and high risk for grade 3-5 chemotherapy toxicity. The score was externally validated; demonstrated to better predict toxicity compared with prior models and physician-rated performance status; and was strongly associated with dose reductions, dose delays, early treatment discontinuation, reduced dose intensity, and hospitalizations.

• Relevance

• These findings may be useful to clinicians for predicting individual probability of chemotherapy toxicity and directing therapy in older adults with early-stage breast cancer. Intensifying supportive care and developing modified treatment regimens may be appropriate for subgroups identified as being vulnerable to greater toxicity.

Unlike tumor-specific genomic testing, which quantifies the potential benefits of adjuvant chemotherapy,¹⁰ limited tools exist to predict the potential harm of chemotherapy in older adults with breast cancer. Existing measures, such as the Karnofsky performance status (KPS)¹¹ or Eastern Cooperative Oncology Group performance status,¹² were developed and validated in younger patients and do not reliably assess the fitness of older adults.^13,14 Incorporating variables from a geriatric assessment (GA)¹⁵ results in more reliable tools that can better predict chemotherapy toxicity in older adults with cancer (eg, Cancer and Aging Research Group [CARG] Chemotherapy Toxicity Tool).^5-7 However, existing toxicity prediction models were developed and validated in a heterogeneous older adult population with various cancer subtypes, stages, and chemotherapy regimens. A tool that accounts for specific disease and treatment variables that are relevant for older patients with early-stage breast cancer may provide more accurate risk estimates.¹⁶

We conducted a multicenter, prospective cohort study of older adults with early-stage breast cancer who were initiating adjuvant or neoadjuvant chemotherapy. Our main objective was to develop and validate a model to predict grade 3-5 chemotherapy toxicity in older adults with early-stage breast cancer.

METHODS

Patients

The Hurria Older PatiEnts (HOPE) with Breast Cancer Cohort Study (ClinicalTrials.gov identifier: NCT01472094) accrued patients from 16 US institutions. Eligible patients were of age ≥ 65 years, with stage I-III breast cancer of any subtype, were fluent in English, and were scheduled to receive adjuvant or neoadjuvant chemotherapy per provider discretion. Between September 2011 and May 2017, 501 patients consented to participate. The study was approved by the institutional review board at each participating institution.

Study Design and Study Cohort

This is a prospective cohort study. On the basis of our prior experience,^5,6 we included a development cohort (first 300 recruited patients) and an external validation cohort (last 201 patients). Although the validation cohort was recruited from the same institutions, they were treated during a different time period, providing evidence of external validity.¹⁷

Assessment of Potential Risk Factors of Chemotherapy Toxicity

Prior to the start of adjuvant or neoadjuvant chemotherapy, we collected demographic variables (age, sex, race/ethnicity, education, marital status, and household composition), clinical characteristics (tumor stage and estrogen or progesterone or human epidemermal growth factor receptor 2 [HER2]-neu receptor status), laboratory data (hemoglobin, WBC count, albumin, creatinine, blood urea nitrogen, and liver function [normal defined as all liver tests are within the normal reference ranges for each institution]), planned treatment regimen and planned duration, and GA variables.¹⁵ The GA included a healthcare provider portion and a patient portion. The healthcare provider portion consisted of physician-rated KPS,¹¹ Timed Up & Go test (an objective measure of physical function),¹⁸ Blessed Orientation-Memory-Concentration test (a cognitive screening test),¹⁹ weight, height, body mass index, and unintentional weight loss. The patient portion⁵ consisted of self-reported measures of functional status,^20,21 comorbidity,²¹ medications, nutrition, psychological state,²⁰ and social support or function²² (Appendix Table A1, online only).

Outcomes

The primary outcome was grade 3-5 chemotherapy toxicity or adverse event (AE) as defined by the National Cancer Institute Common Terminology Criteria for Adverse Events (NCI CTCAE v 4.0).²³ Patients were followed throughout the course of chemotherapy, and AEs were captured at each cycle. Subsequently, AEs were independently reviewed by two physicians (the national study principal investigator [A.H.] and site investigator) to confirm grade 3 (severe), 4 (life-threatening or disabling), and 5 (death) AEs and that the AE was chemotherapy related. For the grade 3-5 toxicity variable, we used the highest grade throughout the treatment to classify the patient. Patients could have different types of toxicities with the same highest grade; all types were reported.

Secondary outcomes included treatment modifications (dose reductions and/or delays and early treatment termination), reduced relative dose intensity (RDI, defined as the ratio of actual dose received to planned dose < 85%), and hospitalizations.

Statistical Analyses

We used descriptive analyses to summarize demographic, clinical, and GA (individual questions from validated measures in each GA domain) variables, and the incidence of grade 3-5 chemotherapy toxicities.

Model Development

In the development cohort, we used chi-squared tests to examine baseline variables in relation to toxicity. The small number of patients with missing information were not included in the analysis (n = 24). Baseline variables associated with grade 3-5 toxicity in the univariate analysis (P < .1) and prespecified variables deemed to be of clinical relevance (planned anthracycline, planned treatment duration, and stage) were further examined in a multivariable logistic regression model. Furthermore, we used stepwise selection to identify the most significant GA variables for inclusion in the best-subsets selection and restricted the number of variables used in the best-subsets selection to 15 variables or fewer.²⁴ Bayesian information criterion (BIC) was then used to identify the best size (number of variables) of the model that predicted chemotherapy toxicity. Finally, authors reviewed the top five models with the smallest BIC scores and chose the final model on the basis of the clinical relevance of the variables included. We also evaluated interactions among the selected variables, and P values < .01 were considered significant.

Developing the Scoring System

We assigned a point value to each variable in the final model by dividing the variable's beta coefficient by the lowest beta coefficient in the model, rounding to the nearest whole number.^5,25,26 The sum of the point values for each patient comprises the individual's risk score. We divided the group into three risk strata (low, intermediate, and high) on the basis of approximate probability of grade 3-5 toxicity < 34%, 34% to < 66%, and ≥ 66%. The difference in grade 3-5 toxicity incidence among the strata was evaluated using the chi-squared test. We evaluated the discrimination ability of the model by calculating the area under the ROC curve (AUC). The model's calibration was assessed by plotting observed and predicted probability of toxicity using LOESS smoothers and by calculating the goodness-of-fit statistics using the Hosmer-Lemeshow test.²⁷

Internal Validation

The model was internally validated using 10-fold internal cross-validation and by bootstrapping to assess the extent to which the model fitting process led to an overfitting of the model.²⁸ To estimate how well the model would perform in new data sets, Harrell's method was used to calculate the overoptimism penalty of the predictive ability of our current model (Harrell's C),²⁹ which was subtracted from the AUC of the final model.

External Validation

Performance of the model was assessed in the validation cohort, the size of which allowed 80% power to detect a decrease of 0.1 in the AUC from the development cohort.³⁰ The AUC for the validation cohort was calculated and compared with the development cohort using the Delong nonparametric approach.³¹

Additional Analyses

Using the validation cohort, the predictive ability of the model was compared with two existing prediction tools: the CARG toxicity tool and the KPS, by comparing the AUC for the three tools using the Delong nonparametric approach.³¹

Additionally, we evaluated associations between our risk groups (categorized as low, intermediate, and high) and secondary outcomes (eg, treatment modifications, RDI, and hospitalization) using chi-squared tests. RDI was dichotomized using 85% as the cutpoint. All statistical analyses were performed using SAS 9.4 (SAS Institute, Cary, NC).

RESULTS

Among the 300 patients in the development cohort, 17 were excluded because of receiving nonstandard treatment regimens. Similarly, among the 201 patients in the validation cohort, 11 were excluded because of receiving nonstandard regimens (Appendix Fig A1, online only).

Sample Characteristics of the Development Cohort

The mean age of patients was 70.5 years (standard deviation [SD] 4.4, median 70 years, range 65-86 years), with 59% between the ages of 65 and 70 years. The majority were married (56.2%), non-Hispanic White (75.5%), lived with someone (72.1%), and had a college education or higher (72.2%). One hundred seventy-one (36.2%) had stage I, 203 (42.9%) stage II, and 99 (20.9%) stage III cancers. Nearly one quarter (23.7%) had triple-negative disease, and 27.7% had HER2-positive disease. Approximately one third of the patients received an anthracycline-based regimen (33.8%) (Appendix Table A2, online only), half of the patients had a planned duration of treatment ≤ 3 months, 17.3% received neoadjuvant chemotherapy, and 74.2% received primary prophylaxis with WBC growth factors (Table 1).

TABLE 1.

Distributions of Patient Demographic, Disease, and Treatment Factors in the Development and Validation Cohorts

Grade 3-5 Chemotherapy Toxicities in the Development Cohort

One hundred thirty-eight patients (48.7%) developed grade 3-5 toxicities (37.5% grade 3, 11.0% grade 4, and 0.4% grade 5 [percentages reflect worst grade of toxicity experienced for patients with multiple AEs]) in the development cohort (Table 2). Grade 3-5 hematologic and nonhematologic toxicity occurred in 26.9% and 38.5% of patients, respectively. The most common grade 3-5 hematologic toxicities were anemia (13.8%), neutropenia (9.5%), and neutropenic fever (7.1%). The most common grade 3-5 nonhematologic toxicities were fatigue (11.7%), infection with normal neutrophil count (9.9%), and dehydration (4.2%). One patient died from infection with normal absolute neutrophil count.

TABLE 2.

Chemotherapy-Related Grade 3-5 Toxicities in Development and Validation Cohorts

Development of the CARG-BC Score

The final model (Tables 3 and 4), named the Cancer and Aging Research Group-Breast Cancer (CARG-BC) score, included the following eight predictors with an AUC of 0.76: stage (II/III), planned anthracycline-based regimen, planned duration of treatment (> 3 months), abnormal liver function, anemia (hemoglobin M ≤ 13/F ≤ 12 g/dL), ≥ 1 fall in the past 6 months, limited ability to walk more than 1 mile, and lack of someone to give good advice in a crisis (Appendix Table A3, online only, with variables considered in best-subsets selection; complete univariate analysis provided in Appendix Table A4, online only). No significant interactions were found among the variables. On the basis of the bootstrapping validation, a minor model overfitting of 0.03 was identified, leaving a final adjusted AUC of 0.73. Compared with a model without the three GA variables, the addition of geriatric variables contributed significantly to the model performance (AUC 0.76 v 0.67, P = .007).

TABLE 3.

Multivariable Predictive Model

TABLE 4.

Cancer and Aging Research Group-Breast Cancer (CARG-BC) score calculator

Risk scores were assigned to each variable included in the final model (Tables 3 and 4), and the CARG-BC score was calculated as the summation of points for each patient (range of potential scores 0-24). The median CARG-BC score in the developmental cohort was 7 (range 0 to 21). The model demonstrated good discrimination and calibration (Appendix Fig A2, online only), with an AUC of 0.75 (95% CI, 0.70 to 0.81) and goodness-of-fit P value of .49. A 10-fold internal cross-validation yielded an AUC of 0.74, indicating the model retained good discrimination. Each 1-point increase in the CARG-BC score was associated with an increased odds of AE (OR = 1.28, 95% CI, 1.19 to 1.38, P < .001).

The development cohort was divided into three risk groups based on the predicted probability of toxicity: low-risk (score 0-5, < 0.34), intermediate-risk (score 6-11, 0.34-< 0.66), and high-risk (score ≥ 12, ≥ 0.6). Compared with patients in the low-risk category (n = 93), the odds of experiencing chemotherapy toxicity were almost five times greater for individuals in the intermediate-risk group (n = 159; OR = 4.91, 95% CI, 2.69 to 8.96), and 28 times greater for those in the high-risk group (n = 30; OR = 28.13, 95% CI, 9.74 to 90.56); all P values < .001.

External Validation of the CARG-BC Score

No significant differences in demographic, disease, and treatment characteristics were noted between the development and validation cohorts (Table 1). The median CARG-BC score in the validation cohort was 8 (range 0 to 18); 59 patients (31%) were classified as low-risk, 98 (52%) as intermediate-risk, and 33 (17%) as high-risk. In the validation cohort, the association between CARG-BC score and chemotherapy toxicity was slightly attenuated but still statistically significant (27% with toxicity in the low-risk group, 45% in intermediate-risk, and 76% in high-risk; P < 0.0001). The AUC for the validation cohort was 0.69 (95% CI, 0.62 to 0.77), which was not significantly different from the development cohort (P = .15). After combining the development and validation cohorts, the overall AUC for the CARG-BC score was 0.73 (95% CI, 0.68 to 0.77).

Comparison of the CARG-BC, CARG, and KPS to Predict Chemotherapy Toxicity

Using data from the validation cohort, the CARG-BC score was superior to the generalized CARG toxicity tool⁵ at predicting grade 3-5 chemotherapy toxicity (AUC = 0.69 for CARG-BC score and AUC = 0.56 for generalized CARG score, P = .004) (Figs 1A-C). Additionally, the CARG-BC score was superior to physician-rated KPS in predicting grade 3-5 chemotherapy toxicity (AUC = 0.50 for KPS, P < .001) (Appendix Table A5, online only).

FIG 1.

Association of CARG-BC score (A) with grade 3-5 chemotherapy toxicity, compared with general CARG toxicity tool (B) and physician-rated KPS (C) in the validation cohort. KPS, Karnofsky performance status. Abbreviations: CARG-BC, Cancer and Aging Research Group-Breast Cancer score; CARG, Cancer and Aging Research Group; and KPS, Karnofsky Performance Status as rated by the physician.

Association Between the CARG-BC Score and Secondary Outcomes

Among the 473 patients in the combined development and validation cohorts, 24% required an unplanned dose reduction during therapy, 26% had a dose delay, and 24% had early discontinuation of therapy. Compared with patients in the low-risk group, those in intermediate-risk and high-risk groups were more likely to have unplanned dose reduction, dose delay, and early discontinuation of therapy (all P values < .001). Twenty-five percent of patients received < 85% of the ideal chemotherapy regimen (RDI), and 23% of patients were hospitalized during treatment. Patients in the intermediate- and high-risk groups were more likely to have received reduced RDI and were more likely to be hospitalized, compared with those in the low-risk group (P < .001 for both) (Figs 2A-F).

FIG 2.

Association of the CARG-BC score with the proportion of patients observed to have grade 3-5 chemotherapy toxicity (A), hospitalizations (B), dose reductions (C), dose delays (D), early treatment discontinuation (E), and reduced relative dose intensity (F) as observed in the overall cohort.

DISCUSSION

Our study demonstrates that the CARG-BC score, derived by combining eight clinical and geriatric variables, accurately classified patients of age 65 years and older with early-stage breast cancer into low-, intermediate-, and high-risk for grade 3-5 chemotherapy toxicity. The score was also validated, demonstrated to outperform the CARG toxicity risk score and KPS, and shown to be strongly associated with treatment modifications (dose reductions, dose delays, and early treatment discontinuation), reduced RDI, and hospitalizations.

This study has several important implications. First, the CARG-BC score fills a critical knowledge gap in estimating the risk of chemotherapy toxicity in the large population of older patients with early-stage breast cancer. The decision to pursue adjuvant or neoadjuvant chemotherapy for early-stage breast cancer is often a complex one for older adults. Many individuals have high-risk disease for which chemotherapy would be indicated to reduce the risk of disease recurrence. However, the development of severe chemotherapy toxicity can compromise an older adult's ability to complete the course of chemotherapy, possibly reducing the potential benefit of treatment. The CARG-BC score was found to be associated with unplanned modifications in treatment with dose reductions, dose delays, or early termination of treatment. Although data in older patients on the impact of RDI are limited, prior studies suggest that patients receiving an RDI < 85% experience poorer relapse-free survival,^32,33 and one quarter of patients on the current study received an RDI < 85%. Although this score should not be used as the only factor in deciding whether to administer and/or alter the dose or schedule of chemotherapy, the CARG-BC score can be used to facilitate this complex decision-making process, along with clinical judgment and patient preferences.

Second, the CARG-BC score is significantly better at predicting toxicity than KPS, adding to the evidence demonstrating that performance status is less useful for older adults with cancer.^5,13 The CARG-BC model is also superior to the generalized CARG toxicity tool. Anemia, falls, limited mobility, and social factors were common predictors of grade 3-5 chemotherapy toxicity identified in both CARG-BC and CARG models. These results are reassuring, given that these variables assess prevalent geriatric-related deficits and are highly predictive of poor outcomes in the general older adult population. However, other variables, including cancer stage, regimen, planned treatment duration, and liver function were important predictors in this study and thus were included in CARG-BC model. Differences between these models suggest that for each specific cancer type, predictive models may have different variables that predict toxicity. Further research should investigate how best to optimize toxicity prediction tools for specific cancer types.

Third, this study underscores the value of integrating geriatric principles in routine oncology practice. Among the eight variables in our final model, three GA variables were found to significantly influence the model's predictive ability, contributing to significant improvement in the model performance. Two of the GA measures included in the CARG-BC are related to functional status: falls history and ability to walk more than 1 mile. It is well-established that functional status predicts morbidity and mortality in older adults across a variety of noncancer settings,^34,35 and falls have been strongly associated with vulnerability in older adults with cancer.³⁶ Through the GA, we also identified limited social support, in the form of a lack of someone to give good advice in a “crisis” from the social support survey, as a predictor of chemotherapy toxicity. Limited social support has previously been associated with increased risk of mortality after a diagnosis of breast cancer.³⁷ Limited social support may influence a patient's timeliness in recognizing symptoms and notifying the healthcare team of difficulties with chemotherapy, potentially delaying interventions that may minimize toxicities.

This study has limitations. One limitation of this analysis is that we examined grade 3-5 chemotherapy-related toxicities throughout the entire treatment period rather than within a defined time window. Hence, patients who had more cycles of treatment might have had a longer at-risk period. However, selecting a specific time window to assess toxicity would make it difficult to interpret the clinical significance of the model and was not used in prior risk predictive models.^5-7 Another limitation is that our population was highly educated, with 72.2% having a college education or higher, and results may be less representative of patients with lower educational status. Also, only a limited number of males were enrolled, and no inferences on a sex effect can be made. Furthermore, although the CARG-BC was validated in a separate cohort of patients, these patients were accrued from the same institutions as the development cohort. While this is an established method for validation,¹⁷ further validation in a more diverse population should be considered in the future. Finally, the CARG-BC model does rely on self-report data, including physical function assessment, although prior studies have demonstrated correlation between patient-reported and objective physical assessment measures.^38,39

In conclusion, we developed and validated a risk score based on eight clinical and geriatric factors that predict grade 3-5 chemotherapy toxicity in older adults with early-stage breast cancer. The risk score was also strongly associated with dose reductions, dose delays, reduced dose intensity, and hospitalizations. These findings may be useful to clinicians for predicting individual probability of chemotherapy toxicity and directing therapy, to researchers for designing and interpreting clinical trials, and to policymakers for allocating future resources for new strategies to mitigate the risk of chemotherapy toxicity.^40,41

APPENDIX

FIG A1.

CONSORT diagram.

FIG A2.

Calibration plot for the development cohort.

TABLE A1.

Geriatric Assessment Domains, Assessment Tools, and Results

TABLE A2.

List of Adjuvant/Neoadjuvant Chemotherapy Regimens

TABLE A3.

Disease, Treatment, Labs and GA Variables included in Best-Subsets Selection: Development Cohort

TABLE A4.

Univariate Analysis

TABLE A5.

Model Performance Comparison Between CARG-BC, CARG, and KPS Utilizing the Development and Validation Cohort

AUTHOR CONTRIBUTIONS

Conception and design: William P. Tew, Hyman B. Muss, Rachel A. Freedman, William Dale, Harvey J. Cohen, Anait Arsenyan

Financial support: William Dale, Arti Hurria

Administrative support: Mina S. Sedrak, William Dale, Vani Katheria

Provision of study materials or patients: Allison Magnuson, Mina S. Sedrak, William P. Tew, Hyman B. Muss, Rachel A. Freedman, Tracey O'Connor, William Dale, Supriya Mohile

Collection and assembly of data: Allison Magnuson, Mina S. Sedrak, Cary P. Gross, William P. Tew, Heidi D. Klepin, Tanya M. Wildes, Hyman B. Muss, Efrat Dotan, Rachel A. Freedman, Tracey O'Connor, William Dale, Vani Katheria, Anait Arsenyan, Abrahm Levi, Heeyoung Kim, Supriya Mohile, Arti Hurria

Data analysis and interpretation: Allison Magnuson, Mina S. Sedrak, William P. Tew, Heidi D. Klepin, Tanya M. Wildes, Hyman B. Muss, Efrat Dotan, Rachel A. Freedman, Tracey O'Connor, William Dale, Harvey J. Cohen, Abrahm Levi, Heeyoung Kim, Supriya Mohile, Can-Lan Sun

Manuscript writing: All authors

Final approval of manuscript: Allison Magnuson, Mina S. Sedrak, Cary P. Gross, William P. Tew, Heidi D. Klepin, Tanya M. Wildes, Hyman B. Muss, Efrat Dotan, Rachel A. Freedman, Tracey O'Connor, William Dale, Harvey J. Cohen, Vani Katheria, Anait Arsenyan, Abrahm Levi, Heeyoung Kim, Supriya Mohile, Can-Lan Sun

Accountable for all aspects of the work: All authors

AUTHORS’ DISCLOSURES OF POTENTIAL CONFLICTS OF INTEREST

Development and Validation of a Risk Tool for Predicting Severe Toxicity in Older Adults Receiving Chemotherapy for Early-Stage Breast Cancer

The following represents disclosure information provided by authors of this manuscript. All relationships are considered compensated unless otherwise noted. Relationships are self-held unless noted. I = Immediate Family Member, Inst = My Institution. Relationships may not relate to the subject matter of this manuscript. For more information about ASCO's conflict of interest policy, please refer to www.asco.org/rwc or ascopubs.org/jco/authors/author-center.

Open Payments is a public database containing information reported by companies about payments made to US-licensed physicians (Open Payments).

No potential conflicts of interest were reported.