Prediction of cognitive decline in older breast cancer survivors: the Thinking and Living with Cancer study

Abstract

Purpose

Cancer survivors commonly report cognitive declines after cancer therapy. Due to the complex etiology of cancer-related cognitive decline (CRCD), predicting who will be at risk of CRCD remains a clinical challenge. We developed a model to predict breast cancer survivors who would experience CRCD after systematic treatment.

Methods

We used the Thinking and Living with Cancer study, a large ongoing multisite prospective study of older breast cancer survivors with complete assessments pre-systemic therapy, 12 months and 24 months after initiation of systemic therapy. Cognition was measured using neuropsychological testing of attention, processing speed, and executive function (APE). CRCD was defined as a 0.25 SD (of observed changes from baseline to 12 months in matched controls) decline or greater in APE score from baseline to 12 months (transient) or persistent as a decline 0.25 SD or greater sustained to 24 months. We used machine learning approaches to predict CRCD using baseline demographics, tumor characteristics and treatment, genotypes, comorbidity, and self-reported physical, psychosocial, and cognitive function.

Results

Thirty-two percent of survivors had transient cognitive decline, and 41% of these women experienced persistent decline. Prediction of CRCD was good: yielding an area under the curve of 0.75 and 0.79 for transient and persistent decline, respectively. Variables most informative in predicting CRCD included apolipoprotein E4 positivity, tumor HER2 positivity, obesity, cardiovascular comorbidities, more prescription medications, and higher baseline APE score.

Conclusions

Our proof-of-concept tool demonstrates our prediction models are potentially useful to predict risk of CRCD. Future research is needed to validate this approach for predicting CRCD in routine practice settings.

Many breast cancer survivors report cognitive decline after cancer therapy, including subtle difficulties with attention and organizing daily tasks, slowness of processing new information and memory difficulties (1-9). This constellation of symptoms is generally referred to as cancer-related cognitive decline (CRCD). CRCD is one of the most concerning adverse effects of cancer and its treatments among survivors (10), although it is not universal, and when it occurs, symptoms are variable in severity and duration (11).

The etiology of CRCD remains elusive. Several studies have linked CRCD to receipt of breast cancer chemotherapy and/or endocrine treatments (5,12-21). Genotypes associated with neurodegenerative diseases, such as apolipoprotein E4 (ApoE4), catechol-O-methyl transferase (COMT), and brain-derived neurotrophic factor (BDNF) have also been associated with CRCD (18,22,23). Tumor characteristics (24), serum inflammatory biomarkers, or magnetic resonance imaging (MRI) changes have also been observed in CRCD (25-27). Finally, lifestyle and factors related to system reserve have also been associated with CRCD, such as frailty, mood or sleep disorders, and low physical activity (28-37). However, there are few large prospective studies that include multiple risk factors and begin assessment of cognition before the initiation of systemic therapy, and even fewer focused on older breast cancer survivors, a group that is at increased risk of CRCD due to the effects of multiple chronic conditions and normal aging processes (9,38-40). Additionally, there are no existing clinical tools that synthesize information about potential CRCD risk factors that could be used to predict risk of CRCD. Machine learning (ML) can be useful to build predictive models for use in medical practice (41-44). In a previous report using machine learning, we found that decline on a constellation of neuropsychological tests characterized older breast cancer survivors with self-reported cognitive problems (45).

In this study, we used pre-systemic therapy data from the Thinking and Living with Cancer (TLC) study (9), a large ongoing multisite prospective study of older breast cancer survivors, to develop a proof-of-concept tool to predict survivors who would experience declines on neuropsychological test scores at 12- and 24-month follow-up. The preliminary results are intended to support future research to refine and validate our tool and test its utility in clinical practice, supporting conversations to facilitate personalized cancer care decisions that consider the risk and potential mitigation of CRCD.

Methods

Study design and sample

The TLC study has been described in detail elsewhere (9). All applicable Institutional Review Boards approved the protocol (NCT03451383). For this secondary analysis, we included breast cancer survivors recruited between August 1, 2010 and March 1, 2020, and followed annually; enrollment in the study is ongoing. Eligible survivors completed assessments in English, were 60 or more years of age, and were newly diagnosed with primary nonmetastatic breast cancer. We excluded survivors with neurological disorders, hearing and vision impairments, a history of stroke, bipolar disorder or schizophrenia, substance abuse, and a prior history of cancer (if active treatment occurred less than 5 years before enrollment or included systemic therapy). Data from survivors who experienced a recurrence were excluded from the period of 6 months before the date of recurrence though the end of follow-up. Women were required to have a score of 24 or greater on the Mini-Mental State Examination (MMSE) (46) and 3rd grade equivalent reading level or higher on the Wide Range Achievement Test, 4th edition (WRAT-4) Word Reading subtest (47).

Among 703 enrolled survivors, we included 228 survivors that had completed all three baseline, 12-, and 24-month assessments on or before March 1, 2020, when the study closed temporarily due to the COVID-19 pandemic (Figure 1). The majority of survivors were excluded due to missing one follow-up assessment but completing other later assessments (n = 76), administrative and site-related losses (n = 5), and death or dropout (n = 5). The analytic sample was comparable to the source population of women enrolled before December 31, 2017 (ie, had the opportunity to complete baseline, 12-, and 24-month assessments before March 2020). Women excluded from this analysis due to missing data were similar to those included in terms of sociodemographic, clinical, and psychosocial characteristics, including baseline cognitive scores (data not shown).

Figure 1.

Sample for longitudinal evaluation of cognition in older breast cancer survivors and matched controls without cancer, from the Thinking and Living with Cancer (TLC) Study. Only survivors who were enrolled and completed baseline (pre-systemic therapy), 12-month, and 24-month neurocognitive assessments before March 1, 2020 were included for analysis.

Measures

The primary endpoint for this analysis was the change over time in neuropsychological performance on a battery of six tests of attention, processing speed, and executive function (APE) (Supplementary Page 1, available online). Test results are z-scored based on a contemporary noncancer control sample matched on age, education, racial group, and recruitment site, since these were the best normative values for the study objectives (48). Scores were summed to create a single domain score based on past analyses (49).

There are no established cut points to define decline on neuropsychological test performance in the setting of CRCD. Here we establish a cut-point to define decline in survivors as a change in APE domain score greater than 0.25 standard deviations (SD) of the change observed in matched controls from their baseline assessment to the 12-month follow-up assessment. This equates to a change of 0.10 or more in standardized APE domain score from the baseline assessment. Then we defined two groups of likely clinical interest: 1) survivors with transient decline from pre-systemic therapy to 12 months who may only be experiencing acute symptoms with subsequent recovery at follow-up visits, and 2) survivors with persistent cognitive decline who had a first decline of 0.25 SD or more at 12 months and sustained that decline of 0.25 SD or more at the 24-month assessment. In our past research, we found that survivors with clinically meaningful self-reported cognitive problems had APE scores 0.25 SD below their matched control group mean (45). Additionally, the 0.25 SD cutoff was believed to represent the subtle deficits commonly reported in CDCD, especially when many consider the failure to improve on repeated testing over time (ie, practice effects) to represent a cognitive deficit (50). Finally, in developing a preliminary tool, we chose the 0.25 cut-point to maximize sensitivity to detection of subtle CRCD as relatively minor changes in objectively measured cognition may manifest as CRCD. This cut-point may increase false-positive results since some women would have this degree of statistical variation in their APE scores without true cognitive decline. This choice was made since we did not want to miss women with mild deficits where monitoring and interventions could be effective.

We included 108 predictive baseline, pre-systemic therapy variables based on published literature (9,18,22-24,29-34,51-57). Supplementary Table 1 (available online) summarizes the full list of variables; variables with greater than 10% missing data were excluded (n = 14 predictors in our dataset), and variables with less than 10% missing were imputed using nearest neighbor imputation (58). Briefly, we included demographic characteristics, pathological tumor features, treatment, genotypes (ApoE4, COMT, BDNF polymorphisms), medical history (eg, family history of dementia, medication use, comorbidities, categories of comorbidities, age of menopause), deficit accumulation index (frailty), lifestyle behaviors (cigarette smoking, alcohol use, physical activity, sleep disturbances), cognitive reserve (education, WRAT-4 score), depressive symptoms, anxiety, social support, and fatigue. Highly correlated variables, such as prescription medications such as statins and cardiovascular comorbidities, were collapsed into single summary variables (see Supplementary Figure 1, available online for correlations among variables).

Statistical approach

Our goal was to predict risk of decline among women with complete data for the three assessments (baseline and 12- and 24-month follow-up). We assumed the missing data mechanism was missing at random since in our past work (45) we found that participants with missing data were similar to the overall cohort and the complete case approach did not lead to biased estimates (59). To identify predictors of risk of cognitive decline, we evaluated several standard machine learning (ML) methods: (1) a logistic regression model with least absolute shrinkage and selection operator (LASSO) (60), (2) a logistic regression model with elastic net regulation (Elastic-net) (61), (3) a decision tree method using random forest (RF) (62), and (4) stochastic gradient boosting machine learning (SGB) (63). Conventional regression models, including those that use backward, forward, or stepwise variable selection methods in which covariates are added or subtracted from the model in an iterative fashion, are constrained by the sample size and rate of events and can suffer from increased error in prediction when the number of covariates is large. In contrast, machine learning approaches such as LASSO and Elastic-net were developed to allow inference on datasets with a large number of covariates relative to the sample size or rate of events while reducing model overfitting and accounting for correlated variables, respectively. These approaches perform variable selection and model prediction simultaneously by shrinking the regression coefficient of less informative covariates toward zero, decreasing their influence on the predictive model. We primarily report LASSO results because it yields coefficients that are clinically quantifiable and interpretable and prioritizes a smaller subset of variables among colinear variables than Elastic-net but preserves comparable and/or better predictive performance than others. Other ML results and model optimization procedures are summarized in Supplementary Materials (available online). All models also evaluated many possible pairwise interactions, including those of ApoE4 with tumor characteristics and therapy, driven by our previously published work (64). Age was not related to cognitive decline in this sample and was not included in the models, but we retained treatment (chemotherapy with or without endocrine treatment vs the reference group of endocrine therapy alone) and HER2 status in final models based on our prior work suggesting that treatment type and this molecular subtype can affect pre- and/or post-systemic therapy cognitive function (9,24). We then fit a logistic regression using the LASSO selected predictors. Regression coefficients for the final models with LASSO selected predictors are reported as odds ratios (OR) in Tables 2 and 3 and interpreted using probability with 95% confidence interval (CI) for decline in Table 4. Supplementary Tables 3 and 4 (available online) report the logistic regression coefficients using LASSO selected predictors with associated P-values, z-values, and standard errors. Given that our study is a proof-of-concept and hypothesis generating, we did not restrict to statistical significance or adjust for multiplicity. This will be important in future studies replicating our approach in other groups of women with breast cancer.

Table 2.

LASSO selected predictors of the risk of 12-month transient cognitive decline among older women with breast cancer^a

Model with baseline APE domain scoreb		Model without baseline APE domain scoref
Variable	Odds ratio a	Variable	Odds ratio a
Treatment type—chemoc	0.99	Treatment type—chemo	0.98
Cardiovascular disease comorbidity	1.64	Cardiovascular disease comorbidity	1.56
BMI >30d	2.07e	BMI >30	1.96
ApoE4 positivity (vs E4 negative)	3.60	ApoE4 positivity (vs E4 negative)	3.51
Number of prescription drugs regularly taken, per one drug	1.12	Number of prescription drugs regularly taken, per one drug	1.10
HER2 positivity (vs negative)	0.19	HER2 positivity (vs negative)	0.25
Baseline APE domain z-score	2.35	Baseline APE domain score	—

^aLogistic regression model with least absolute shrinkage and selection operator (LASSO) was used to identify predictors indicative of risk of cognitive decline. Fourfold cross-validation was used during the model optimization of tuning and penalty parameters to obtain the model selected variables and corresponding regression coefficient estimates for the displayed LASSO model outputs. Regression coefficients are summarized using odds ratios (OR) of being decliners in LASSO logistic regression. The model output in the left panel corresponds to an area under the curve (AUC) of 0.750, while the model without a measure of baseline cognitive reserve (APE domain score) corresponds to an AUC of 0.710.

^bThe APE domain is a summary Z-score of performance on six neuropsychologic tests of Attention, Processing speed, and Executive function.

^cSystemic treatment categories are for chemotherapy with or without endocrine therapy vs the referent group of endocrine therapy alone.

^dBMI = weight (kg)/height (m²) at baseline.

^eExample interpretation: For a cancer survivor, moving from the category BMI <30 to the category BMI > 30, the odds of cognitive decline increase by 2.069 times, more than doubling the odds of cognitive decline.

^fMMSE, WRAT-4, years of education as a substitute measure of baseline cognitive reserves.

Table 3.

LASSO selected predictors of the risk of 24-month persistent cognitive decline among older women with breast cancer^a

Model with baseline APE domain scoreb		Model without baseline APE domain scoreg
Variable	Odds ratio a	Variable	Odds ratio a
Treatment type—chemoc	0.76	Treatment type—chemo	0.58
Cardiovascular disease comorbidity	1.35	Cardiovascular disease comorbidity	1.62
BMI >30d	2.21f	BMI >30	2.68
ApoE4 positivity (vs E4 negative)	2.47	ApoE4 positivity (vs E4 negative)	2.60
Number of prescription drugs regularly taken, per one drug	1.33	Number of prescription drugs regularly taken, per one drug	1.36
HER2 positivity (vs negative)	0.66	HER2 positivity (vs negative)	0.62
IPAQ daily sitting hoursh	0.86	IPAQ daily sitting hours	0.86
SF12 mental component scorei	0.87	SF12 mental component score	0.84
Baseline APE domain score	1.77	Baseline APE domain score	—
Baseline mini mental status exame	—	Baseline mini mental status exam	0.55
Years of education	—	Years of education	1.18

^aLogistic regression model with least absolute shrinkage and selection operator (LASSO) was used to identify predictors indicative of risk of cognitive decline. Fourfold cross-validation was used during the model optimization of tuning/penalty parameters to obtain the model selected variables and corresponding regression coefficient estimates for the displayed LASSO model outputs. Regression coefficients are summarized using odds ratios (OR) of being decliners in LASSO logistic regression. The model output in the left panel corresponds to an area under the curve (AUC) of 0.79, while the model on the right panel with MMSE and years of education as a substitute measure of baseline cognitive reserve corresponds to an AUC of 0.83.

^bThe APE domain is a summary z-score of performance on six neuropsychologic tests of Attention, Processing speed, and Executive function.

^cSystemic treatment categories are for chemotherapy with or without endocrine therapy vs the referent group of endocrine therapy alone.

^dBMI = weight (kg)/height (m²) at baseline.

^eThe Mini Mental Status Exam scores are truncated at >24 based on study eligibility.

^fExample interpretation: For a cancer survivor, moving from the category BMI <30 to the category BMI > 30, the odds of cognitive decline increase by 2.21 times, more than doubling the odds of cognitive decline.

^gMMSE, WRAT-4, years of education as a substitute measure of baseline cognitive reserves.

^hIPAQ = International Physical Activity Questionnaire.

ⁱThe SF12 questionnaire is a health-related quality-of-life questionnaire consisting of twelve questions that measure eight health domains to assess physical and mental health. The mental health-related component includes Vitality, Social Functioning, Role Emotional, and Mental Health scales.

Table 4.

LASSO model prediction for cognitive decline under varying scenarios

Scenarioa	Treatment typeb	ApoE4 positivity	Cardiovascular disease comorbidityc	BMI ≥30	Number of prescription drugs regularly takend	HER2 Positivity	Baseline APE domain score	IPAQ daily sedentary timef	SF12 Mental Component Scoreg	Probability of transient decline (95% CI)e	Probability of persistent decline (95% CI)
a	Chemo	No	No	No	2	No	0.30	8	57	0.19 (0.09 to 0.34)	0.05 (0.02 to 0.13)
b	Chemo	Yes	No	No	2	No	0.30	8	57	0.45 (0.25 to 0.68)	0.11 (0.032 to 0.30)
c	Chemo	Yes	Yes	No	2	No	0.30	8	57	0.57 (0.35 to 0.78)	0.14 (0.04 to 0.36)
d	Chemo	Yes	Yes	Yes	2	No	0.30	8	57	0.74 (0.49 to 0.89)	0.26 (0.09 to 0.56)
e	Chemo	Yes	Yes	Yes	5	No	0.30	8	57	0.80 (0.60 to 0.92)	0.45 (0.20 to 0.73)
f	Chemo	Yes	Yes	Yes	5	Yes	0.30	8	57	0.43 (0.11 to 0.88)	0.32 (0.06 to 0.78)
g	Chemo	Yes	Yes	Yes	5	Yes	0.30	14	57		0.15 (0.02 to 0.63)
h	Chemo	Yes	Yes	Yes	5	Yes	0.30	14	40		0.65 (0.10 to 0.97)

^aScenario represents an actual survivor risk profile from our analytical sample. The prediction probabilities using our LASSO model for transient 12-month and persistent 24-month cognitive declines, and their corresponding 95% CIs are shown. The top row represents this survivor’s actual profile, following rows illustrate the effects of this survivor’s risk of cognitive decline if her profile is modified. Risk factors changes from scenario a are noted in bold text in the table.

^bSystemic treatment categories are for chemotherapy with or without endocrine therapy vs the referent group of endocrine therapy alone.

^cIncludes angina, arrhythmia, hypertension, heart attack, peripheral vascular disease, and other cardiovascular disease.

^dIncludes statins, medication for mood or nerves, blood pressure medication, pain medication, over-the-counter medications, multivitamins, and other prescription medications.

^eLASSO does not select IPAQ daily sedentary time and SF12 Mental Component score for transient declines; probabilities of transient decline for scenarios g and h are not considered.

^fIPAQ = International Physical Activity Questionnaire.

^gThe SF12 questionnaire is a health-related quality-of-life questionnaire consisting of twelve questions that measure eight health domains to assess physical and mental health. The mental health-related component includes Vitality, Social Functioning, Role Emotional, and Mental Health scales.

Since APE domain scores may be difficult to readily ascertain at a routine clinic visit, in secondary analyses we explored predictive performances excluding baseline APE domain scores and/or excluding baseline APE scores and substituting Mini-Mental State Examination (MMSE) screening scores (truncated at greater than 24 based on study eligibility), WRAT-4 score, and years of education as proxies for cognitive reserve.

For each model, we present receiver-operating characteristics (ROC), including the means and 95% confidence intervals of the area under the curve (AUC), sensitivity and specificity at Youden’s index. We performed all analyses with R version 3.5.1 glmnet, randomforest, gbm, pROC, and caret packages (R Foundation for Statistical Computing) (65,66).

Results

There were 228 older breast cancer survivors included in this analysis; most were well educated and had an average age of 68 (Table 1). One-third (n = 73) of the survivors in this complete case sample exhibited decline in scores on the attention, processing speed, and executive (APE) function domain at 12 months. Among the 73 survivors with 12-month cognitive decline, 59% (n = 43) had improved cognitive scores, and 41% (n = 30) continued to exhibit persistent cognitive decline at 24 months (Figure 1). The survivors with either 12- or 24-month declines on APE scores were comparable in terms of demographic characteristics and treatment exposures to women without any declines. Detailed summary statistics for select variables of the analytic groups are shown in Table 1, with accompanying P-values for the difference in groups means from ANOVA.

Table 1.

Demographic and clinical pre-systemic therapy/enrollment characteristics of older women with nonmetastatic breast cancer by cognitive outcomes

	Overall sample (n = 228)	No decline (n = 155)	Transient cognitive decline (n = 43)	Persistent cognitive decline (n = 30)
Predictor	Mean (SD) or percent (n)				P-valuea
Sociodemographic factors
Age in years, mean (SD), range	68.10 (5.64), 60.00-91.14	67.91 (5.55), 60.00-84.47	68.20 (5.77), 60.19–84.33	68.99 (6.04), 60.11–91.14	.635
Race, % (n)					.851
White	82.02 (187)	82.58 (128)	79.07 (34)	83.33 (25)
Nonwhite	17.98 (41)	17.42 (27)	20.93 (9)	16.67 (5)
Marital status, % (n)					.669
Married	67.11 (153)	68.39 (106)	67.44 (29)	60.00 (18)
Widowed, divorced, single	32.89 (75)	31.61 (49)	32.56 (14)	40.00 (12)
Mean education, years, mean (SD)	15.82 (2.03)	15.87 (2.02)	15.67 (2.15)	15.73 (1.96)	.831
Clinical factors
Treatment, % (n)					.733
Chemotherapy with our without hormonal therapy	28.07 (64)	29.03 (45)	23.26 (10)	30.00 (9)
Hormonal therapy only	71.93 (164)	70.97 (110)	76.74 (33)	70.00 (21)
AJCC v. 6 stage, % (n)					.453
0	6.14 (14)	6.45 (10)	9.30 (4)	0.00 (0)
I	68.42 (156)	67.10 (104)	67.44 (29)	76.67 (23)
II	21.05 (48)	21.29 (33)	23.26 (10)	16.67 (5)
III	4.39 (10)	5.16 (8)	0.00 (0)	6.67 (2)
Surgery type, % (n)					.253
BCS with/without RT	66.08 (150)	63.23 (98)	76.74 (33)	65.52 (19)
Mastectomy	33.92 (77)	36.77 (57)	23.26 (10)	34.48 (10)
ER status, % (n)					.871
Positive	90.75 (206)	90.32 (140)	92.86 (39)	90.00 (27)
Negative	9.25 (21)	9.68 (15)	7.14 (3)	10.00 (3)
HER2 status, % (n)					.110
Positive	7.80 (17)	10.07 (15)	10.26 (4)	6.67 (2)
Negative	92.20 (201)	89.93 (134)	90.70 (39)	93.33 (28)
Depression (≥16 on CES-D), % (n)	9.87 (22)	11.84 (18)	9.30 (4)	0.00 (0)	.154
Mean STAIb score, mean (SD)	28.49 (7.30)	29.03 (7.86)	27.98 (6.67)	26.47 (4.42)	.189
Mean FACT-Fc score, mean (SD)	43.83 (7.78)	43.93 (8.51)	44.48 (4.98)	42.42 (7.18)	.522
BMI, mean (SD)e	28.63 (6.40)	27.90 (6.58)	29.91 (6.28)	30.61 (4.93)	.036
Number of prescription drugs regularly taken, mean (SD)	3.59 (2.06)	3.38 (2.00)	3.65 (2.17)	4.60 (1.94)	.011
ApoE4 status, % (n)					.002
ApoE4 Positive	21.10 (46)	14.29 (21)	37.21 (16)	32.14 (9)
ApoE4 Negative	78.90 (172)	85.71 (126)	62.79 (27)	67.86 (19)
Cardiovascular, % (n) comorbiditiesd (including peripheral vascular disease)					.010
Yes	53.07 (121)	46.45 (72)	62.79 (27)	73.33 (22)
No	46.93 (107)	53.55 (83)	37.21 (16)	26.67 (8)

^a P-values reflect overall comparisons using analysis of variance (ANOVA) among groups. BCS = breast-conserving surgery; ER = estrogen receptor; CES-D = Center for Epidemiologic Studies Depression Scale; STAI = State Trait Anxiety Inventory, state version; FACT-F = Functional Assessment of Cancer Therapy—Fatigue Subscale; SD = standard deviation; CRCD = cancer-related cognitive decline.

^bState-Trait Anxiety Inventory scores range from 20 to 80; higher scores reflecting more anxiety.

^cFACT-Fatigue Subscale scores range from 0 to 52; higher scores reflect less fatigue.

^dIncludes angina, arrhythmia, hypertension, heart attack, peripheral vascular disease, and other cardiovascular disease.

^eBMI = weight (kg)/height (m2) at baseline.

Machine learning classification of cognitive decline

Prediction of survivors with transient decline in APE scores vs those without decline was acceptable, with an AUC of 0.75 (LASSO sensitivity 83.6%; specificity 58.1%) (Figure 2, left panel). Excluding baseline APE score slightly lowered the predictive accuracy (AUC of 0.71 with a loss of sensitivity [LASSO sensitivity 57.5%, specificity 73.6%]) (Figure 2, right panel).

Figure 2.

Receiver operating characteristic (ROC) curves from Least Absolute Shrinkage and Selection Operator (LASSO) models classifying 12-month transient cognitive decline using baseline to 12-month change scores on neuropsychological tests of attention, processing speed, and executive function (APE). A) Final LASSO machine learning model using 4-fold cross-validation including baseline APE scores as a selected predictor among 94 possible predictors. B) Final LASSO machine learning model excluding baseline APE scores as a predictor. The area under the ROC curve (AUC) and its 95% confidence interval (CI), sensitivity (%, 95% CI), and specificity (%, 95% CI) are presented at the Youden Index. Polygon 95 confidence regions for the ROC curve are displayed.

There was also slightly better prediction of survivors with persistent decline in APE domain scores at 24 months vs survivors with transient decline (AUC = 0.79, LASSO sensitivity 83.6%; specificity 58.1%) (Figure 3, left panel). In the model excluding baseline APE, the AUC was maintained (AUC = 0.83, LASSO sensitivity 86.7%; specificity 69.75%) (Figure 3, right panel).

Figure 3.

Receiver operating characteristic (ROC) curves from Least Absolue Shrinkage and Selection Operator (LASSO) models classifying 12-month transient cognitive decline using baseline to 12-month change scores on neuropsychological tests of attention, processing speed, and executive function (APE). A) Final LASSO machine learning model using 4-fold cross-validation including baseline APE scores as a selected predictor among 94 possible predictors. B) Final LASSO machine learning model excluding baseline APE scores as a predictor. The area under the ROC curve (AUC) and its 95% confidence interval (CI), sensitivity (%, 95% CI), and specificity (%, 95% CI) are presented at the Youden Index. Polygon 95 confidence regions for the ROC curve are displayed.

Several variables were consistently informative in predicting older breast cancer survivors who experienced transient and persistent decline in APE scores, including having an ApoE4 allele, having a HER2 positive tumor, being obese (BMI ≥30 kg/m²), having any cardiovascular comorbidity, taking more prescription medications regularly, and having a higher baseline APE score (Table 2). Self-reported sedentary time and mental health function (SF-12 Mental Component) scores predicted risk of those who would experience sustained, persistent decline but did not help further predict women at risk for transient decline (vs no decline) (Table 3). Other genotypes, tumor characteristics, and treatment separately or in interaction with each other and other variables did not significantly contribute to predicting risk of being in either decline group in this sample (not shown).

In secondary analyses, other machine learning variable selection approaches had lower prediction accuracy but yielded similar top-ranked variables, including ApoE4 positivity, HER2 positivity, obesity, cardiovascular comorbidities, an increasing number of regularly taken prescription medications, and baseline APE score (Supplementary Table 2, available online).

To place these results in context of clinical practice, we examined the probability of cognitive decline based on specific combinations of risk factors included in our LASSO logistic model (Table 4). For example, an older survivor who received chemotherapy treatment for a HER2 negative tumor, was ApoE4 negative, had a BMI less than 30 kg/m², no cardiovascular comorbidities, regularly took two prescription medications, and had relatively high baseline cognitive reserve would have an 18.7% probability of transient decline (95% CI: 9.5% to 33.5%). Taking into account this survivor’s self-reported sedentary hours and high SF-12 Mental Component score, our model predicts this survivor has a 4.5% probability of persistent decline at 24-month follow-up (95% CI: 1.5% to 12.8%). By comparison, if this survivor were ApoE4 positive, had existing cardiovascular comorbidities, and had a BMI greater than or equal to 30 kg/m², her risk of a transient cognitive decline would be 73.8% (95% CI: 49.5% to 89.0%). If her sedentary hours were low and she had a high SF-12 Mental Component, her probability of persistent decline would be 25.7% (95% CI: 8.6% to 56.0%).

Discussion

This study used machine learning methods to test the proof of concept that pre-systemic therapy patient-reported and clinically available data can be used in a tool to predict the risk that an older woman newly diagnosed with breast cancer would develop cognitive declines over a 24-month time horizon. The preliminary risk prediction tool was able to achieve good sensitivity in detecting survivors who would decline, but the conservative cut-point used to define decline resulted in low specificity. Several factors related to neurodegenerative and cardiovascular disease, including apolipoprotein E4 (ApoE4) allele positivity, obesity, cardiovascular comorbidities, and greater use of prescription medication increased the risk of experiencing cognitive declines. Baseline pre-systemic therapy cognitive performance also predicted subsequent cognitive declines, while systematic treatment exposure and tumor characteristics were not as strongly predictive of declines. Retention of these less predictive variables such as systemic therapy and HER2 status in our model also help illustrate how physicians might use future clinical decision tools and communication aids based on these methods to determine a range of predicted risk of persistent cognitive problems based on a patients’ specific combinations of personal and tumor-related variables. Combinations of exemplar survivor profiles illustrate the potential application of the tool in practice.

There are no current tools to predict risk of cancer-related cognitive decline using comprehensive risk factors. Previous clinical oncology decision tools have focused mainly on early detection (67), having cancer predisposing gene mutations (68), estimating adjuvant treatment benefit (69), and guiding prostate cancer therapy (70). There are fewer tools focused on the probability of experiencing adverse treatment events and those have mainly concentrated on hospitalizations for chemotherapy toxicity among older patients (71, 72). In noncancer settings, machine learning methods have been employed to predict mild to severe dementia in general populations using data from electronic health records, medical claims, results of neuroimaging or clinical cognitive assessments, and genomics or other “omics” data (73, 74). The lack of tools for estimating risk of cancer-related adverse cognitive outcomes likely reflects the limited number of prospective studies that include cognitive evaluations, the subtle nature of CRCD, and the variability in domains affected and severity of impairment. Although preliminary, the machine-learning tool we developed in this study provides an important proof of concept that should be useful to guide future efforts to refine, validate, and test clinical utility practice settings.

In our application, we used a liberal cut-point of having a 0.25 SD decrease in performance on six tests of APE to capture the subtle nature of CRCD and types of problems reported by women. Although this approach resulted in acceptable predictive ability and sensitivity to detecting this level of cognitive decline, this cut-point had poor specificity and would lead to false-positive results. Given how difficult cognitive symptoms are for breast cancer survivors, having good sensitivity may be more useful in developing strategies to prevent or ameliorate this adverse effect in survivorship care. This is especially important since we found that a woman with cognitive decline at 12 months is less likely to have persistent decline if she is less sedentary and had maintained better mental health, both factors that can be targeted with clinical interventions. Future iterations of our tool will focus on identifying variables in heterogeneous populations of survivors that can increase specificity without sacrificing sensitivity.

Development of future tools will also have to consider other issues in setting a threshold and gold standard to define CRCD. Although performance on neuropsychological tests is considered a standard, this mode of assessment is known to have practice effects that result in improved performance over time (50). Hence, some consider a failure to improve to be an indication of cognitive loss and/or limitations in cognitive reserve that may signal a trajectory of declining cognitive function (75). Alternative cut-points to detect risk of loss in cognitive performance should be tested as this approach is refined and tested in other survivor populations. Additionally, although we selected the most common domains reported to be affected in CRCD, testing of broader domains using new, more sensitive assessment paradigms and patient-reported cognition would also be useful (7).

The rates of cognitive decline in the sample we used to develop this preliminary risk prediction tool were similar to or lower than other studies (76,77), limiting our sample size and discriminatory power. Additionally, we used data from a well-educated sample from a study that excluded survivors with a history of neurological disease or MMSE scores suggesting dementia. It will be important to assess the predictive ability of our tool in more representative groups of survivors, including younger survivors who may have higher rates of chemotherapy use than seen in our older sample. It is also possible that different sets of factors predict CRCD in younger vs older survivors. Such larger survivor population analyses may impact the directionality we observed in baseline APE domain score predictor, as previous studies have shown greater cognitive reserve to help guard against cognitive decline (10).

The factors that predicted cognitive decline in older breast cancer survivors are similar to those associated with risk of dementia, including cardiovascular disease and genetic polymorphisms (78). Notably, having an ApoE4 allele was the strongest predictor of transient and persistent cognitive declines in our sample. This genotype is known to be related to inflammation and reduced brain plasticity and decreases in the blood-brain barrier (10,18,79). Unfortunately, this complete case sample did not have the power to detect the ApoE4 by treatment interactions we previously reported in the full target sample (9). Although COMT and BDNF polymorphisms have been associated with cognitive dysfunction (22,23), these were not informative parameters in our models. It is possible that these latter genotypes have small effects that we could not detect in this sample. Overall, the similarity between CRCD and dementia risk factors suggests that cancer or its therapies might be unmasking or accelerating underlying neurodegenerative disease in some survivors. Alternatively, the mechanisms underlying CRCD and Alzheimer disease-related dementias (ADRD) may share common pathways (79), underscoring the complexity in defining CRCD risk. We are currently studying ADRD pathology-related biomarkers in this cohort, and those data could be used to refine our tool.

Baseline pre-systemic therapy cognitive performance predicted subsequent cognitive declines, in that higher baselines scores were predictive of both transient and persistent decline. Since the cognitive measure involved formal neuropsychological testing, we also tested accuracy using other approaches that may be more feasible in busy practice settings. We were able to achieve comparable predictive performance when we omitted the baseline score and/or used other measures, including markers of cognitive reserve, self-reported cognitive problems, or the MMSE, although the latter was developed to detect dementia. In some cases, this substitution resulted in a loss of sensitivity. Similarly, to assess if the observed declines could be explained by “regression to the mean,” we omitted baseline APE score from our model and achieved comparable predictive performance. Additionally, while the “regression to the mean” phenomena may be present on shorter timescales, we do not expect this to be a factor in the persistent decline group at 24 months. While other factors did not improve the model fit (eg, age, smoking, sleep problems, self-reported cognitive problems), we also likely had limited variability and power to detect their effects. These variables will be important to examine as we refine and test the tool in other settings and populations.

This machine learning approach has many important strengths including rigorous statistical methods and using comprehensive data from a large, well-defined prospective cohort. However, there are several caveats that should be considered when interpreting our findings. Our cohort had limited demographic or clinical variability and was recruited from largely urban academic centers and their community partners (9). These factors likely limited our ability to detect relationships that might improve accuracy in broader, more diverse older populations (39,78). Such variables will be interesting to evaluate in future research based on feasibility and availability in clinical practice. We evaluated persistent decline only through 24 months. As TLC continues, it will be useful to extend our results to later follow-up points and to determine rates of later onset of decline or recovery after transient decline. We also did not examine whether women who did not decline at 12 months declined at a later time point, Finally, future tools will need to consider ease of implementation in practice, since our final models had 7 to 10 predictors, even after controlling for overfitting using cross-validation approaches. Overall, our findings should be confirmed in the future work in larger samples via data harmonization and pooling and/or use of longitudinal, larger national survey, or medical record data that includes well-defined cancer information.

This study demonstrates the proof of principle that machine learning can identify a set of variables that have reasonable accuracy in predicting risk of cognitive decline. Our results represent an initial step for the development of a tool for testing in clinical settings (69). Overall, accurate tools to predict adverse cognitive effects in cancer patients would be useful to inform treatment decisions when certain regimens may have equivocal benefits, support plans to prevent or ameliorate cognitive problems, and/or include cognitive monitoring during survivorship care visits (51). Overall, development of risk prediction tools for use in busy clinical settings could provide a practical method to synthesize a large and ever-growing body of diverse data readily available to clinicians, supporting conversations to facilitate personalized cancer care decisions that consider the risk and potential mitigation of CRCD.

Data availability

The Thinking and Living with Cancer data are available for sharing following the national institute of health (NIH) requirements and FAIR principles (Findability, Accessibility, Interoperability, Reproducibility) for data access. Data access is via requests to the first author. The SAS, R, or Mplus code and data for the analyses included in the paper are available on request within the constraints of the TLC institutional review board (IRB) requirements.

Author contributions

Arthur Patrick McDeed, MS (Conceptualization; Formal analysis; Investigation; Methodology; Writing—original draft; Writing—review & editing), Brenna C. McDonald, PsyD (Funding acquisition; Writing—review & editing), Sunita K. Patel, PhD (Conceptualization; Funding acquisition; Writing—review & editing), Heather Jim, PhD (Conceptualization; Funding acquisition; Writing—review & editing), James C. Root, PhD (Funding acquisition; Investigation; Writing—review & editing), Brent J. Small, PhD (Conceptualization; Data curation; Investigation; Methodology; Supervision; Writing—review & editing), Andrew J. Saykin, PsyD (Conceptualization; Investigation; Writing—review & editing), Kelley E. Rentscher, PhD (Conceptualization; Funding acquisition; Investigation; Writing—review & editing), Zev M. Nakamura, MD (Conceptualization; Funding acquisition; Investigation; Writing—review & editing), Harvey Jay Cohen, MD (Conceptualization; Funding acquisition; Investigation; Supervision; Writing—review & editing), Judith E. Carroll, PhD (Conceptualization; Funding acquisition; Investigation; Writing—review & editing), Traci N. Bethea, PhD (Conceptualization; Funding acquisition; Investigation; Methodology; Writing—review & editing), Tim A. Ahles, PhD (Conceptualization; Funding acquisition; Investigation; Supervision; Writing—review & editing), Wangting Zhai, MS (Conceptualization; Data curation; Resources; Writing—review & editing), Xingtao Zhou, MS (Data curation; Methodology; Writing—review & editing), Kathleen Van Dyk, PhD (Conceptualization; Funding acquisition; Investigation; Supervision; Writing—review & editing), Jeanne S. Mandelblatt, MD (Conceptualization; Data curation; Funding acquisition; Investigation; Methodology; Project administration; Resources; Supervision; Writing—review & editing), Jaeil Ahn, PhD (Conceptualization; Formal analysis; Methodology; Supervision; Validation; Writing—original draft; Writing—review & editing).

Funding

This research was supported by the National Cancer Institute at the National Institutes of Health grant R01CA129769 to JM, TAA, and AJS and National Institute on Aging at the National Institutes of Health grants R01AG068193 to JM and AJS and R56AG068086 to JEC and JM. This research was also supported in part by National Cancer Institute grant R35CA197289 and National Institute on Aging grant R21AG075008 to JM. This study was also supported in part by the National Cancer Institute under Award K08CA241337 to KVD and the National Cancer Institute at the National Institutes of Health grant P30CA51008 to Georgetown University Lombardi Comprehensive Cancer Center for support of the Biostatistics and Bioinformatics Resource and the Non-Therapeutic Shared Resource. The work of AJS and BCM was supported in part by the National Institute on Aging at the National Institutes of Health grants P30AG010133, P30AG072976, and R01AG19771 and National Cancer Institute at the National Institutes of Health grant R01CA244673. TAA was supported in part by National Cancer Institute at the National Institutes of Health grants R01CA172119 and P30CA008748. JCR was supported by the National Cancer Institute at the National Institutes of Health grant R01CA172119. The work of JC was supported in part by the National Cancer Institute at the National Institutes of Health grant R01CA237535. HJC was supported in part by the National Institute on Aging at the National Institutes of Health grant P30AG028716 for the Duke Pepper Center. KER is partially supported by the National Institute on Aging at the National Institutes of Health grant K01AG065485 and the Cousins Center for Psychoneuroimmunology. TNB was supported in part by the National Cancer Institute grant K01CA212056. SKP was supported in part by the National Cancer Institute grant R01CA261793.

Conflicts of interest

Dr Saykin has received support from Avid Radiopharmaceuticals, a subsidiary of Eli Lilly (in kind contribution of positron emission tomography tracer precursor); Bayer Oncology (Scientific Advisory Board); Eisai (Scientific Advisory Board); Siemens Medical Solutions USA, Inc (Dementia Advisory Board); and Springer-Nature Publishing (Editorial Office Support as Editor-in-Chief, Brain Imaging and Behavior).

All other authors have declared no conflicts of interest.