Abstract
Cancer-related cognitive impairment (CRCI) is a concerning problem for many cancer survivors. Evaluating patients for CRCI has been a challenge, in part because of a lack of standardized practices. Self-report instruments are often used to assess CRCI, but there are no validated cutpoints. We present the results of receiver operating characteristic curve analysis identifying cutpoints of the Functional Assessment of Cancer Therapy—Cognition perceived cognitive impairment (PCI) in female breast cancer survivors for identifying CRCI cases. We defined presence of CRCI based on elevated complaints on the Patient’s Assessment of Own Functioning Inventory compared with healthy control scores. Our results indicate that scores less than 54 in PCI scores using 18 items and scores less than 60 in PCI scores using 20 items exhibited good ability to discriminate CRCI cases from noncases (area under the receiver operating characteristic curve was 0.84 [95% CI = 0.73 to 0.94]). These preliminary results represent an important contribution toward standardizing practices across CRCI studies.
Cognitive problems after cancer and its treatment, often referred to as cancer-related cognitive impairment (CRCI), are a concerning adverse effect for many survivors. Evaluating CRCI in research and clinical settings is challenging without standardized methods for classification. Self-report measures of cognitive dysfunction (1) are often preferred for assessing CRCI, but there are no established cutpoints for some of the most commonly used instruments. Common measures and cutpoints for classifications are important for standardizing definitions of CRCI. Further, it is difficult to compare study samples and evaluate potential interventions without definitions of “caseness.”
The Functional Assessment of Cancer Therapy—Cognition (FACT-Cog) (2,3) was developed specifically to assess cognitive difficulties in cancer survivors, and it is regularly employed in observational and treatment studies (4,5). For version 3 of the FACT-Cog, the scale developers recommend using one of the four subscores, the perceived cognitive impairment (PCI) score, as the preferred outcome (6), and it is the one most often cited in the literature. Surprisingly, no cutpoints have yet been established for the PCI score. To address this gap, we examined the receiver operating characteristic (ROC) curves for the PCI score against a validated CRCI classification measure, and then examined the performance of emergent cutpoints.
The Mind Body Study was a prospective longitudinal study examining the effects of breast cancer treatments on cognitive functioning from the end of primary treatment through 6 years. The study has been previously described in detail, and it involved intensive cognitive and psychosocial assessments (7–9). The FACT-Cog version 3 was added at the final timepoint of this study (approximately 3–6 years after participants completed primary cancer treatment), and we examined data only from this time point in a cross-sectional manner for this report.
Participants were recruited through clinical oncology practices and rapid case ascertainment of stage-eligible patients from collaborating physicians and hospitals identified through the Los Angeles Surveillance, Epidemiology, and End Results Program registry. Participants were aged 21–65 years, had a recent early-stage breast cancer diagnosis, and completed primary treatment within the last 3 months (8). Women with active psychotic or major depressive disorders, or any history of treatments or conditions with known effects on cognition or inflammation, were excluded. The UCLA institutional review board approved the study, and all participants provided written informed consent.
All participants completed the FACT-Cog version 3 at the last longitudinal follow-up assessment; this version was after study initiation and added to this time point because it was a standard in studies of this population (2,3). Standard scoring yields four domain scores, and for this report we focus only on the PCI, which includes 20 items total. Historically, only 18 were scored to get the PCI (PCI-18; range 0-72), but there is now more evidence to support the validity of including all 20 items in the score (PCI-20; range 0-80) (10); this report examines both PCI-18 and PCI-20 scores.
Participants also completed the Patient’s Assessment of Own Functioning Inventory (PAOFI) (11), commonly used to capture self-assessed difficulties in various populations across several domains of cognitive function. The PAOFI total is calculated based on the number of items rated as high severity, ranging from 0–33, in which higher is worse functioning. Comparison PAOFI data were available from healthy control women without breast cancer enrolled in a study at the University of California, San Diego (UCSD) (12) (n = 63; mean age 51.96 [± 9.35 years]; 63% ≤ college education; 79% white), and used in earlier Mind Body Study reports to classify cognitive impairment in the breast cancer patients (8). Cognitive impairment on the PAOFI total was established as greater than 2SDS above the mean of the healthy control sample (ie, > 6 on the PAOFI total in healthy women) (8).
Sixty percent of the breast cancer survivor sample was randomly selected for the training dataset, and 40% for the validation dataset using a random number generator in SPSS. We plotted two ROC curves using the training dataset for each PCI score and manually identified the cutpoint yielding highest sensitivity and specificity estimates. We then calculated estimates of sensitivity specificity, positive predictive validity (PPV), and negative predictive validity (NPV) in the validation dataset. Analyses were conducted using SPSS (IBM SPSS Statistics for Windows, V.24.0. Armonk, NY: IBM Corp).
A total of 133 breast cancer survivors were evaluated; see Table 1 for sample demographics. There were 80 survivors in the training dataset and 53 in the validation data set. The training dataset consisted of 17 impaired and 63 not impaired participants; the validation dataset consisted of 14 and 39, respectively.
Table 1.
Demographics of breast cancer survivor sample
| Whole sample | Training set* | Validation set* | |
|---|---|---|---|
| (n = 133) | (n = 80) | (n = 53) | |
| Age, mean (SD), y | 56.61 (7.82) | 56.35 (8.7) | 57 (6.32) |
| Education (≤ college), No. (%) | 68 (51) | 62 (78) | 45 (85) |
| Race (% white), No. (%) | 106 (80) | 62 (78) | 44 (83) |
| Years since Dx, mean (SD) | 4.31 (0.64) | 4.23 (0.63) | 4.26 (0.66) |
| Chemotherapy Tx No. (%) | 73 (55) | 49 (61) | 24 (45) |
| Radiation Tx No. (%) | 100 (75) | 59 (74) | 41 (77) |
| Stage at diagnosis No. (%) | |||
| 0 | 19 (14) | 11 (14) | 8 (15) |
| I | 58 (44) | 30 (38) | 28 (53) |
| II | 44 (33) | 30 (38) | 14 (26) |
| III | 12 (9) | 9 (11) | 3 (6) |
| Estimated verbal IQ (WTAR), mean (SD) | 114.74 (8.78) | 115.60 (8.67) | 113.43 (8.85) |
No differences between groups on any variable based on t test or X2, p greater than .05. Dx = diagnosis; Tx = treatment; WTAR = Wechsler Test of Adult Reading.
The area under the ROC curve was 0.84 (95% CI = 0.73 to 0.94) for both PCI-18 and PCI-20. Examining the coordinates of the ROC curves for each measure, the following cutpoint scores were identified in the training set: PCI-18 less than 54, 76% sensitivity, and 82% specificity; PCI-20 less than 60, 76% sensitivity, 84% specificity. These cutpoints exhibited good classification performance in the validation dataset; see Table 2.
Table 2.
Classification performance of FACT-Cog PCI in the validation dataset*
| Classification |
Sensitivity | Specificity | PPV | NPV | |||
|---|---|---|---|---|---|---|---|
| Impaired | Not impaired | (95% CI)† | (95% CI) | (95% CI) | (95% CI) | ||
| PCI - 18 (< 54) | Impaired | 11 | 3 | 79 | 92 | 79 | 92 |
| Not impaired | 3 | 36 | (49 to 95) | (79 to 98) | (49 to 95) | (79 to 98) | |
| PCI - 20 (< 60) | Impaired | 11 | 3 | 79 | 92 | 79 | 92 |
| Not impaired | 3 | 36 | (49 to 95) | (79 to 98) | (49 to 95) | (79 to 98) | |
*FCIs are exact binomial confidence intervals. FACT-Cog = Functional Assessment of Cancer Therapy—Cognition PCI = perceived cognitive impairment; PPV = positive predictive value; NPV = negative predictive value.
†CIs are exact binomial confidence intervals.
This report is the first to describe cutpoints on the FACT-Cog PCI to classify CRCI. In addition to the commonly used 18-item PCI, we also examined the full 20-item PCI, which is currently being validated and used in the research community (10). Both PCI-18 and PCI-20 showed good discriminative ability to classify CRCI.
The field of CRCI is presently lacking in standardized classification methods using targeted subjective instruments, and these findings offer a valuable starting point. Cutpoints can help standardize methods and CRCI definitions across research studies to improve congruity and accelerate progress. Cutpoints may also be useful for identifying patients in need of services, but this would require further study.
This study has limitations in that it is a relatively small and exploratory extension of a larger study of breast cancer survivors—not specifically designed to test whether these cutpoints have relevance for a variety of important outcomes (eg, functioning). In addition, participants in our sample were several years out from completing cancer treatments, which could call into question the validity of these cutpoints in patients and survivors more proximal to undergoing treatment. Future research will be important to validate the relevance of these cutpoints in other varied cancer populations.
Funding
This research is supported by the National Cancer Institute at the National Institutes of Health (grants R01 CA109650, P30 CA16042, and K08CA241337), the National Institutes of Health Clinical Loan Repayment Program (National Cancer Institute) and the American Cancer Society (to Kathleen Van Dyk), and the Breast Cancer Research Foundation (to Patricia A. Ganz).
Notes
Affiliations of authors: UCLA Semel Institute for Neuroscience and Human Behavior, Los Angeles, CA (KVD); UCLA Jonsson Comprehensive Cancer Center, Los Angeles, CA (KVD, CMC, LP); Department of Biostatistics, UCLA Fielding School of Public Health, Los Angeles, CA (CMP); UCLA David Geffen School of Medicine, Los Angeles, CA (PAG); UCLA Fielding School of Public Health, Los Angeles, CA (PAG).
Patricia A. Ganz is a scientific advisory board member for the Breast Cancer Research Foundation. The funders had no role in the design of the study; the collection, analysis, and interpretation of the data; the writing of the manuscript; and the decision to submit the manuscript for publication.
References
- 1. Van Dyk K, Ganz PA.. Doctor, now that my chemotherapy treatment is over, when will my “chemofog” lift? J Clin Oncol. 2017;35(5):482–484. [DOI] [PubMed] [Google Scholar]
- 2. Wagner LI, Sweet J, Butt Z, Lai J, Cella D.. Measuring patient self-reported cognitive function: development of the functional assessment of cancer therapy-cognitive function instrument. J Support Oncol. 2009;7(6):W32–W39. [Google Scholar]
- 3. Wagner LI, Lai JS, Cella D, Sweet J, Forrestal S.. Chemotherapy-related cognitive deficits: development of the FACT-Cog instrument. Ann Behav Med. 2004;27:S10. [Google Scholar]
- 4. Bray VJ, Dhillon HM, Bell ML, et al. Evaluation of a web-based cognitive rehabilitation program in cancer survivors reporting cognitive symptoms after chemotherapy. J Clin Oncol. 2017;35(2):217–225. [DOI] [PubMed] [Google Scholar]
- 5. Janelsins MC, Heckler CE, Peppone LJ, et al. Cognitive complaints in survivors of breast cancer after chemotherapy compared with age-matched controls: an analysis from a nationwide, multicenter, prospective longitudinal study. J Clin Oncol. 2017;35(5):506–514. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. FACIT Questionnaires: FACT-Cog Scoring and Interpretation Materials. https://www.facit.org/facitorg/questionnaires.
- 7. Ganz PA, Petersen L, Castellon SA, et al. Cognitive function after the initiation of adjuvant endocrine therapy in early-stage breast cancer: an observational cohort study. J Clin Oncol. 2014;32(31):3559–3567. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 8. Ganz PA, Kwan L, Castellon SA, et al. Cognitive complaints after breast cancer treatments: examining the relationship with neuropsychological test performance. J Natl Cancer Inst. 2013;105(11):djt073. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Van Dyk K, Crespi CM, Bower JE, Castellon SA, Petersen L, Ganz PA.. The cognitive effects of endocrine therapy in survivors of breast cancer: a prospective longitudinal study up to 6 years after treatment. Cancer. 2019;125(5):681–689. [DOI] [PMC free article] [PubMed] [Google Scholar]
- 10. Costa DSJ, Loh V, Birney DP, et al. The structure of the FACT-Cog v3 in cancer patients, students, and older adults. J Pain Symptom Manage. 2018;55(4):1173–1178. [DOI] [PubMed] [Google Scholar]
- 11. Chelune GJ, Heaton RK, Lehman RAW.. Neuropsychological and personality correlates of patients’ complaints of disability In: Goldstein G, Tarter RE, eds. Advances in Clinical Neuropsychology. Berlin: Springer; 1986:95–126. [Google Scholar]
- 12. Liu L, Mills PJ, Rissling M, et al. Fatigue and sleep quality are associated with changes in inflammatory markers in breast cancer patients undergoing chemotherapy. Brain Behav Immun. 2012;26(5):706–713. [DOI] [PMC free article] [PubMed] [Google Scholar]