Abstract
Objectives
Cognitive symptoms are commonly reported among cancer patients and survivors, yet guidance on when self‐reported cognitive symptoms warrant follow‐up is lacking. We sought to establish cut‐off scores for identifying patients with perceived low cognitive functioning on widely used self‐report measures of cognition and a novel single item Cognitive Change Score.
Methods
Adult patients diagnosed with invasive cancer who had completed at least one cycle of chemotherapy completed a questionnaire containing the EORTC‐Cognitive Function (CF) subscale, Functional Assessment of Cancer Therapy‐Cognitive Function (FACT‐COG) Perceived Cognitive Impairment (PCI) and our Cognitive Change Score (CCS). We used receiver operating characteristic analyses to establish the discriminative ability of these measures against the Patient's Assessment of Own Functioning Inventory (PAOFI) as our reference standard. We chose cut‐off scores on each measure that maximised both sensitivity and specificity for identifying patients with self‐reported low CF.
Results
We recruited 294 participants (55.8% women, mean age 56.6 years) with mixed cancer diagnoses (25.5 months since diagnosis). On the CCS, 77.6% reported some cognitive change since starting chemotherapy. On the PAOFI 36% had low CF. The following cut‐off scores identified cases of low CF: ≥28.5 on the CCS (75.5% sensitivity, 67.6% specificity); ≤75.0 on the European Organisation for Research and Treatment of Cancer, QLQ‐C30 Cognitive Functioning scale (90.9% sensitivity, 57.1% specificity); ≤55.1 on the FACT‐COG PCI‐18 (84.8% sensitivity, 76.2% specificity), and ≤59.5 on the FACT‐COG PCI‐20 (78.8% sensitivity, 84.1% specificity).
Conclusions
We found a single item question asking about cognitive change has acceptable discrimination between patients with self‐reported normal and low CF when compared to other more comprehensive self‐report measures of cognitive symptoms. Further validation work is required.
Keywords: cancer, cancer‐related cognitive impairment, cognition, concentration, memory, neuropsychology, psycho‐oncology, screening
1. BACKGROUND
Cognitive symptoms, such as poor concentration, difficulties with memory, and slowed or foggy thinking, are frequently described by cancer patients. Research has shown that up to 75% of patients report cognitive symptoms during chemotherapy treatment. 1 , 2 For some, cognitive symptoms can persist beyond treatment, with 15%–35% demonstrating cognitive impairment when assessed using standardised neuropsychological tests. 2 , 3 Almost all studies have shown a disassociation between self‐reported cognitive symptoms and performance on neuropsychological tests, although recent studies have shown some association between self‐reported cognitive symptoms and changes in functional neuroimaging. 4 , 5 , 6
Both self‐reported symptoms and impairment identified on neuropsychological assessment are important to patients, and both are valid endpoints in clinical studies. 7 Critically, self‐reported cognitive symptoms offer a gateway to further services, be it further neuropsychological assessment or intervention and support. However, variation in prevalence of self‐reported cognitive symptoms is evident in the literature. In longitudinal studies, 45% of breast cancer survivors reported decline in Cognitive Function (CF) in the 6 months after chemotherapy, 8 and 30% of colorectal cancer survivors reported cognitive impairment at 2 years after treatment. 9 In contrast, in a cross‐sectional study of survivors of breast cancer (N = 1889) more than 5 years after treatment, prevalence of cognitive symptoms after cancer treatment was not different to normative data (i.e. 7%). 10 Differences in study design, sample and measures employed likely contribute to the observed variation.
Some of the most widely used self‐report measures for cognitive symptoms in people with cancer include the Functional Assessment of Cancer Therapy ‐ Cognitive Function (FACT‐COG), Perceived Cognitive Impairment (PCI) subscale of the FACT‐COG, Patient's Assessment of Own Functioning Inventory (PAOFI), European Organisation for Research and Treatment of Cancer, QLQ‐C30 Cognitive Functioning scale (EORTC‐CF) and Patient‐Reported Outcomes Measurement Information System (PROMIS®) CF item bank and short forms. 11 , 12 Despite availability of numerous instruments to assess self‐reported cognitive symptoms, research describing cut‐off scores on these measures that indicate meaningful impairment and need for further follow‐up of neuropsychological function among cancer patients and survivors is scant. Van Dyk et al 13 recently identified clinical cut‐points on the FACT‐COG PCI subscale, in 133 breast cancer survivors approximately 4 years since diagnosis. Using the PAOFI as reference standard, in receiver operating characteristic (ROC) curve analyses, they found a cut‐off score of below 54 had 76% sensitivity, and 82% specificity for identifying cases of impaired CF on the 18‐item PCI (recommended scoring), 14 while a cut‐off score of below 60 had 76% sensitivity, and 84% specificity for identifying cases of impaired CF on the PCI‐20.
Rothrock et al 15 developed provisional clinical interpretations for PROMIS CF Bank of items T‐scores using the bookmarking method, in which vignettes are rated on severity and test scores assigned meaningful labels with a sample of 10 experienced oncology clinicians and 6 patients. T‐scores less than 35 (1.5 standard deviations below the standardized mean of 50) were considered indicative of severe cognitive impairment by clinicians and T‐scores less than 30 (2 standard deviations below the mean) by patients. Moderate impairment was identified as a T‐score between 35 and 40 by clinicians and 30 and 35 by patients. In addition, Terwee et al 16 published guidance on minimally important change for PROMIS CF. A T‐score point change of 2–6 over time was reported as a threshold for a minimal within‐person change which patients perceived as an important change. PROMIS Cognitive Function Short Form 8a has recently been recommended as a minimum inclusion in studies of CF in adults due to its extensive development and validation using item response theory. 11
Data supporting these cut‐off scores and T‐score interpretations are preliminary. Further, the self‐reported measures evaluated can be long and time consuming to administer, score and interpret (8 items for PROMIS Cognitive Function Short Form 8a, 18 or 20 items for the FACT‐COG PCI, 35 items for FACT‐COG, and 33 items for PAOFI), limiting ease of implementation in clinical settings. Short, reliable, and valid screening measures of self‐reported CF are needed to aid clinicians in identifying cognitive concerns warranting follow‐up and support. This study sought to establish cut points on a new single item measure of change in CF since chemotherapy, and a widely used short‐form measure of self‐reported cognitive symptoms, the 2‐item EORTC‐CF. 17 , 18 , 19 As a secondary aim we sought to validate previously reported cut‐offs on the FACT‐COG PCI. 13
2. METHOD
2.1. Study design
Data for the current analysis are derived from a previously conducted cross sectional study (called ‘On the receiving end’) which evaluated patients' perceived frequency and severity of chemotherapy side effects and included patient reported outcome measures (PROMs) evaluating cognitive symptoms. 20
2.2. Participants
We recruited participants from 2 metropolitan teaching hospitals in Sydney (Concord Repatriation General Hospital and Royal Prince Alfred Hospital) and 1 rural hospital (Dubbo Base Hospital) in New South Wales, from January 2008 to October 2016. Eligible participants were adult patients with a diagnosis of invasive cancer who were receiving chemotherapy and had completed at least 1 cycle. There were no restrictions based on stage of disease or age. Participants were required to have written English fluency. Ethics approval was obtained (HREC/07/RPAH) and all patients provided written consent.
2.3. Procedure
We conducted a face‐to‐face interview on a single occasion which included a side effects survey and multiple PROMs, as reported elsewhere. 20 Demographic, disease and treatment characteristics were obtained from the patients' medical records.
The Cognitive Change Screen (CCS) is a single‐item question asking, ‘How much change have you noticed in your memory and concentration since you started chemotherapy’? It is scored on a 10‐cm visual analogue scale and reported as an integer between 0 (no change) and 100 (much worse). Higher scores indicate greater self‐reported change, or worsening, in memory and concentration. A subgroup of patients completed the CCS on a second occasion two to 6 weeks later to obtain information on test‐retest reliability.
Patients Assessment of Own Functioning Inventory (PAOFI) comprises of 33 items and 4 subscales: memory (10 items), language and communication (9 items), sensory‐perceptual (3 items), use of hands (2 items) higher level cognitive and intellectual function (9 items). 21 Participants rate each item on a Likert six‐point scale from almost always to almost never. According to the PAOFI's scoring instructions, we assigned a score of 1 to each item rated as ‘almost always’, ‘very often’, and ‘fairly often’. Items rated ‘once in a while’, ‘very infrequently’, or ‘almost never’ were assigned a score of 0. 21 Scores are summed for each subscale such that the maximum score is the number of items (e.g. memory subscale ranges 0–10). A higher score indicates poorer functioning. The PAOFI has demonstrated good reliability and validity among cancer survivors. 22 , 23
The FACT‐COG version three questionnaire assesses CF in cancer subjects across four domains: PCI, Perceived Cognitive Abilities (PCA), Noticed by others, and Impact on quality of life. 14 , 19 , 24 Each item is scored on a 5‐point Likert scale. Subscales are summed, total scores range from 0 to 156, and higher scores are associated with better CF. The FACT‐COG has demonstrated good reliability and validity among cancer survivors. 24 , 25
EORTC‐CF is a 2‐item questionnaire each with a possible integer value between 0 and 3. 26 Total score is reported on a scale from 0 to 100; however, due to the discrete nature of item responses and only 2 items, only 7 total score values are possible. Higher scores indicate better CF. 27 The EORTC‐CF is well validated and demonstrates good reliability among cancer patients for self‐reported CF. 18 , 19
2.4. Statistical methods
We used SPSS Version 25.0 (IBM, USA) to conduct all analyses. We described our sample using means and standard deviations for continuous variables and proportions for categorical variables. We compared demographic and clinical characteristics of participants identified as having perceived low or normal CF based on the PAOFI score. We identified cases of self‐reported low CF where three or more items were endorsed as ‘almost always’, ‘very often’, and ‘fairly often’ (i.e. PAOFI total score ≥3). 28 , 29 We used four logistic regression models with the (dichotomized) PAOFI as a reference standard as the outcome, and the CCS, EORTC‐CF, FACT‐COG PCI‐18 and PCI‐20 as the explanatory variables to estimate ROC curves to determine an optimal cut‐off for each measure for identifying cases of cognitive impairment. We used the PAOFI as a reference standard as this measure has previously been validated as diagnostically useful for identifying cases of HIV‐associated neurocognitive disorders. 28 , 29 We also considered an alternative method for identifying cases of self‐reported low CF using total PAOFI scores >2SD above the mean of a healthy group of women (N = 63, mean age 52 years old) as previously reported. 13 , 30 We selected cut‐off scores to maximise test sensitivity and specificity, erring on the side of sensitivity as we aim to use CCS as a preliminary screening tool for further cognitive assessment. Test‐retest reliability of the CCS was evaluated by calculating Cronbach's Alpha (α), values ≥ 0.7 were considered representative of acceptable test‐retest reliability. 31
3. RESULTS
We invited 391 patients to participate, 308 consented, and 294 participants had evaluable self‐report cognitive data for this study. Participants were on average 56.6 years old and 25.5 months since diagnosis. Just over half of our participants were women (55.8%), most were currently in a relationship (70.1%) and employed in either a part‐ or full‐time capacity (63.3%). The most common diagnosis was colorectal cancer (27.6%), followed by breast cancer (22.4%). Just under half the participants were receiving chemotherapy with curative intent (45.6%). Table 1 displays demographic details for participants stratified by low or normal self‐reported CF on the PAOFI. There was a significant impact of sex, smoking status, tumour type, and receipt of surgery on self‐reported CF. Women, non‐smokers, patients with breast cancer, and patients having surgery were more likely to self‐report reduced CF.
TABLE 1.
Demographic and clinical characteristics for 294 subjects stratified by self‐reported cognitive function a
| Total (N = 294 b ) | Normal cognitive function (N = 188) | Low cognitive function (N = 106) | P‐value | |
|---|---|---|---|---|
| Age (years), mean (SD) | 56.6 (12.8) | 57.4 (12.8) | 55.1 (12.7) | 0.167 |
| Times since diagnosis (months), mean (SD) | 25.5 (46.2) | 23.8 (41.7) | 28.4 (53.3) | 0.418 |
| Female sex | 164 (55.8%) | 94 (57.7%) | 70 (42.3%) | 0.008 |
| Educational attainment | ||||
| High school or less | 136 (46.4%) | 85 (62.5%) | 51 (37.5%) | 0.661 |
| College or greater | 157 (53.6%) | 102 (65.0%) | 55 (35.0%) | |
| Employment status | ||||
| Retired/unemployed/home duties | 108 (36.7%) | 71 (65.7%) | 37 (34.3%) | 0.625 |
| Part‐time/full‐time | 186 (63.3%) | 117 (62.9%) | 69 (37.1%) | |
| Married/domestic partner | 206 (70.1%) | 130 (63.1%) | 76 (36.9%) | 0.647 |
| English — native speaker | 250 (85.0%) | 165 (66.0%) | 85 (34.0%) | 0.080 |
| Current or previous smoking history | 150 (51.4%) | 104 (69.3%) | 46 (30.7%) | 0.053 |
| Tumour type | ||||
| Breast | 66 (22.4%) | 31 (47.0%) | 35 (53.0%) | 0.019 |
| Colorectal | 81 (27.6%) | 56 (69.1%) | 25 (30.9%) | |
| Gynaecological | 30 (10.2%) | 17 (56.7%) | 13 (43.3%) | |
| Lung | 47 (16.0%) | 32 (68.1%) | 15 (31.9%) | |
| Genitourinary and prostate | 20 (6.8%) | 15 (75.0%) | 5 (25.0%) | |
| Other c | 50 (17.0%) | 37 (74.0%) | 13 (26.0%) | |
| Stage | ||||
| I | 9 (3.1%) | 6 (66.7%) | 3 (33.3%) | 0.556 |
| II | 25 (8.6%) | 13 (52.0%) | 12 (48.0%) | |
| III | 99 (34.0%) | 62 (62.6%) | 37 (37.4%) | |
| IV | 158 (54.3%) | 105 (66.5%) | 53 (33.5%) | |
| Chemotherapy intent | ||||
| Adjuvant/neoadjuvant | 129 (45.6%) | 79 (61.2%) | 50 (38.8%) | 0.449 |
| Palliative | 154 (54.4%) | 101 (65.6%) | 53 (34.4%) | |
| Surgery | 212 (72.1%) | 126 (59.4%) | 86 (40.6%) | 0.016 |
| Radiotherapy | 73 (24.8%) | 45 (61.6%) | 28 (38.4%) | 0.692 |
| Hormonal therapy | 28 (9.5%) | 17 (60.7%) | 11 (39.3%) | 0.695 |
Abbreviation: PAOFI, Patients Assessment of Own Functioning Inventory.
Cognitive function as measured by Patients Assessment of Own Functioning Inventory (PAOFI), for low functioning defined as responding ‘almost always’, ‘very often,’ or ‘fairly often’ ≥3 items.
Based on available self‐report data on single item cognitive screen.
Other tumour types included upper gastrointestinal, head and neck, sarcoma, lymphoma, melanoma.
Table 2 displays CF across each of the PROMs. The average Cognitive Change Score (CCS) was 32.0 (possible range 0–100), and 22.4% (N = 66) reported a CCS of 0 indicative of no perceived change in CF since starting chemotherapy, meaning 77.6% (N = 228) of our sample reported some degree of change or worsening of cognitive functioning since starting chemotherapy. On the PAOFI, 36% (N = 108) had self‐reported low CF.
TABLE 2.
Cognitive function stratified by self‐reported cognitive function, mean (SD) a
| Total (N = 294) | Normal cognitive function (N = 188) | Low cognitive function (N = 106) | |
|---|---|---|---|
| CCS | 32.0 (29.1) | 22.0 (23.9) | 49.7 (29.4) |
| PAOFI | |||
| Total | 2.6 (3.9) | 0.4 (0.6) | 6.4 (4.1) |
| Memory | 1.1 (1.8) | 0.1 (0.4) | 2.8 (2.0) |
| Language and communication | 0.6 (1.2) | 0.1 (0.3) | 1.5 (1.5) |
| Use of hands | 0.2 (0.5) | 0.1 (0.3) | 0.4 (0.7) |
| Sensory‐perceptual | 0.2 (0.5) | 0.04 (0.2) | 0.5 (0.8) |
| Higher level cognitive | 0.5 (1.2) | 0.03 (0.2) | 1.2 (1.7) |
| EORTC‐CF b | 66.7 (27.3) | 75.9 (24.1) | 49.0 (24.3) |
| FACT‐COG b | |||
| Perceived cognitive impairment | 55.1 (14.8) | 61.9 (9.7) | 42.1 (14.2) |
| Perceived cognitive abilities | 19.3 (6.1) | 22.0 (5.0) | 14.2 (4.7) |
| Impact on quality of life | 10.3 (4.6) | 11.1 (4.6) | 8.8 (4.2) |
| Noticed by others | 14.7 (2.5) | 15.6 (1.2) | 13.1 (3.4) |
Note: Cognitive Change Score (CCS), higher scores indicate worse cognitive functioning; Patients Assessment of Own Functioning Inventory (POAFI), higher scores indicate worse cognitive function; European Organisation for Research and Treatment of Cancer, QLQ‐C30 Cognitive Functioning Scale (EORTC‐CF), higher scores indicate better cognitive function; Functional Assessment of Cancer Therapy ‐ Cognitive Function (FACT‐COG), higher scores indicate better cognitive function; Functional Assessment of Cancer Therapy‐General (FACT‐G), higher scores indicated better quality of life.
Abbreviations: CCS, Cognitive Change Score; EORTC‐CF, European Organisation for Research and Treatment of Cancer; FACT‐CF, Functional Assessment of Cancer Therapy ‐ Cognitive Function; QLQ‐C30 Cognitive Functioning scale; PAOFI, Patients Assessment of Own Functioning Inventory.
Cognitive function as measured by Patients Assessment of Own Functioning Inventory (PAOFI), for low functioning defined as responding ‘almost always’, ‘very often,’ or ‘fairly often’ ≥3 items.
N = 96 completed the EORTC‐CF and FACT‐COG. N = 63 had normal cognitive function and N = 33 had low cognitive function according to PAOFI.
Using the PAOFI as a reference standard we identified a cut‐off score of ≥28.5 on the CCS would identify cases of self‐reported low CF with 75.5% sensitivity and 67.6% specificity (area under receiver operator characteristics curve (AUC) = 0.767, 95% CI: 0.711–0.824, Table 3). On the EORTC‐CF, a cut‐off score of ≤75.0 had 90.9% sensitivity and 57.1% specificity for identifying cases of self‐reported low CF (AUC = 0.784, 95% CI: 0.688–0.879). A cut‐off score of ≤55.1 on the FACT‐COG PCI‐18 had 84.8% sensitivity and 76.2% specificity (AUC = 0.880, 95% CI: 0.813–0.946), and ≤59.5 on the FACT‐COG PCI‐20 had 78.8% sensitivity and 84.1% specificity (AUC = 0.882, 95% CI: 0.816–0.948). A cut‐off score of ≤19.5 for the FACT‐COG PCA had 78.8% sensitivity and 71.9% specificity (AUC = 0.864, 95% CI: 0.793–0.934).
TABLE 3.
Area under the ROC curve for each screening measure
| Screening measure | Proposed cut‐off | Sensitivity | Specificity | Cases/sample size | AUC (95% CI) |
|---|---|---|---|---|---|
| Reference standard of PAOFI total score ≥3 28 , 29 | |||||
| CCS | ≥28.5 | 75.5% | 67.6% | 106/294 | 0.767 (0.711, 0.824) |
| EORTC‐CF | ≤75.0 | 90.9% | 57.1% | 33/96 | 0.784 (0.688, 0.879) |
| FACT‐COG | |||||
| PCI‐18 | ≤55.1 | 84.8% | 76.2% | 33/96 | 0.880 (0.813, 0.946) |
| PCI‐20 | ≤59.5 | 78.8% | 84.1% | 33/96 | 0.882 (0.816, 0.948) |
| PCA | ≤19.5 | 78.8% | 71.9% | 33/96 | 0.864 (0.793, 0.934) |
| Reference standard of >2SD above the mean total PAOFI a | |||||
| CCS | ≥40.5 | 74.5% | 71.3% | 47/294 | 0.791 (0.721, 0.860) |
| EORTC‐CF | ≤58.3 | 76.5% | 26.3% | 17/96 | 0.827 (0.735, 0.918) |
| FACT‐COG | |||||
| PCI‐18 | ≤51.3 | 94.1% | 78.9% | 17/96 | 0.917 (0.851, 0.983) |
| PCI‐20 | ≤57.1 | 94.1% | 78.9% | 17/96 | 0.921 (0.855, 0.987) |
| PCA | ≤17.5 | 94.1% | 72.7% | 17/96 | 0.903 (0.840, 0.965) |
Note: Cognitive Change Score (CCS), higher scores indicate worse cognitive functioning; Patients Assessment of Own Functioning Inventory (POAFI), higher scores indicate worse cognitive function; European Organisation for Research and Treatment of Cancer, QLQ‐C30 Cognitive Functioning scale (EORTC‐CF), higher scores indicate better cognitive function; Functional Assessment of Cancer Therapy ‐ Cognitive Function (FACT‐COG), higher scores indicate better cognitive function; Functional Assessment of Cancer Therapy ‐General (FACT‐G), higher scores indicated better quality of life.
Abbreviations: AUC, area under receiver operator characteristics curve; CCS, Cognitive Change Score; EORTC‐CF, European Organisation for Research and Treatment of Cancer; FACT‐COG, Functional Assessment of Cancer Therapy ‐ Cognitive Function; QLQ‐C30 Cognitive Functioning scale; ROC, Receiver Operating Characteristic; PAOFI, Patients Assessment of Own Functioning Inventory; PCA, Perceived Cognitive Abilities; PCI, Perceived Cognitive Impairment.
Using the alternate definition of low CF as total PAOFI scores >2SD above the mean of a healthy group of women (N = 63, mean age 52 years old), 13 , 30 a cut‐off score of ≥40.5 on the CCS would identify cases of self‐reported low CF with 74.5% sensitivity and 71.3% specificity (AUC = 0.791, 95% CI: 0.721–0.860, Table 3). Cut‐off scores using this alternate definition of a case of low CF for the EORTC‐CF, FACT‐COG PCI‐18, PCI‐20 and PCA are also provided in Table 3.
The CCS was readministered to a subsample (N = 24) between two and 6 weeks after the initial assessment. On retest the mean CCS was 23.8 (SD = 28.8). Cronbach's α = 0.72 was rated as acceptable (α = 0.72, 95%CI: 0.35–0.88, p = 0.002).
4. DISCUSSION
This study identifies cut‐off scores on a range of commonly used self‐report measures of CF in a large sample (N = 294) of cancer patients with mixed cancer diagnoses. On our single item measure of cognitive symptoms after chemotherapy a large proportion (77.6%) of participants reported some degree of change or deterioration in cognitive functioning since starting chemotherapy. Using the PAOFI as a reference standard, indicated a subset of these survivors (N = 106, 36.1%) may require further follow‐up and support.
Our results suggest a cut‐off of ≥28.5 on the CCS has good sensitivity and reasonable specificity for identifying patients with low self‐reported CF who may require further follow‐up. The AUC for the CCS was in the acceptable range, 32 , 33 suggesting the CCS was able to discriminate between cases of perceived low and normal cognition. Based on the ROC AUC, the CCS and EORTC‐CF had comparable discriminative ability (both in the acceptable range). 32 , 33 Comparing EORTC‐CF cut‐off of ≤75.0 to the CCS cut‐off of ≥28.5, the EORTC‐CF had better sensitivity, but worse specificity for identifying patients with low CF. This is possibly due to the discrete nature of item responses on the two items of EORTC‐CF, meaning only 7 total score values are possible (i.e. 0, 16.7, 33.3, 50.0, 66.7, 83.3, 100). However, the low specificity means patients reporting minor symptoms, that is reporting ‘a little bit’ of difficulty with concentration and memory, may be flagged for unwarranted follow‐up. We identified cut‐off scores on the FACT‐COG PCI18 (≤55.1) and PCI20 (≤59.5) similar to those published previously in sample of breast cancer survivors. 13 Our analysis indicated cut‐off scores with greater sensitivity that may be more applicable for cancer patients with other tumour types.
The test‐retest reliability of the CCS was in the acceptable range. The mean CCS score decreased on retest suggesting less self‐reported impairment. Based on longitudinal studies using standardised neuropsychological measures, we may have expected more cognitive symptoms with cumulative doses of chemotherapy. 18 , 34 However, other studies employing self‐report of cognitive symptoms suggest some reduction in symptom reporting occurs over time. 8 Response shift may account for reductions in symptom reporting, with patients' own ‘internal reference of normality’ changing during treatment, such that what they perceive as ‘abnormal’ in the early phase of treatment, may later in treatment come to be viewed as acceptable. 35 Alternatively cognitive symptoms may improve with time, and persist only in a subset of people with cancer. However, the test re‐test analysis was conducted in a small sub‐population of 24 participants and these results need to be interpreted with caution, as other factors such as a short interval between test and retest may contribute to this finding.
4.1. Clinical implications
Brief screening measures that are easy to use and interpret supports integration into routine clinical practice, 15 and offers an economical and efficient way to identify potential difficulties. 36 Multidimensional questionnaires, such as the PAOFI and FACT‐COG, should theoretically provide additional information compared to a single item questionnaire, but their testing time is approximately 15 min or more, and additional scoring time is required, thereby limiting their acceptability as screening tools. Scoring the EORTC‐CF also requires a calculation before interpretation, while the single item visual analogue CCS used here requires no further scoring, and therefore may have greater clinical utility. Furthermore, while different language translations of more comprehensive measure are available (e.g. FACT‐COG is available in 25 languages), single item measures may be better suited to settings and patient populations with diverse cultural and linguistic backgrounds, or lower health literacy. 37 Ease of interpretation of results would support integration of screening for cognitive symptoms into clinical care, 15 but it does require delineation of ‘next steps’ and appropriate action to be taken. 38 Screening for cognitive symptoms has the potential to validate patients' concerns about their cognition and offers an opportunity to provide further assessment, either more detailed assessment of self‐reported cognitive symptoms or neuropsychological assessment, or support for managing cognitive symptoms. As such, clinicians and researchers should consider which measure, and cut‐off, best suits their setting. 7
4.2. Study limitations
Several limitations are worth noting. Our sample was relatively well‐educated and so our reported cut‐off score may not apply to patients with less education. In addition, the reference standard used here, and elsewhere, 13 was a self‐reported measure. While the PAOFI has previously been used diagnostically in the context of identifying cases of HIV‐associated neurocognitive disorders, 28 , 29 further study investigating the discriminability of the CSS compared to standardised neuropsychological assessments is warranted to validate the cut‐off score. Furthermore, the requirement to dichotomise for identifying ‘case’ of low CF or normal CF for ROC analyses may lose granular characterisation of subtle changes people with cancer experience to their cognition that is otherwise captured by scoring the full range of responses. 39 Finally, the utility of CCS may also be improved by changing the wording to cover the full range of cognitive change (i.e., worse – better CF), and further consideration of changes since diagnosis and/or other treatments. For example, ‘How much change have you noticed in your thinking, memory or concentration since your cancer diagnosis or cancer treatment’ with response options ranging from −100 (much worse), 0 (no change), to +100 (much better), may support use with patients who receive treatments other than chemotherapy. Further, use of a verbal only scale, rather than a visual analogue may support future implementation and use in the context of telehealth. Such modifications will require validation. This study also has several strengths including validation of a single item measure of cognitive symptoms in a sample with reasonably balanced gender representation. It is also the largest sample to date with inclusion of tumour types other than breast and different staging.
Establishing simple screening measures for cognitive symptoms with clinical utility during and after cancer treatment is a critical first step to ensuring patients get adequate support. Our study found a single item question about the degree of cognitive change noticed since chemotherapy has acceptable discrimination between patients with normal and low CF when compared against a well validated and comprehensive self‐report measure of cognitive symptoms used in diagnostic settings. Further work is needed, but our results suggest single item measures of self‐reported cognitive symptoms have merit and offer sensitivity and specificity on par with more common comprehensive measures of cognitive symptoms.
ACKNOWLEDGEMENTS
We would like to acknowledge the assistance of the following team members who collected the data for the original study: Anne Warby, Alexander Elder, Itay Keshet, Rhonda Devine, Calina Ouliaris, Shirley Lundie‐Jenkins, Felicity Leslie and Corrinne Renton.
Open access publishing facilitated by The University of Sydney, as part of the Wiley ‐ The University of Sydney agreement via the Council of Australian University Librarians.
Fardell JE, Bray V, Bell ML, Rabe B, Dhillon H, Vardy JL. Screening for cognitive symptoms among cancer patients during chemotherapy: sensitivity and specificity of a single item self‐report cognitive change score. Psychooncology. 2022;31(8):1294‐1301. 10.1002/pon.5928
DATA AVAILABILITY STATEMENT
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.
REFERENCES
- 1. Boykoff N, Moieni M, Subramanian SK. Confronting chemobrain: an in‐depth look at survivors' reports of impact on work, social networks, and health care response. J Cancer Surviv. 2009;3(4):223. 10.1007/s11764-009-0098-x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 2. Janelsins MC, Kesler SR, Ahles TA, Morrow GR. Prevalence, mechanisms, and management of cancer‐related cognitive impairment. Int Rev Psychiatry. 2014;26(1):102‐113. 10.3109/09540261.2013.864260 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 3. Lange M, Joly F, Vardy J, et al. Cancer‐related cognitive impairment: an update on state of the art, detection, and management strategies in cancer survivors. Ann Oncol. 2019;30(12):1925‐1940. 10.1093/annonc/mdz410 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 4. Kesler S, Janelsins M, Koovakkattu D, et al. Reduced hippocampal volume and verbal memory performance associated with interleukin‐6 and tumor necrosis factor‐alpha levels in chemotherapy‐treated breast cancer survivors. Brain Behav Immun. 2013;30(Suppl l):S109‐S116. 10.1016/j.bbi.2012.05.017 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 5. Dumas JA, Makarewicz J, Schaubhut GJ, et al. Chemotherapy altered brain functional connectivity in women with breast cancer: a pilot study. Brain Imaging Behav. 2013;7(4):524‐532. 10.1007/s11682-013-9244-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 6. Pullens MJ, De Vries J, Roukema JA. Subjective cognitive dysfunction in breast cancer patients: a systematic review. Psycho‐Oncol. 2010;19(11):1127‐1138. 10.1002/pon.1673 [DOI] [PubMed] [Google Scholar]
- 7. Costa DSJ, Fardell JE. Why are objective and perceived cognitive function weakly correlated in patients with cancer? J Clin Oncol. 2019;37(14):1154‐1158. 10.1200/jco.18.02363 [DOI] [PubMed] [Google Scholar]
- 8. Janelsins MC, Heckler CE, Peppone LJ, et al. Cognitive complaints in survivors of breast cancer after chemotherapy compared with age‐matched controls: an analysis from a nationwide, multicenter, prospective longitudinal study. J Clin Oncol. 2017;35(5):506‐514. 10.1200/jco.2016.68.5826 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 9. Dhillon HM, Tannock IF, Pond GR, Renton C, Rourke SB, Vardy JL. Perceived cognitive impairment in people with colorectal cancer who do and do not receive chemotherapy. J Cancer Surviv. 2018;12(2):178‐185. 10.1007/s11764-017-0656-6 [DOI] [PubMed] [Google Scholar]
- 10. Amidi A, Christensen S, Mehlsen M, Jensen AB, Pedersen AD, Zachariae R. Long‐term subjective cognitive functioning following adjuvant systemic treatment: 7‐9 years follow‐up of a nationwide cohort of women treated for primary breast cancer. Br J Cancer. 2015;113(5):794‐801. 10.1038/bjc.2015.243 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 11. Henneghan AM, Van Dyk K, Kaufmann T, et al. Measuring self‐reported cancer‐related cognitive impairment: recommendations from the cancer neuroscience initiative working group. J Natl Cancer Inst. 2021;113(12):1625‐1633. 10.1093/jnci/djab027 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 12. Bray VJ, Dhillon HM, Vardy JL. Systematic review of self‐reported cognitive function in cancer patients following chemotherapy treatment. J Cancer Surviv. 2018;12(4):537‐559. 10.1007/s11764-018-0692-x [DOI] [PubMed] [Google Scholar]
- 13. Dyk KV, Crespi CM, Petersen L, Ganz PA. Identifying cancer‐related cognitive impairment using the FACT‐cog perceived cognitive impairment. JNCI Cancer Spectr. 2019;4(1). 10.1093/jncics/pkz099 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 14. Wagner L, Sweet J, Butt Z, Lai J, Cella D. Measuring patient self‐reported cognitive function: development of the functional assessment of cancer therapy ‐ cognitive function instrument. J Support Oncol. 2009;7(6):W32‐W39. [Google Scholar]
- 15. Rothrock NE, Cook KF, O’Connor M, Cella D, Smith AW, Yount SE. Establishing clinically‐relevant terms and severity thresholds for patient‐reported outcomes measurement information system® (PROMIS®) measures of physical function, cognitive function, and sleep disturbance in people with cancer using standard setting. Qual Life Res. 2019;28(12):3355‐3362. 10.1007/s11136-019-02261-2 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 16. Terwee CB, Peipert JD, Chapman R, et al. Minimal important change (MIC): a conceptual clarification and systematic review of MIC estimates of PROMIS measures. Qual Life Res. 2021;30(10):1‐26. 10.1007/s11136-021-02925-y [DOI] [PMC free article] [PubMed] [Google Scholar]
- 17. Mercieca‐Bebber R, Costa DS, Norman R, et al. The EORTC quality of life questionnaire for cancer patients (QLQ‐C30): Australian general population reference values. Med J Aust. 2019;210(11):499‐506. 10.5694/mja2.50207 [DOI] [PubMed] [Google Scholar]
- 18. vanDam FS, Schagen SB, Muller MJ, et al. Impairment of cognitive function in women receiving adjuvant treatment for high‐risk breast cancer: high‐dose versus standard‐dose chemotherapy. J Natl Cancer Inst. 1998;90(3):210‐218. 10.1093/jnci/90.3.210 [DOI] [PubMed] [Google Scholar]
- 19. Jacobs SR, Jacobsen PB, Booth‐Jones M, Wagner LI, Anasetti C. Evaluation of the functional assessment of cancer therapy cognitive scale with hematopoietic stem cell transplant patients. J Pain Symptom Manag. 2007;33(1):13‐23. 10.1016/j.jpainsymman.2006.06.011 [DOI] [PubMed] [Google Scholar]
- 20. Vardy JL, Liew A, Warby A, et al. On the receiving end: have patient perceptions of the side‐effects of cancer chemotherapy changed since the twentieth century? Support Care Cancer. 2022;30(4):3503‐3512. 10.1007/s00520-022-06804-1 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 21. Chelune GJ, Heaton RK, Lehman RA. Neuropsychological and personality correlates of patients’ complaints of disability. Advances in Clinical Neuropsychology. Springer; 1986:95‐126. [Google Scholar]
- 22. Bell MJ, Terhorst L, Bender CM. Psychometric analysis of the patient assessment of own functioning inventory in women with breast cancer. J Nurs Meas. 2013;21(2):320‐334. 10.1891/1061-3749.21.2.320 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 23. Van Dyk K, Ganz PA, Ercoli L, Petersen L, Crespi CM. Measuring cognitive complaints in breast cancer survivors: psychometric properties of the patient's assessment of own functioning inventory. Support Care Cancer. 2016;24(12):4939‐4949. 10.1007/s00520-016-3352-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 24. Koch V, Wagner LI, Green HJ. Assessing neurocognitive symptoms in cancer patients and controls: psychometric properties of the FACT‐Cog3. Curr Psychol. 2021. 10.1007/s12144-021-02088-6 [DOI] [Google Scholar]
- 25. Costa DSJ, Loh V, Birney DP, et al. The structure of the FACT‐cog v3 in cancer patients, students, and older adults. J Pain Symptom Manag. 2018;55(4):1173‐1178. 10.1016/j.jpainsymman.2017.12.486 [DOI] [PubMed] [Google Scholar]
- 26. Aaronson NK, Ahmedzai S, Bergman B, et al. The European organization for research and treatment of Cancer QLQ‐C30: a quality‐of‐life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85(5):365‐376. 10.1093/jnci/85.5.365 [DOI] [PubMed] [Google Scholar]
- 27. Fayers PMAN, Bjordal K, Groenvold M, Curran D, Bottomley A, on behalf of the EORTC. Quality of Life Group. The EORTC QLQ‐C30 Scoring Manual. European Organisation for Research and Treatment of Cancer. 3rd ed. 2001. [Google Scholar]
- 28. Blackstone K, Moore D, Heaton R, et al. Diagnosing symptomatic HIV‐associated neurocognitive disorders: self‐report versus performance‐based assessment of everyday functioning. J Int Neuropsychological Soc. 2012;18(1):79‐88. 10.1017/s135561771100141x [DOI] [PMC free article] [PubMed] [Google Scholar]
- 29. Woods SP, Rippeth JD, Frol AB, et al. Interrater reliability of clinical ratings and neurocognitive diagnoses in HIV. J Clin Exp Neuropsychol. 2004;26(6):759‐778. 10.1080/13803390490509565 [DOI] [PubMed] [Google Scholar]
- 30. Ganz PA, Kwan L, Castellon SA, et al. Cognitive complaints after breast cancer treatments: examining the relationship with neuropsychological test performance. J Natl Cancer Inst. 2013;105(11):791‐801. 10.1093/jnci/djt073 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 31. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ. 2011;2:53‐55. 10.5116/ijme.4dfb.8dfd [DOI] [PMC free article] [PubMed] [Google Scholar]
- 32. Mandrekar JN. Receiver operating characteristic curve in diagnostic test assessment. J Thorac Oncol. 2010;5(9):1315‐1316. 10.1097/jto.0b013e3181ec173d [DOI] [PubMed] [Google Scholar]
- 33. Hosmer DW, Jr , Lemeshow S, Sturdivant RX. Applied Logistic Regression. John Wiley & Sons; 2013. [Google Scholar]
- 34. Wieneke M, Dienst E. Neuropsychological assessment of cognitive functioning following chemotherapy for breast cancer. Psycho‐oncol. 1995;4(1):61‐66. 10.1002/pon.2960040108 [DOI] [Google Scholar]
- 35. Osoba D. Health‐related quality of life and cancer clinical trials. Ther Adv Med Oncol. 2011;3(2):57‐71. 10.1177/1758834010395342 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 36. Cutillo A, O'Hea E, Person S, Lessard D, Harralson T, Boudreaux E. The distress thermometer: cutoff points and clinical use. Oncol Nurs Forum. 2017;44(3):329‐336. 10.1188/17.onf.329-336 [DOI] [PMC free article] [PubMed] [Google Scholar]
- 37. Scanlon B, Brough M, Wyld D, Durham J. Equity across the cancer care continuum for culturally and linguistically diverse migrants living in Australia: a scoping review. Glob Health. 2021;17(1):87. 10.1186/s12992-021-00737-w [DOI] [PMC free article] [PubMed] [Google Scholar]
- 38. Mitchell AJ. Screening for cancer‐related distress: when is implementation successful and when is it unsuccessful? Acta Oncol. 2013;52(2):216‐224. 10.3109/0284186x.2012.745949 [DOI] [PubMed] [Google Scholar]
- 39. Van Dyk K, Ganz PA, Ercoli L, Petersen L, Crespi CM. Measuring cognitive complaints in breast cancer survivors: psychometric properties of the patient’s assessment of own functioning inventory. Support Care Cancer. 2016;24(12):4939‐4949. 10.1007/s00520-016-3352-6 [DOI] [PMC free article] [PubMed] [Google Scholar]
Associated Data
This section collects any data citations, data availability statements, or supplementary materials included in this article.
Data Availability Statement
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.